Two recent polls on abortion provide yet another demonstration of lack of adequate care in designing and reporting on polling results. Conservatives are furious about a Washington Post poll for its wording of questions concerning recent efforts to restrict abortions.
The objection? That the question on abortion clinics biases the poll outcome.
Obviously, the Post didn’t have the courage to ask the question honestly. Had they, it would have looked something like this:
Q: Several states are considering or recently have passed legislation increasing health and safety regulations and oversight that might make it more difficult for abortion clinics to operate there. Overall is this something you support or oppose?
I disagree that this is the “honest” way of asking the question. The goal of these laws is to restrict abortion access under the cloak of spurious health-and-safety rhetoric. In this respect, they’re a continuation of the access-restriction strategies adopted by the pro-life movement since almost immediately after Roe v. Wade. Even putting aside these issues, there’s no “might” going on here: the laws make it more difficult for abortion clinics to operate.
Nonetheless, the Post‘s failure to test the anti-abortion frame does limit the information provided the poll. Testing both kinds of arguments would have provided far more information about the state of public opinion and how different frames shape attitudes toward this sort of legislation.
The poor design of the poll — and its reporting — extends to other abortion questions. It phrases the “20-week” question in terms of the timing of “restrictions.” The laws in question are pretty straightforward abortion bans, although they (IIRC) include variants on the “mother’s life” exception without which even very, very conservative populations won’t stomach abortion bans. The Post does report the literal wording of the question, but it is clear from their interviews that a lot of people don’t understand the nuance here:
Advocates on both sides said the polling suggests their side is winning the battle over public opinion.
“Polls like today’s prove again and again that championing the brutal dismemberment of children is a losing proposition among Americans,” wrote Live Action’s Lila Rose, whose group makes videos of abortion clinics using undercover footage, in an e-mail. “Americans will continue to see that the horror of abortion doesn’t vanish before 20 weeks, or 12 weeks, or 8 weeks.”
Donna Crane vice president for policy at NARAL Pro-Choice America noted that while respondents might have given an “emotional response” in saying they would back limiting abortion at 20 weeks, “By an even starker margin Americans support a constitutional right to chose.”
A recent NBC/Wall Street Journal poll (PDF) does a better job with respect to these kinds of considerations. Here’s the question it asks on the 20-week bans:
The question has a much better explanation of what the laws actually do. The competing frames strike me as a bit problematic — weighing fetal “pain” against doctor-patient autonomy — but at least the poll provides them. Although we need to be careful about comparing polls, note that these numbers are much less favorable for the ban than the numbers are for “restricting” abortions after 20-weeks. Note also that “should ban” number isn’t all that different from the total for “abortion should be illegal” (11%) and “abortion should be illegal, with exceptions” (37%) question the poll asks earlier. Unfortunately, the NBC/WSJ poll doesn’t examine the access-restriction provisions.
None of these are huge deals, but I find it frustrating not only that major polls continue to provide less useful information than they otherwise might, but that reporting on them still doesn’t consistently align with the actual questions asked of respondents.
Interesting post. The question i found a bit more off hasn’t been mentioned as much:
“Do you think each state should make its own
laws on abortion, or do you think this should be decided for all states
on the basis of the U.S. Constitution?”, where the majority strongly favored a federal solution.
If the question were asks differently–or do you think this should be decided for all states by the federal government, and especially the Supreme Court?–then the answers would likely be different.