Wording Matters: Why I don’t trust Issue Polling

Since Governor Scott Walker and the Wisconsin Senate had a big uproar over unions of government employees, and that uproar has spread to states like Ohio, there’s been a great deal of issue polling on unions and collective bargaining. I tend to ignore all of it, just as I ignore most issue polling. I’ve gotten comments that this is a failure of the site in fact, that I don’t hit these things more.

I have a standard answer for why I tend to ignore issue polling, though: the results are volatile and easily manipulated, either accidentally or intentionally.

When one conducts a horserace poll for an election, it’s easy to do that fairly: simply list the candidate names in a random order. But issue polling isn’t so simple. There are just so many ways that issues can be presented that influence the results. That’s the point discussed today at National Journal’s Hotline On Call, and I agree with the headline when it says that “wording matters.”

The article specifically addresses a poll by Quinnipiac suggesting voters dislike Governor John Kasich’s plans to follow in Walker’s footsteps and place limits on these government employee unions, but the margin is unclear. Adam Geller predicts that the word “rights” will change the results, and sure enough, the Q poll went as predicted: With the word rights, the unions are favored among registered voters by a strong 54-35 margin (MoE 2.6). However that margin goes from blowout to close when the word rights is removed: unions are then only favored 48-41 (MoE 2.6). That’s a 12 point swing, with 6 points coming from each side of the question.

So, Quinnpiac seems to confirm Adam Geller’s theory, but the truth is that these polling instabilities happen with any controversial (that is: interesting to poll) issue. Do we say unborn baby, or fetus? Do we ask about terrorist sleeper cells or racial profiling? If just one word can swing a poll 12 points, just imagine what entirely different wordings might cause. And of course, who’s to say what wording is neutral or not? Unlike a horserace poll, which can be calibrated by actual election results, we have no way of getting a baseline for an issue poll.

So that’s why I don’t cover issue polls here. I just don’t see any value in tests of competing wordings of the same issue.