Bad polling discredits good polling

“Lies, damn lies, and statistics.” “You can use a poll to prove anything.” We all hear the lines like these, which reflect the popular view of statistical analysis and opinion polling. Nobody believes it. It’s all made up, says the conventional wisdom, or at least doctored to a degree that it can’t be trusted.

It’s not actually so bad at all, but it’s hard to make that case when people like the Daily Beast’s John Avlon undermine the field with examples like these.

Avlon discredits polling when he writes an article such as the one above. He’s ready, willing, and able to attempt an attack on the political right, but reluctant to give key facts about the poll. What was the methodology? What was the sampling pool? What are the definitions of the terms used in the poll?

That last question is an important one but was not even addressed by Avlon. Normally it doesn’t matter at this site, because Unlikely Voter mostly deals in horserace election polling, where there are no undefined terms, but just people. But a poll which discusses a concept like “the Antichrist” without defining that term is useless to an extreme.

“The Antichrist” is of course referencing certain Christian views on the end of the world. Yes, I said certain. One thing a person learns when delving into these matters is that not everyone agrees. Under the umbrella of Christian one sees a broad range of views of the end of the world. You’ve got postmillennials, premillennials, amillennials, and probably a few variations of each that you have to have studied to differentiate. These differences matter, too. Not all Christians believe that the end of the world will be structured as the Left Behind novels describe.

To each of these groups, the concept of Barack Obama as “the Antichrist” will have a different meaning, or even no meaning at all. And of course, if you ask a follower of a different or no religion, the question has even less meaning. An Anti-Christ is a concept that only makes sense in opposition to a belief in a Christ that already came.

But no, Avlon and Harris sweep all this under the rug as they ask a question off the cuff in a way that makes no sense at all, and whose results are entirely meaningless.

Scientific polling only makes sense if the questions themselves are clear and unambiguous. If the question itself adds uncertainty to the poll, to the point where different people are in their minds answering different questions, then there’s no reasonable way to interpret the results.

After seeing this question, I didn’t even look at the rest. I think they have to be just as bad. Oh wait, here’s another one: did Obama do “many of the things Hitler did?” Well, Hitler discouraged smoking, built roads, and stepped up diplomacy with Iran. Once again, a poorly defined question leads to a nonsense result.

Bad polls like this need to be drug out into the street and shot, so that the good polls can live in peace and harmony.