There are ledes, and there are ledes. This one at Daily Kos literally dropped my jaw:
I have just published a report by three statistics wizards showing, quite convincingly, that the weekly Research 2000 State of the Nation poll we ran the past year and a half was likely bunk.
The report appears to deliver, too.
I just skimmed it, but right away the plot of the week on week changes in the President’s popularity just about convinces me. If you look at the chart Kos provides, it’s so irregularly shaped it looks like a chart of a tiny sample. +1 and -1 are huge, but 0 is well below. But then the Y axis reveals we’re talking about dozens and dozens of trials. Gallup in a similarly sized sample comes up with a clear bell curve, as we might expect from a poll which will combine small incremental changes of public opinion with random fluctuations of sampling.
Getting past the pretty pictures and on to the reading brings more compelling evidence. Not only were the R2k guys faking it, but they were sloppy:
Were the results in our little table a fluke? The R2K weekly polls report 778 M-F pairs. For their favorable ratings (Fav), the even-odd property matched 776 times. For unfavorable (Unf) there were 777 matches.
Once I get past the sputtering, I have to say that I would think that people faking polls would at least put a little random element into them to prevent basic mistakes like this. From this alone, I have no doubt in my mind that Research 2000 defrauded Daily Kos.
House of Representatives Swingometer
Electoral College Swingometer
“Once I get past the sputtering, I have to say that I would think that people faking polls would at least put a little random element into them to prevent basic mistakes like this.”
Read toward the end of the report and they’ll talk about how it was actually the opposite case: R2K was so busy trying to “randomly” move Obama’s approval numbers up and down each week, that they forgot to occasionally NOT adjust it at all. FTA: “it has long been known that when people write down imagined random sequences they typically avoid repetition, i.e. show too few changes of zero.”
Yeah that’s what I mean. They should have done week to week adjustments with a random element instead of doing it by hand, which they obviously did, since they got far too few zeroes.