Silver reverses course, attacks Rasmussen anyway

The New York Times’ Nate Silver is now going after Rasmussen Reports again. After the primaries he said Rasmussen was in his crosshairs for ducking out on a number of races by not polling primaries.

According to Silver’s own chart though, Rasmussen polled twice as often as the second place firm, and is still Silver’s primary target. Funny that.

In fact here’s the exact quote from the pre-New York Times Nate Silver from back in August:

Rasmussen — which polled the McCain-Hayworth primary eight times in a race where there was some disagreement among pollsters — was not willing to do so during the final four weeks of the campaign. Our pollster ratings are always becoming more sophisticated and we’re going to be looking at appropriate ways to punish pollsters who dodge putting their necks on the line.

And yet here we are, with Silver combining the Fox News/Pulse Opinion Research polls with the Rasmussen Reports polls, and the result is that Rasmussen is credited with 105 polls, a total number that no other pollster is even close to. Second place is Public Policy Polling, way down at 45. The rest of the list was notable for absent pollsters CNN/Time, Ipsos/Reuters, Greenberg Quinlan Rosner, and Suffolk University. I wonder why those were omitted.

Of course, even if we leave aside the validity of combining the Fox polls with the Rasmussen polls, does Silver give any credit for Rasmussen going out on a limb and making lots of polls of lots of races? Of course not.

Further, does Silver give any credit for making heavy polling on races which may have tightened late, as Democrats made their final pushes, such as West Virginia Senate, Pennsylvania Senate, or Illinois Governor? Of course not.

So in essence, Silver is punishing Rasmussen for making too many polls on too many interesting, volatile races, which is just the opposite of his last excuse for hitting Rasmussen. This is proof that Silver is just reaching for any excuse of the day to blame his current target of hate in the polling world, now that Strategic Vision moved on from public political polling. Silver can’t even keep consistent his reasons for hating Rasmussen Reports.

Further, though, the aggregation of Pulse Opinion Research and Rasmussen Reports, just because both were founded by Scott Rasmussen, is pretty unfair. It’s clear that Fox gets to dictate its own parameters for its own polls, because Fox didn’t have the same results or Margins of Error even that Rasmussen Reports used.

Yes, I know as a matter of fact that Fox News/POR polls typically had a margin of error of 3. Rasmussen Reports by default had a margin of error of 4.5, which then dropped to 4 for important races, late. When you’ve written up as many of these polls as I have, you know this without even checking.

The MoE difference alone is proof that Fox was dictating terms. If Silver wanted to be taken seriously he’d do an analysis proving that the two polls are conducted the same way, with the same screening and movement over time. He didn’t though. He just lumps the two together, throws on a double asterisk, and moves on with a handwave of saying they are “essentially identical,” but without telling us how we know that.

I have my own post in the works looking at just why so many polls were wrong in favor of Republicans this year. I think it’s a systemic thing and in particular will make SurveyUSA look very good. SUSA came in second place in Silver’s ratings, and that doesn’t surprise me.

Another interesting calculation that Silver didn’t do would be to compare the actual errors in the polls with their listed Margins of Error. Rasmussen I believe is able to get that high volume in part by consistently hitting MoEs in the 4s, rather than the 3s that a PPP does, or the 2.5 that CNN and Time did. That sacrifice in MoE for volume would inherently result in a higher average error. But we’ll never know if that combined 5.8 makes Rasmussen look bad, Fox News really bad, or some combination of the two. Nate Silver didn’t do the right analysis, and he combined the data in a misleading way. He actually destroyed information rather than creating it.

Nate Silver has proven himself to be technically proficient when he wants to be by producing sports statistical works like PECOTA (which he has now turned over to Baseball Prospectus, presumably to focus on politics). However the kind of handwaving that Silver does, time and again, to push his political agenda is not up to that standard. That is why I’m inclined to file his slipshod commentaries against certain pollsters under dishonesty, not simple incompetence.