Make a New Account



Forget your username or password?


Blog Roll
Who's My Legislator?

Virginia Blogs
All Things Education
Augusta Free Press
Crew of 42
Dixie Pig
Greater Greater Washington
Loudoun Progress
Ox Road South Blog
Pilot on Politics" blog
Power for the People
Richmond Sunlight
RVA Politics
Shad Plank
Vivian Paige

Find more about Weather in Arlington, VA
Click for weather forecast

Advanced Search

Nate Silver Ranks the Best and Worst Pollsters of 2012

by: lowkell

Mon Nov 12, 2012 at 20:33:52 PM EST

natesilverpollsteraccuracyDoes Nate Silver ever sleep, or even rest? Apparently not. Even after a grueling election season in which he (rightfully) became a superstar, and also somehow found time to write and promote a new book (The Signal and the Noise, which I'm currently reading), he's still hard at work. The latest example: this past Saturday night (as I said, he never rests), Silver posted his rankings of "the 90 polling firms that conducted at least one likely voter poll in the final three weeks of the campaign." You can see those on the "flip."

But first, here are Nate Silver's rankings of the "roughly two dozen polling firms that issued at least five surveys in the final three weeks of the campaign" (click on the image to "embiggen!"). As you can see, Gallup wasn't just bad, it was abysmally, disastrously off course - both inaccurate AND wildly biased (towards Republicans). Sadly for Gallup, this election cycle wasn't a fluke; as Nate Silver points out, this debacle makes "three poor elections in a row" for Gallup. So much for being a respected pollster anymore, at least by anyone who knows anything (e.g., not many pundits or much of the corporate media).

Other than Gallup, other godawful pollsters were - shocker - Rasmussen (both inaccurate and heavily Republican biased), Mason-Dixon (inaccurate and Republican biased), and American Research Group (ditto). The best pollsters, interestingly, were ones that much of the "mainstream" media (e.g., the Washington Post, whose own polling unit was mediocre at best) disdain, such as internet pollsters Google Consumer Surveys, RAND Corporation, Ipsos/Reuters, and Angus Reid. Also doing very well were live phone (including cell) pollsters IBD/TIPP, Mellman, CNN/Opinion Research, Quinnipiac, and Marist. Robodialer PPP, which for whatever reason the "mainstream media" insists on calling a "Democratic" firm (do they identify Rasmussen as wildly biased towards Republicans?), turned out to actually have a 1.6-point Republican bias. Pretty funny. Anyway, check out the list of pollsters with a minimum of 5 polls in the last 21 days of the election, and when you're done, head to the "flip" for the longer list, and a few more comments by yours truly (including on the utterly abysmal, laughable, Roanoke College polling unit).

lowkell :: Nate Silver Ranks the Best and Worst Pollsters of 2012
A few more thoughts on these pollster rankings. First, note that almost all the polls had a bias - not towards Democrats, as the crazy "unskewed" folks would have us believe, but towards Republicans! I mean, it's not even close; there is a significant pro-Republican bias, even among supposedly pro-Democratic polling firms like PPP. So much for THAT theory!

Second, to quote Nate Silver, "I'm not as certain about the future for automated telephone polls." Instead, surprisingly, it's the internet pollsters of all people who seem to be doing better in this day and age. Fascinating.

Third, one Virginia pollster, Roanoke College, ranks close to the bottom in terms of both average error (they only did one poll in the final 21 days of the election, but man was it a bad one - off by 8.1 points!) and also bias (8.1 points in the direction of Republicans). Ouch. Sadly, this finding isn't an aberration, as Roanoke College's polling efforts have been wildly erratic, with all kinds of "internal" problems, and just a horrible track record in general. In the future, until proven otherwise, these polls should be ignored (along with Christopher Newport University's, not to mention any poll that's conducted over several weeks, as a lot of these college polls are...). Anyway, enjoy the full list, and thanks to Nate Silver for his continued hard (and impressive) work!



Tags: , (All Tags)
Print Friendly View Send As Email

I'd love to see this for political pundits and ANALysts as well (0.00 / 0)
That would be hilarious.

Follow me on Twitter. Follow Blue Virginia on Facebook and Twitter.

Gallup's Daily Tracking Poll (0.00 / 0)
was a disaster, but the Republicans did a good job of promoting it, while our side didn't as much as they should have to discredit it.  

It surprises me that the Marist/NBC News polls had a republican bias.

Only 1 point bias for Marist, not too bad. (0.00 / 0)
As for Gallup, I think it's way more than the Republicans, it's the lazy, mindless mainstream media (MMM?) that keeps citing Gallup just because it has the name. Other than that, as Nate Silver points out, Gallup has sucked for air in three straight cycles. #FAIL

Follow me on Twitter. Follow Blue Virginia on Facebook and Twitter.

[ Parent ]
Gallup (0.00 / 0)
That polling outfit has produced garbage results for years, but  you are absolutely correct. The corporate media treats Gallup as gospel because it has a name easily recognized by a mass audience. As for Roanoke College polling, it is a total joke. I always discount whatever "results" they find.

[ Parent ]
What about Mellman? (0.00 / 0)
Nate shows them as R 0.0+ bias, which, one would think, means no bias? Looking at the over-all level of bias tilted Republican, especially among the most widely quoted polls, I am reminded of a certain stage in my teenage life when some authority figure told me what I was supposed to do/believe, and I immediately did the opposite, assuming that the authority figure was full of it. Maybe that's how we should look at these polls, and immediately assume that the opposite of whatever they say is true. So, maybe they did serve some useful purpose after all.

Not that accurate though (0.00 / 0)
The main thing is "how accurate were they?"  If a shop was only .25% off, but always off by .25% toward Republicans, then it is biased - but highly accurate.

[ Parent ]


Donate to Blue Virginia

The purpose of Blue Virginia is to cover Virginia politics from a progressive and Democratic perspective. This is a group blog and a community blog. We invite everyone to comment here, but please be aware that profanity, personal attacks, bigotry, insults, rudeness, frequent unsupported or off-point statements, "trolling" (NOTE: that includes outright lies, whether about climate science, or what other people said, or whatever), and "troll ratings abuse" (e.g., "troll" rating someone simply because you disagree with their argument) are not permitted and, if continued, will lead to banning. For more on trolling, see the Daily Kos FAQs. Also note that diaries may be deleted if they do not contain at least 2 solid paragraphs of original text; if not, please use the comments section of a relevant diary. For more on writing diaries, click here. Thanks, and enjoy!

P.S. You can contact us at and you can subscribe to Lowell's Twitter feed here. If you'd like to subscribe to Miles Grant's Twitter feed, click here. For Teacherken, click here. For Kindler, click here.

P.P.S. To see the Blue Virginia archive, please click here. To see the Raising Kaine archive, please click here. To see the Blue Commonwealth archive, please click here.

RSS Feed

Subscribe to Blue Virginia - Front Page

Powered by: SoapBlox