Nate Silver is a statistician that gained some fame after developing a system for predicting the performance of baseball players. That field is now known as sabermetrics, and is the basis for Moneyball – both the book and the movie. In 2008, Silver turned his attention to politics and started a blog called Five Thirty Eight (which is the total number of electoral votes), where he developed a system that, using all polling data available, correctly called not only the presidential election, but how 49 out of the 50 states voted, and how all of the 35 senate races turned out. (Indiana was the one state he missed, which went for Obama by 1%.)
For the 2012 election, the algorithms were even better: he got the all 50 states completely right.
On the morning of the election, his blog — which was bought by the New York Times in 2010 — predicted, with a confidence level of 90.9%, that Obama would win the election. To anyone following this and other polling data, like Princeton’s Election Consortium‘s, the results on election night amounted to a complete lack of surprise. But of course, all pundits on the news coverage of the election ignored these predictions and instead spent the entire night making their own, much less rigorous forecasts, based on personal knowledge and gut instincts instead of mountains of polling data and algorithms. Then again, how else would a dozen news networks fill as many hours of programming, except by pretending that the future is unknowable?
The always insightful xkcd hit the nail on the head:
The comic’s hover text reads:
As of this writing, the only thing that’s ‘razor-thin’ or ‘too close to call’ is the gap between the consensus poll forecast and the result.
(Updated November 9th to reflect that Florida was called for Obama.)