Jeff's Lunchbreak: Reflections on the Polls

A lot's been made since the election about how wrong the pollsters got it. And while most of the polls were wrong, I think the magnitude of their error is being overhyped. This election has also reinforced my respect for FiveThirtyEight.

On the day before the election, I made a comment about the polls. Here's what I wrote:

On an election related note, while I've been following 538 like probably everybody else, I've also started checking out some of the other election 'prophets', and 538 seems like an outlier this year. I hope so, but hope doesn't change reality. It'll be interesting to see how things play out tomorrow and which predictions were the most accurate.

I also included a table of different sites and their predictions:

Site	Chance Clinton Wins	Chance Dems Win Senate
538	69.4%	49.2%
PredictWise	89%	67%
Huffington Post	98.1%	66%
Princeton Election Consortium	>99%	79%

I also wrote that, "I'll still be biting my nails until the official results start coming in." Well, history now shows that I had reason to be biting my nails.

PredictWise, Huffington Post, and the Princeton Election Consortium were certainly way over confident in their predictions. But 538, while getting the prediction wrong, was more reasonable in their odds. Granted, their confidence in a Clinton win did increase slightly up until the election, but on Tuesday morning, they were still giving Clinton only a 71.4% chance of winning, and Trump a 28.6% chance of winning. And while that was certainly still in favor of Clinton, it wasn't overwhelmingly in her favor, and a win for Trump wasn't unthinkable. Just a few days before the election, Nate Silver emphasized this in the article, Trump Is Just A Normal Polling Error Behind Clinton.

538 also has a few post-mortem articles examining the polling errors this year, including The Polls Missed Trump. We Asked Pollsters Why. In the end, it looks like the polls missed by about 4 points, on average. Only being off by 4-points in gauging public opinion isn't a huge error, but in a close race like this one, it is enough to mean the difference between winning and losing.

They also quote Barbara Carvalho, who made a point about the number of quality polls being conducted.

Pollsters, and the media companies whose dwindling budgets have left them commissioning fewer polls, have to decide where to go from here. "Traditional methods are not in crisis, just expensive," said Barbara Carvalho of Marist College, whose final poll of the race showed Clinton leading by 1 point, in line with her current lead. "Few want to pay for scientific polling."

So, I disagree with the many commentators who have claimed that the polls were wildly off. They were off, but not by an unprecedented amount. The problem is that people put too much confidence in the polls, including some of the pollsters themselves. I would have thought that wouldn't have been an issue after the mid terms two years ago, but I can almost guarantee that people won't be over-confident based on the polls next time around.

Image Source: FiveThirtyEight, with a little Photoshopping by me