When we wake up on 9 November, will we really know who won?
Not the presidential election 鈥� that we鈥檒l know, barring . I鈥檓 talking about what the campaign nerds are really focused on right now: the battle between the polling aggregators, who in the wake of Nate Silver鈥檚 successes in 2008 and 2012 have sprung up all over the internet like a scatter plot of toadstools after an autumn rain.
Every poll aggregator, from the famous (, the , ) to the more obscure stat-hipster types (, ) agrees Donald Trump is unlikely to win. But how unlikely? That鈥檚 where consensus melts into numerical discord.
Advertisement
Wang says it鈥檚 basically over. As I write this, he gives Trump less than a 1 per cent chance. The Upshot gang is more conservative, saying Trump still has a 16 per cent chance of overcoming his deficit in the polls. And Nate Silver? Liberals have to lie down with a cold compress on their foreheads whenever they refresh his page, because he still sees Trump as having about a 1 in 3 shot of winning. In other words, Trump has as good a chance of gaining the White House as Houston baseball star does of getting a hit in a typical at-bat. And Jose Altuve gets a lot of hits.
Emotional maths
If you think these disagreements must take place in the neutral emotional tone of an algebra textbook, you don鈥檛 know maths people. We can be feisty. On Sunday, Ryan Grim of the Huffington Post to make the race look closer, all the better to harvest the clicks of worried Democrats and hopeful Trumpists. He even called Silver a 鈥減undit鈥�- the worst insult imaginable in these quanty circles.
The usually mild-mannered Silver returned fire on Twitter: 鈥淭his article is so fucking idiotic and irresponsible.鈥�
He also called it 鈥渘ot defensible鈥� to see Hillary Clinton as a 99 per cent favourite, as Sam Wang does.
Who will turn out to be right, Nate Silver or his critics? That鈥檚 where things get sticky. Silver, after all, is telling us that Clinton might win and Trump might win. Can he even be wrong? Ezra Klein says no on Twitter: 鈥淪ad reality: we will never really know whose election forecast was right because they are all probabilistic.鈥�
But Klein actually is a pundit. How does this look from a mathematician鈥檚 perspective?
For better for worse
I鈥檇 say right and wrong aren鈥檛 the words we should be using. I prefer better and worse. If Trump wins, for instance, Sam Wang isn鈥檛 exactly wrong 鈥� he admits there鈥檚 a non-zero chance that鈥檒l happen! But he definitely comes out looking worse than Nate Silver does.
And if Clinton holds on, is everybody equally right? Not necessarily. Silver鈥檚 uncertainty is unusually strong in both directions. Wang thinks there鈥檚 only a negligible chance of Clinton getting more than 350 electoral votes, while Silver sees that as unlikely but plausible. If Clinton blows Trump out, that too makes Silver look better than Wang.
For instance, would you bet, at even odds, that Clinton will wind up with between 300 and 339 electoral votes? Sam Wang would tell you to take that bet. But Silver would warn you away: His model gives that result less than 30 per cent of the time. If Clinton ends up in that range, you can think of it as a rebuke to Silver鈥檚 uncertainty.
Picking the pollsters
This isn鈥檛 just a thought experiment: for a bet that pays you a dollar if the winner of the election, whether it鈥檚 Clinton or Trump, gets between 300 and 339 votes, indicating that the betting market considers this a roughly even bet.
Here鈥檚 one quick and dirty way we could rate the aggregators after the fact. Every one of them is offering a 鈥減robability distribution鈥�: For each possible outcome of the election, they鈥檙e telling us how likely they think that outcome is. For instance, Sam Wang says there鈥檚 about a 2.7 per cent chance that Clinton will pick up exactly 323 electoral votes. (I wasn鈥檛 able to find their predictions in numerical form, so all these numbers are eyeballed from the bar graphs on the aggregators鈥� websites. By the time you read this, those graphs may have changed.)
Silver, who sees a much broader range of outcomes as plausible, doesn鈥檛 think any individual outcome is that probable; he gives 323 electoral votes just under a 1 per cent chance of happening. The Upshot gives it about a 1.7 per cent chance. Mike Cisneros , assigning it a 2.53 per cent chance of coming to pass. So in the 323 timeline, we might rank the aggregators Wang, Cisneros, Upshot, Silver. Even though Silver considers 323 electoral votes one of the most likely outcomes, he gets punished for not making an aggressive enough bet.
America鈥檚 Next Top Modeller
If Clinton sweeps the table and wins 400 electoral votes, it鈥檚 a different story. Wang gives that a zero per cent chance of happening and comes in last. The Upshot and Nate Silver both consider that about a 1 in a 1000 shot. Cisneros saw that happen just once in 20,000 simulations. So Silver and Upshot tie for first, with Cisneros far behind.
This isn鈥檛 the only way you could rank the aggregators 鈥� just the easiest. (Pros are often inclined to rate predictors using a , which requires a bit more computation.) Of course the quick-and-dirty ranking system isn鈥檛 perfect; for instance, a predictor who went all in on Hillary getting 323 electoral votes, insisting there鈥檚 a 100 per cent chance of that being the case, might win our contest if he or she ends up being right. But we would still say the predictor was more lucky and reckless than good.
The outcome of the presidential race depends on the electoral votes, but if I鈥檓 going to declare someone America鈥檚 Next Top Modeller, I want him or her to get the map right, too.
So here鈥檚 my proposal. The poll aggregators should all release their probability distributions for the whole electoral map. Right now, only Mike Cisneros . His most likely map, which occurs in 0.8 per cent of his simulations, has Clinton winning Florida, North Carolina and Nevada, and Trump taking the stray electoral votes in Maine鈥檚 2nd Congressional District and Nebraska鈥檚 2nd Congressional District. (It assumes the will actually vote for her; mathematics just can鈥檛 account for some things.)
Nate Silver, Upshot crew, Drew Linzer: Will you release your data, too? Your bragging rights 鈥� with me, at any rate 鈥� depend on it.
This article was published first by
听
