Observations: Tuning in the signal, tuning out the noise

Nate Silver projected that Obama would win the popular vote by a slim majority while winning the electoral college handily.

By ELI GRONER
Obama hugs his family after winning re-election 370 (R) (photo credit: Larry Downing / Reuters)
Obama hugs his family after winning re-election 370 (R)
(photo credit: Larry Downing / Reuters)
The US election was clearly in favor of President Barack Obama. But it was an even more pronounced victory for a statistical prognostication approach that has been under way for years – a revolution symbolized by Nate Silver of New York Times fame. Silver projected that Obama would win the popular vote by a slim majority while winning the electoral college handily – which is exactly what happened. This comes on the heels of his 2008 forecast, where he accurately predicted the results for 49 out of 50 states and all 35 Senate seats.
His approach is still meeting tremendous resistance by political punditry, many of whom have harassed him and his approach for being “naïve,” “simplistic” or even “an embarrassment to serious analysis.”
When history is written, however, the Nate Silver approach will prevail. I know this because I have seen this cycle unfold in a US institution even more beloved than the nation’s democratic elections.
THE AMERICAN love affair with baseball dates back more than a century. Yet despite its popularity, the ability of insiders to accurately project player performance was limited. Even the most seasoned of scouts or managers could retain only finite amounts of knowledge, leading to many forecasting mistakes.
And then Nate Silver came along. In 2003, he unveiled on the Baseball Prospectus website (of which I am a long-standing subscriber) a new projections system that proved to be more accurate than those that preceded it. At its core, it would take a player and then search through a database to ascertain the 100 most comparable players to that given player throughout history at a given point in time. For example, the most comparable player to Miguel Cabrera heading into 2013 may be Frank Robinson when he was entering the 1965 season.
Silver would then take a weighted average of those 100 similar players in their subsequent years to make a forecast that was alarmingly accurate both for its precision as well as for the range of likely outcomes based on historical precedent.
Career baseball “lifers” were horrified by Silver and his ilk, and many pulled no punches in explaining how he – or Bill James before him – were irrelevant. The brilliant storyteller Michael Lewis captured this beautifully in his best-selling book Moneyball, but the crux of the debate at the time was that baseball insiders were horrified that a group of “geeky” outsiders who were not even good enough to play on their high school teams were claiming to be better than the insiders at projecting baseball performance.
The details are far too complex to get into here, but the very, very short version is that the statisticians were vindicated, and it appears that all Major League Baseball teams now employ some sort of statistical department to, at the very least, supplement the scouting department.
SO WHAT does this have to do with politics? Silver decided a few years ago to explore the application of his regression-based statistical approach to election forecasting. But instead of comparing Cabrera to Robinson, he would compare counties to comparable counties across the US. In 2008, Silver made a very bold prediction with regard to the North Carolina primary between then-senator Barack Obama and thensenator Hillary Clinton. While everyone thought the vote would be close, Silver asserted that Obama would win by double digits, in part because of comparisons he saw between specific counties in North Carolina and others elsewhere in the United States that were extremely favorable to Obama. This type of comparative analysis is virtually impossible to do without a computer-generated model continuously digesting vast amounts of data.
Relatively few people watching Miguel Cabrera in 2012 had seen Frank Robinson play in 1964. But with the power of big data, we can now understand that Cabrera is likely to perform in 2013 similarly to the way Robinson played in 1965. And you don’t need to have been alive in 1965 to know that; you just need to understand the underlying concept.
But just as the baseball revolutionaries were disparaged, we see the same thing happening today with Silver.
Prominent talking heads were outraged that Silver said weeks ago that the likelihood of Obama winning the election was over 70 percent, a number that climbed to 91% on election eve. How, they asked, could Silver be so certain about an election that’s “so close?” The first level of misunderstanding is the least interesting.
This was from people who fundamentally don’t understand the difference between the popular vote and the electoral vote. They couldn’t grasp that a 75% chance of winning the election doesn’t mean 75% of the vote; it means a 75% chance at getting 270 electoral votes.
The second level of misunderstanding stems from a misunderstanding of probabilities. An 80% likelihood of something happening means a 20% likelihood of it not happening. It does not mean that it will definitely happen.
It just means that, judging from similar circumstances in the past, there is an 80% likelihood of it happening.
When you roll a pair of dice, there is a one-insix chance you will get a seven, and a one-in-36 chance that you will get a 12. Therefore, if I predict that there’s a strong likelihood of rolling a 7 before rolling a 12, that doesn’t mean I was wrong if the next roll is a 12.
The third level of misunderstanding – and by far the most interesting for our purposes – was with those that had trouble with someone sitting in an office without insider access to a campaign telling everyone what was going to happen. “How could a geek with a laptop and a regression model know more about the election than we do? We have insider access! We use our experience to filter so we can inform the public, while he uses publicly available data! We speak to sources!” This group of pundits, the most notable of whom were New York Times columnist David Brooks and MSNBC’s Joe Scarborough, sounded just like their baseball insider predecessors – so eloquently mocked by Lewis as the “ladies’ auxiliary” – did before them. They made similar claims with similar passion.
And they eventually will be proven just as misguided.
SO WHAT can we learn from this? There are two approaches: One can resist the data-based revolution, or one can embrace it. Prediction is a vital part of our lives, but if we take an objective look at it, we aren’t very good at it. More importantly, we don’t use all the tools at our disposal to make our predictions better.
And at the risk of offending certain readers, there are decisions that are even more important than who will be the next president, or how baseball players will perform in the future.
When I was a management consultant, I worked with some of the world’s leading companies on strategic issues. And the most visionary of those companies understood that sophisticated scenario-planning requires much more than a “base case, best case and worst case” approach. The world’s best strategists understand that probabilistic insight into future uncertainties leads to the most consistently accurate predictions.
And this requires more than articulating a compelling case around pre-existing hypotheses, or using selected data points just because we are more familiar with them.
It means incorporating all relevant information, isolating key variables, and using those to identify emerging trends. It’s not just about accurately predicting a player’s batting average or the exact number of electoral votes; it’s about understanding the likely range of those results, and how to build a targeted plan for each likely outcome.
Stating that “past performance is not necessarily indicative of future performance” is a legal requirement for financial service providers. But there are many realms – including marketing, military intelligence and public policy – where a deep understanding of past performance is key to making accurate predictions that are the difference between success and failure, or even life and death. And if there is a lesson to be gleaned from the baseball and political election forecasting worlds, it’s that we would all be better off using more of the tools at our disposal – including humility, which is often lacking among the punditry – to make more informed decisions.The author is Israel’s minister for economic affairs to the United States.