1

The problem with statistical models and forecasting

A portion of the cover for the book, "The Signal and The Noise: Why Most Predictions Fail — But Some Don't" by Nate Silver.

Image of The Signal and the Noise: Why So Many Predictions Fail — but Some Don't
Author: Nate Silver
Publisher: Penguin Press HC, The (2012)
Binding: Hardcover, 544 pages

If you really what to know what's going on in any even-remotely quantifiable discipline, nothing paints a clearer picture than the numbers, right? They end arguments. They can help forecast what's going to happen. They are, simply put, the facts. Not always.

In his new book, "The Signal and the Noise," author Nate Silver argues that numbers are part of the challenge of painting a clear picture of the American economy. The first example that Silver uses to illustrate his point is the housing crisis.

"You had these credit rating agencies who were taking these very novel types of debt instruments, mortgage-backed securities, and giving them very high ratings -- implying that they had almost no chance of defaulting when there really wasn't much reason to believe that if you actually sorted through the details of these," he says.

Writing this book, Silver says he learned that people have too much trust in statistical models and not enough cognizance to realize just how noisy the data can be. In his book, he stresses the importance of communicating uncertainty. He says that numbers can be misread and that the American economy is constantly changing -- making it difficult to forecast.

"It is very difficult, for example, to measure the effects of different types of stimulus. The reason is that you're going back and saying, 'Well when the economy was bad before, what happened to it?' But there were different economic policies attempted in every one of those recessions," he says. "So it's never a total experiment where you're just sitting there and observing things. You're trying to intervene in the economy and make it better."

Read more from Silver at his political forecasting blog FiveThirtyEight, which currently appears on the website of the New York Times. And listen to our full interview with him by clicking play on the audio player above.

About the author

Kai Ryssdal is the host and senior editor of Marketplace, public radio’s program on business and the economy.
Image of The Signal and the Noise: Why So Many Predictions Fail — but Some Don't
Author: Nate Silver
Publisher: Penguin Press HC, The (2012)
Binding: Hardcover, 544 pages
Log in to post1 Comment

The analogy is admittedly imperfect, but forecasting economic trends should be akin to forecasting the weather.

The input variables are dynamic and only somewhat predictable, intervening forces can alter the basic parameters on a whim, and a workable algorithm must be as long as the Bible but, that said, we predict the weather with an accuracy that shames the best economic prognosticators.

I suppose the missing components are an economics equivalent to satellite imagery and doppler radar, on the one hand and, on the other, weather doesn't have a bevy of "analysts" who predict one forecast and when that forecast doesn't come true, the stock market reacts with crazy, wild abandon.

My particular schooling was in the improbably-named field called "political science." If we can make a science out of politics, we surely ought to be able to fumble our way to a vague semblance of an economic divination.

With Generous Support From...