How do you know if a poll is right?

People watch the debate between U.S. President Barack Obama and Republican presidential candidate and former Massachusetts Gov. Mitt Romney as they are seen on television screens at the Electric Avenue electronics store on October 16, 2012 in Miami, Fla.

Each week, Gallup's editor-in-chief Frank Newport joins us to discuss how people are feeling about the news by looking at what Gallup's polls have uncovered from our collective subconscious.  But in the past week, Gallup's polls themselves have been in the news.

Nate Silver, who runs the election forecasting site FiveThirtyEight Blog, published a critique over the weekend of Gallup's polling. He questioned the size of the swings in opinion Gallup sometimes measures.

"Apart from Gallup's final poll not having been especially accurate in recent years," Silver wrote, "it has often been a wild ride to get there. Their polls, for whatever reason, have often found implausibly large swings in the race."

Newport disputes Silver's accusations of inaccuracy.

"If we didn't have swings, we wouldn't track," says Newport. "We've gone to a seven-day rolling average now in part to kind of iron out some of those short-term variations. But trust me, this is the sixth election I've been involved in, and I just think there are ups and downs...particularly in the last month of an election."

The beauty of polling, though, is after the election there's a winner and a loser, and you either called it right -- or you didn't.

About the author

Frank Newport, Ph.D., is the editor-in-chief at Gallup and appears regularly on Marketplace.
Log in to post8 Comments

"you either called it right -- or you didn't. "

OK, now that their bias has yielded a "you didn't," can we chuck Gallup and put Nate in?

Well, maybe one more interview, with "where did your methodology go wrong?"

When referring to New York Times polls in 2008, Mr. Newport referred to them as "his [Nate Silver's] polls." In 2008, Nate Silver was blogging independently; he was not yet writing for the New York Times. Furthermore, Nate Silver does not conduct polls of his own, nor is he affiliated with any polls that the New York Times may conduct. His election forecasting model simply takes data that is already out there. His numbers are not biased, unlike Mr. Newport's comments.

Mr. Newport says that his poll was the same as Silver's in 2008, but that seems wrong. In 2008, Gallup projected an 11 point win for Obama and Silver's meta-analysis pointed to a 6 point win. The final result was Obama by 7.

I have long been mystified about why you would use Gallup after their leader publicly declared he would "Win elections for Republicans." Why does ANYONE take them seriously? Looking through Gallup's fine print polling methodology, I discovered that they are using voter I. D. data from their March 3-6 2011 survey to weight their Likely Voter polls. Interestingly, that poll showed voter I. D. at 29% Republican, 39% Independent, & 29% Democrat. Their "leaner" numbers were 45% Republican and 43% Democrat. Using those stats, they then factor in turnout rates in which Republican always have higher turnout. It is no wonder their poll is so skewed.

If they were using their own current data, voter I. D. would be 28% Republican, 38% Independent, and 32% Democrat, with "leaners" going 43% Republican and 50% Democrat. Applying these percentages would bring their poll in line with all the others - makes you question their motives.

What is with the softball questions. You asked the Gallup Editor in Chief about the criticism from Nate Silver. He says, no it isn't true that our polls are off, and you leave it at that. How about some follow up. Either they are way off or not. Just because you interview Mr. Newport every week doesn't mean he gets a free pass.

Gallup is so outdated because the still call landline phone for their polls. I don't know anyone under 35 that has a landline phone as a major contact. Their pollsters at outside event very rarely approach people of color.

They actually are calling cell phones but are purposefully using and EIGHTEEN MONTH OLD "Likely voter model". See my other post here...

With Generous Support From...