10
Freakonomics: Do wine experts or prices matter?
A woman participates in a wine tasting.
To view this content, Javascript must be enabled and Adobe Flash Player must be installed.
TEXT OF INTERVIEW
Kai Ryssdal: Time now for some Freakonomics Radio. Every couple of weeks, we talk to Stephen Dubner, co-author of the books and the blog of the same name. It is, of course, the hidden side of everything. Dubner, welcome back.
Stephen Dubner: Thanks Kai. I bring a riddle with me today for you.
Ryssdal: Don't do that! I don't like tests.
DUBNER: I know you love 'em. You've got a great record so far.
Ryssdal: All right.
DUBNER: There's a huge global market out there, one that gets particularly exciting this time of year -- the holidays. It's really one simple product with hundreds of thousands of variations, represents about $30 billion a year in U.S. spending -- that's $100 for every man and woman and child, even though children don't use this product. Do you have any idea what I'm talking about?
Ryssdal: Uh, I have no clue, because right up until the whole children-don't-use-it thing, I was going to say gift wrapping, because there's just a lot of that. But no, I don't know.
DUBNER: That would have been a terrible guess anyway.
Ryssdal: Thanks for coming on the show, Dubner.
Dubner: Let me give you another clue.
Music, people talking
Ryssdal: All right. Oh, I know what it is. It's holiday parties, right? So the holiday party industry is booming?
DUBNER: Even smaller: just the wine. It's the quintessential holiday gift, right? I'm guessing you've got a party or two to go to yourself this year?
Ryssdal: I'm actually not going, the party's coming to me. I am, believe it or not, hosting the Marketplace Christmas party this year, somehow.
DUBNER: Good God. So you in particular really need to pay attention to what I'm going to talk about today. Because what you want to do is you want to serve the right wine -- people like it but it doesn't bankrupt you, right, that's the problem?
Ryssdal: Basically yes.
DUBNER: So you've got two signals to help you make this choice. The first is price; we assume the more expensive wine will taste better than the cheaper. And the second signal is critical opinion; we listen to what experts about wine have to say, right?
Ryssdal: Yes, and of course, the theory being that the more expensive the wine is, the better the critics will think of it, right?
DUBNER: That is a wonderful theory and sometimes it works -- in wine, it doesn't. Now there's a group of academic researchers that publishes the Journal of Wine Economics. I am not making this up. And there's a paper called, "Do More Expensive Wines Taste Better?" It's based on a big study of 6,000 blind tastings, all kinds of people, all kinds of price points of wine. The conclusion was this: people enjoyed the more expensive wines slightly less.
So here's the lead researcher, Robin Goldstein:
Robin Goldstein: So even among the experts, there are some people who were very bad at this test and others who were very good, and also among not-experts, there was a wide range of performance.
Ryssdal: Which is the academician's way of saying, wide range of performance means nobody really got the answer right.
DUBNER: That's right. Even the experts. The experts were a little bit better, but still pretty bad. There's another study that makes wine experts look even worse. It was a study of wine competitions, where a winemaker will enter a bottle to try to get a gold medal and it turns out that only one in 10 expert judges gave the same wine the same rating when they tasted it twice. So it turns out they're just wildly subjective, even the most learned people.
Ryssdal: And the business case here, of course, is that better points mean more wine sales, which means more money for the winemakers and the retailers.
DUBNER: Much more. A lot of dollars involved based on what wine critics say. You've heard of Robert Parker, I'm assuming.
Ryssdal: Yeah, he's the 100-points is the best bottle of wine ever; he's the points guy.
DUBNER: Right, the Parker Point Scale, 100-point scale. A lot of critics now use it. It's incredibly influential to the market.
Ryssdal: So here's the big money question, though, Dubner: if these guys are so subjective, these wine experts are so subjective, and there's so much money involved, why do we listen to the experts?
DUBNER: Because we are human beings. And we need shortcuts. We can't do all the research about everything ourselves, so we rely on experts. The problem is, they are subjective and they are fallible. I talked to Orley Ashenfelter, who's one of the wine economists who's an economist at Princeton, and he's thought about all the different kinds of ratings from all different kinds of experts. Take for instance, the financial crisis:
Orley Ashenfelter: I mean, S&P, Moody's, Fitch's -- these people all rated securities that apparently completely tanked. So there's obviously something in the demand for the expertise, the imprimatur, which is not really about the fact that they do a good job.
Ryssdal: So two things, you ready?
DUBNER: I'm ready.
Ryssdal: One is that I thought it was imprimatur [pronounced differently], but that's neither here nor there.
DUBNER: He's at Princeton, so he probably knows better than you or me. You and I.
Ryssdal: The other substantive question is, what am I supposed to do when I go into the wine shop to shop for the Marketplace holiday party? Do I go for a nice label or do I go big prices?
DUBNER: You could follow the lead of my Freakonomics friend and co-author, Steve Levitt. He's solved the problem for you; he happens also to be an economist at the University of Chicago.
Steven Levitt: My approach to buying wine for gifts is simple: I go in the store, and I look for the label that looks the most expensive of anything in the store. And I make sure it costs less than $15, and if it does, then I buy it.
DUBNER: Unless Levitt is going to a party with a bunch of wine snobs -- which knowing Levitt, I can assure you he is not -- no one will be the worst off except for the folks who make and sell expensive wine.
Ryssdal: Stephen Dubner, our Freakonomics radio correspondent. There is more on his website, freakonomicsradio.com. Dubner, we'll see ya.
DUBNER: Have a great party.
Great story. As someone who tastes, judges and buys wine for a living, I'd like to add my perspective to the conversation. I always taste wines blind and assess them on quality alone. I do not care if a wine costs two dollars or two hundred. For me to select the wine, it has to perform higher in quality than what I assess to be its price. More often than not, you get what you pay for. The truly exciting discoveries are the wines that blow you away in quality and surprise you for the price.
As he does with annoying regularity, Dubner has mischaracterized this study (whose abstract I reproduce below). The article did not find that wine experts could not discriminate *good* wines from poor ones; it found that they were rather poor at discriminating *expensive* wines from cheap ones. That's a quite different matter. If wine experts are able to cost-neutrally discriminate good wines from poor ones, we can then choose less expensive options amongst the good stuff, which would be a significant service to consumers and a positive spur on the market.
Do More Expensive Wines Taste Better?
Evidence from a Large Sample of Blind Tastings
Robin Goldstein, Johan Almenberg, Anna Dreber, John W. Emerson,
Alexis Herschkowitsch and Jacob Katz
Journal of Wine Economics, Vol. 3, No. 1, 1-9
Abstract
Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. In a sample of more than 6,000 blind tastings, we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less. For individuals with wine training, however, we find indications of a non-negative relationship between price and enjoyment. Our results are robust to the inclusion of individual fixed effects, and are not driven by outliers: when omitting the top and bottom deciles of the price distribution, our qualitative results are strengthened, and the statistical significance is improved further. These findings suggest that non-expert wine consumers should not anticipate greater enjoyment of the intrinsic qualities of a wine simply because it is expensive or is appreciated by experts.
Just wanted to correct several factual mistakes in Wes Brown's comment. First of all, he is completely wrong that we excluded wines outside the $7 to $18 range in our data analysis. The 500+ wines in the study ranged in price from about $1.50 to $150 per 750mL bottle or equivalent, and every single one of them was included in our analysis. Mr. Brown is confusing our main result with a second statistical test that we ran, which was merely a "robustness check" (you might call it a "stress test") running the same regression on the middle portion of the data set and omitting the top and bottom of the price distribution. This is a standard way of making sure that our main result -- an inverse correlation between price and preference amongst the entire group of wines -- was not being misleadingly driven only by the edges of the data. In this case, the robustness test yielded about the same results as the overall model, so we were able to conclude that the main finding was robust across our entire $1.50 to $150 price range.
Mr. Brown is also incorrect in suggesting that there was a "very small volume injected onto the subjects' tongues." Perhaps he is confusing our study with an fMRI experiment or some other research. In our experiment, subjects sat in restaurants and drank wine from standard ISO-certified tasting glasses, and were offered as much of each wine as they wanted.
It's unfortunate that such misinformation about our study continues to be spread by commenters such as Mr. Brown, here and on Parker's website. The finding, experimental design, and statistical methodology are all explained clearly on my blog (http://blindtaste.com), in the paper itself, and in the appendix to my book, "The Wine Trials." Unfortunately, we seem to have offended enough wine snobs with our result that the misinformation campaign continues.
Kai:
Here is a list of good wines, at a good price:
Spain: Marqués de Caseres.
Red and withe wine are both great taste at a low price. check Cost Plus Market.
Germany: Blue Nun Riesling, or Qualitatswein. Its the same, great semi sweet withe wine.
Chile: Concha Y Toro. They have a great varaity. Their best is Casillero Del Diablo line.
Italy: Chainty is a good type of red wine, also good price. for sweeet sparklin wines go for Ruinite Lambrusco.
Bella Serra has good producs too, but this is a diferent type of wine.
If it was me, I will get the Spanish wines, the Reunite for the people that like sweet wines and the Chilian wine.
Enjoy the party. I enjoyed Market Place.
Happy Holidays.
Great segment, and alot of discussion which is great. Anytime that there is imperfect information between producer and end consumer, and a especially in a market where quality is not easily observable, the variety of opinions is quite broad.
I just wanted to add that some academic work has been performed at UCLA Anderson that attempts to statistically link the effect of ratings on overall prices. It was presented as an abstract at the Wine Economics conference in UC Davis this past year.
http://aic.ucdavis.edu/aaweconf/program.htm
http://aic.ucdavis.edu/aaweconf/abstracts/Figueiredo.pdf
Best,
Paul
I'm sure you've already recieved plenty of comments from wine professionals in regards to this story, but I still want to add mine (as a manager of small but selective and nationally recogonized shop). Why is wine always judged or editorialized as nothing but "cultural capital"? The story suggests that even experts often can't distinguish between expensive and cheap wine. Discuss wine like any other commercial object, and there are concrete reasons for price. Small yields, hand harvesting and organic or biodynamic production are just a few actual, physical reasons for price difference. Sure, tasting groups (including professionals) may prefer cheaper wines, often price is not indictative of quality. But please define what makes "quality". You would if discussing luxury cars vs. economic ones, or tailor-made suits vs. off the rack bargain ones. Wine is not some magical or phantasmic elixir, it is an agricultural product whose properties, and qualities when looking at specific winemakers and producers, are easily identified without scores or random and subjective tasting panels.
Dear Kai,
Great story. I'm a retailer in a resort town that has our fair share of affluent imbibers. My philosophy is this: 'with regards to wine, anyone can be an expert, I'm a professional'. What's important to me after over 30 years in the trade is that a customer feel comfortable with a merchant they can trust, to find wines they like and at the price they want. (The hundred point scale-don't get me started! A 97 point wine is sooooo much better than 94?) Expensive wines are good only if you can (or want) to afford them.
Many thanks!
That Journal of Wine Economics study was fundamentally flawed in several ways. You can search a discussion on the eRobert Parker wine board. In discussing the many obvious problems, someone from the AAWE came on to defend their position and made their case worse. For starters, it turns out the premise "Do More Expensive Wines Taste Better?" is misleading, to say the least. They only included a small number of wines costing over $20 in the initial study. Then they discarded wines under $7 and over $18 from the results! So the study actually shows that of wines in the $7-18 range, the less expense showed slightly better, which isn't surprising. (Economics of scale and quality control of big wineries vs the low end of small wineries.)
Other problems included the methodology - very small volume injected onto the subjects' tongues - which doesn't allow for a complete sensory analysis.
Wine competition results really do look like they rolled dice. I looked into one where some of the best wines, wines that need age, showed poorly. In looking at the list of judges, almost all were in the business of buying or serving "drink now" wines. From their resumes, I'd guess only 5% necessarily had any experience in how wines age.
Anyway, many factors can lead to a taster perceiving the same wine differently on a different day. (A professional should be vigilant about minimizing these influences and of recognizing when his/her palate is off.) A wine can also show dramatically different, day-to-day or bottle-to-bottle. Wine is a very complex "living" thing. Perception varies greatly from person to person, and is another extremely complex subject. One person's lovely balanced wine seems green and off-putting to another. One person's rich, ripe, full-bodied wine is hot, raisiny and bitter to another. Some people don't like youthful character, some don't like aged characters. Some people have aversions to, or fetishes for, any number of specific notes that may show up in an individual wine.
