Shelf Life

“Wherever you look at human judgements, you are likely to find noise”

Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein May 18, 2021
Heard on:
HTML EMBED:
COPY
Human judgements can be noisy. In a new book, Nobel Prize winner Daniel Kahneman discusses strategies to reduce noise in organizations. General Photographic Agency/Getty Images
Shelf Life

“Wherever you look at human judgements, you are likely to find noise”

Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein May 18, 2021
Heard on:
Human judgements can be noisy. In a new book, Nobel Prize winner Daniel Kahneman discusses strategies to reduce noise in organizations. General Photographic Agency/Getty Images
HTML EMBED:
COPY

We know that decisions involving human judgement — hiring, corporate strategy, criminal sentencing — can be tainted by bias. But it’s also true that something as simple as the weather or the results of a football game can affect the decisions people make. That’s called noise. Nobel Prize winner Daniel Kahneman, along with co-authors Olivier Sibony and Cass R. Sunstein, explore that idea in a new book called “Noise: A Flaw in Human Judgement.”

Click the audio player above to hear Kahneman’s conversation with “Marketplace” host Kai Ryssdal about the economic implications of noise and what organizations can do to combat it. The following is an excerpt from the book.

To understand error in judgment, we must understand both bias and noise. Sometimes, as we will see, noise is the more important problem. But in public conversations about human error and in organizations all over the world, noise is rarely recognized. Bias is the star of the show. Noise is a bit player, usually offstage. The topic of bias has been discussed in thousands of scientific articles and dozens of popular books, few of which even mention the issue of noise. This book is our attempt to redress the balance.

In real‑world decisions, the amount of noise is often scandalously high. Here are a few examples of the alarming amount of noise in situations in which accuracy matters:

Cover image courtesy of Little, Brown Spark

Medicine is noisy. Faced with the same patient, different doctors make different judgments about whether patients have skin cancer, breast cancer, heart disease, tuberculosis, pneumonia, depression, and a host of other conditions. Noise is especially high in psychiatry, where subjective judgment is obviously important. However, considerable noise is also found in areas where it might not be expected, such as in the reading of X‑rays.

Child custody decisions are noisy. Case managers in child protection agencies must assess whether children are at risk of abuse and, if so, whether to place them in foster care. The system is noisy, given that some managers are much more likely than others to send a child to foster care. Years later, more of the unlucky children who have been assigned to foster care by these heavy‑handed managers have poor life outcomes: higher delinquency rates, higher teen birth rates, and lower earnings.

Forecasts are noisy. Professional forecasters offer highly variable predictions about likely sales of a new product, likely growth in the unemployment rate, the likelihood of bankruptcy for troubled companies, and just about everything else. Not only do they disagree with each other, but they also disagree with themselves. For example, when the same software developers were asked on two separate days to estimate the completion time for the same task, the hours they projected differed by 71%, on average.

Asylum decisions are noisy. Whether an asylum seeker will be admitted into the United States depends on something like a lottery. A study of cases that were randomly allotted to different judges found that one judge admitted 5% of applicants, while another admitted 88%. The title of the study says it all: “Refugee Roulette.” (We are going to see a lot of roulette.)

Personnel decisions are noisy. Interviewers of job candidates make widely different assessments of the same people. Performance ratings of the same employees are also highly variable and depend more on the person doing the assessment than on the performance being assessed.

Bail decisions are noisy. Whether an accused person will be granted bail or instead sent to jail pending trial depends partly on the identity of the judge who ends up hearing the case. Some judges are far more lenient than others. Judges also differ markedly in their assessment of which defendants present the highest risk of flight or reoffending.

Forensic science is noisy. We have been trained to think of fingerprint identification as infallible. But fingerprint examiners sometimes differ in deciding whether a print found at a crime scene matches that of a suspect. Not only do experts disagree, but the same experts sometimes make inconsistent decisions when presented with the same print on different occasions. Similar variability has been documented in other forensic science disciplines, even DNA analysis.

Decisions to grant patents are noisy. The authors of a leading study on patent applications emphasize the noise involved: “Whether the patent office grants or rejects a patent is significantly related to the happenstance of which examiner is assigned the application.” This variability is obviously troublesome from the standpoint of equity. 

All these noisy situations are the tip of a large iceberg. Wherever you look at human judgments, you are likely to find noise. To improve the quality of our judgments, we need to overcome noise as well as bias.

Excerpted from “Noise.” Copyright © 2021 by Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein. Used with permission of Little, Brown Spark, an imprint of Little, Brown and Company. New York, NY. All rights reserved.

We’re here to help you navigate this changed world and economy.

Our mission at Marketplace is to raise the economic intelligence of the country. It’s a tough task, but it’s never been more important.

In the past year, we’ve seen record unemployment, stimulus bills, and reddit users influencing the stock market. Marketplace helps you understand it all, will fact-based, approachable, and unbiased reporting.

Generous support from listeners and readers is what powers our nonprofit news—and your donation today will help provide this essential service. For just $5/month, you can sustain independent journalism that keeps you and thousands of others informed.