"One shock away from catastrophe": Understanding the existential risks to humanity

The choices we make today can shape the future in profound ways — helping humanity thrive, or in the worst case, threatening its survival.

Around the world, researchers are studying what are known as existential risks: dangers that could threaten civilization itself. These range from nuclear weapons and engineered pandemics to climate change and rapidly advancing artificial intelligence.

In a new series, “Marketplace Morning Report” host David Brancaccio is checking in with some of the institutions focused on these threats. For the first installment, he spoke with physician, epidemiologist, and professor Steve Luby, who helps lead the Stanford Existential Risks Initiative. The following is an edited transcript of their conversation.

David Brancaccio: I mean, at first blush, I want to think about these existential risks as very low probability, but, of course, high potential impact. But that's not really the right description here. I mean, the probability may not be low enough for some of these things.

Steve Luby: If you think about that, we are on hair trigger launch on warning of nuclear weapons. If you start thinking about this along a decades-long timeline, all of a sudden these probabilities aren't very small, because they do add up. We've got a lot of things that we are one shock away from catastrophe.

Brancaccio: Just to rest for a moment on one of these grim topics: a biological disaster that's human-engineered. I mean, that's often bio weapons, but it could be just biomedical research gone wrong.

Luby: One of the big problems here is it's not just large, well-funded states that can do this kind of research. Because the biological tools have become increasingly powerful, lower cost, and increasingly democratized. It means that there are more and more folks out there who are capable of doing serious mischief.

Brancaccio: And I assume a newer one: artificial intelligence? You know, back in the Eisenhower administration, that administration got behind the International Atomic Energy Agency, you know, international cooperation on arms control. Think we need something like that for AI?

Luby: I think we definitely need to be talking, but I do think a difference with AI is that the real leadership in AI is coming from private firms. It's not coming from governments. I mean, nuclear was very much a state-sponsored technology. The challenge here is that the incentive for those folks who are developing these AI models is to grab market share, so they want to work quickly, and they are not incentivized to prioritize safety.

Brancaccio: You worry about significant hazards from AI? This isn't science fiction stuff?

Luby: No, absolutely. One of the things we see is you don't need to have great developments in AI for it to have politically disastrous consequences. And similarly, if we think about what you can do with AI technology when you combine it with these biological tools in the wrong hands, we see that with the current and the rapidly developing tools, we are making destructive weaponry more and more potent and more and more available.