The good and bad of AI voice cloning

This story was produced by our colleagues at the BBC.

Voice cloning is becoming easier, faster and more convincing. Artificial Intelligence makes it possible to change the age of an actor’s voice, translate words into any language, and replace a voice lost through illness. But it’s also increasingly being used by criminals to impersonate a loved one, extort money or compromise bank accounts. It’s changing how we communicate with each other and how we trust each other.

Respeecher is a Ukrainian AI company at the forefront of cloning speech that’s indistinguishable from a human voice. They’ve worked on Disney’s Obi-Wan Kenobi, cloning James Earl Jones’ voice for Darth Vader. And along with the Massachusetts Institute of Technology, the company won an Emmy Award for turning an actor’s voice into President Nixon in the short film, “In Event of Moon Disaster.”

Alex Serdiuk, Respeecher’s founder and CEO, said the process is much quicker.

“We had to get several hours of Nixon’s voice from the Nixon library. Now we can do the same work with having just several minutes of it.”

So how close to real time conversations are we? Serdiuk said we’re already there.

“We have a piece of technology that allows change in real time, meaning that there’s only a very small delay and that would make it quite smooth for a phone conversation.”

And Serdiuk said, there are other fields where the tech can change lives.

“We’ve applied our real time models to things like technology for health care. For example, laryngectomy patients who went through throat cancer and had their voice box removed.”

But Serdiuk adds a note of caution and said the technology needs to be controllable, so that bad players don’t ruin the market.

“Some irresponsible companies let people create anyone’s voice within seconds and with such good quality that it would trick people.”

A case in point is Jennifer DeStefano. Last year she told the U.S. Senate Judiciary Committee about a frightening anonymous phone call she thought was a call for help from her daughter. Panicking, she told a friend, who called 911, which warned Jennifer there was a scam going around involving voice cloning.

It’s a problem jurisdictions around the world are trying to grapple with, according to Professor Brent Mittelstadt from the Oxford Internet Institute. He specializes in AI ethics, law and policy.

“There are a number of proposals in the U.S. to basically fight robo-calls or scam calls or fraud calls that are using AI-generated voice profiles,” Mittelstadt said. “America is further ahead.”

And Europe is also taking action.

“The AI Act is a new regulatory framework that has recently been passed in the European Union,” he said. “There is political will whereas in the UK there is hesitancy at this stage in AI’s development.”

The UK does have a national reporting center for fraud and cybercrime, Action Fraud. The body says that in the first three months of this year, it had reports of voice cloning to compromise bank accounts, impersonate someone and request payment for a fictional emergency. There have also been cases of blackmail and impersonating a celebrity to encourage investing in a fraudulent scheme.

Noting these risks, Professor Mittelstadt isn’t impressed with the UK’s approach.

“To not regulate such transformative technology seems like a huge missed opportunity. And I think that’s why we see the EU and the U.S. acting now to actually regulate it,” he said.

Stories You Might Like

Now, software can replicate your boss’s voice and tell you what to do

What’s behind New York’s sky-high rents?

Voice-over artists allege an AI company cloned their voices in lawsuit

AI concerns stall contract negotiations between game companies and actors

Senators advance bipartisan effort to regulate deepfakes

TikTok is changing the voice of brands online