Anthropic loosens safety pledge to compete with its AI peers

Anthropic is loosening its safety standards around when to deploy new, more advanced versions of its artificial intelligence. The startup said it risks being left behind by competitors. 

A number of AI safety advocates have been warning for some time now that the industry is moving too fast. And in announcing its changes, Anthropic said federal policy is prioritizing growth over safety. 

In AI’s incipiency of 2021, some OpenAI executives left to form what they called a safer alternative — Anthropic, said Owen Daniels, associate director of analysis at Georgetown University’s Center for Security and Emerging Technology.

“Anthropic has long been one of the frontier labs that's been most acutely focused on ensuring that its systems and applications are used in kind of a safe, responsible and ethical way,” Daniels said.

Core to Anthropic’s safety effort had been a pledge called the responsible scaling policy, said Sarah Myers West, co-executive director of the AI Now Institute. 

“If they believe that the capabilities of these tools outstrip their ability to control them and ensure that they’re safe, they would stop building them,” she said of the policy.

Anthropic said it was trying to stave off a race to a bottom with that policy, but it admitted it failed to persuade other companies to take its stricter safety approach. 

The change it announced yesterday means they may keep building more advanced models, even if they’re riskier.

Brent Thill, a tech industry analyst at Jefferies, said AI companies are getting more ambitious. 

“There’s incredible pressure from OpenAI, from Microsoft, from many of the top technology companies,” Thill said. “AI is going from a consumer-based technology to now, inside the enterprise. The race is on.”

To run financial systems, companies need more advanced AI than what’s necessary to find the best latte. And as companies spend hundreds of billions of dollars to build those tools, Daniels said they need to start making money, too.

“There is a tension that's difficult for these companies to navigate in, deploying the latest versions of models in ways that are economically useful and can be beneficial to their bottom lines,” he said.

They’re balancing that with the risks AI may pose.

An Anthropic spokesperson said the company is now committing to more transparency to publish detailed reports about the risks and capabilities of its AI.