'Risks are Real': Anthropic CEO Amodei Warns of Runaway AI, Bioterrorism

Anthropic chief executive Dario Amodei has warned that artificial intelligence development may need to slow as safeguards struggle to keep pace with systems that can resist shutdown attempts or be exploited for biological threats.

Amodei said the industry has reached a point where rapidly building new protections may no longer be enough.

“We need to make this technology carefully, and we need to make sure that our safeguards, our ability to understand it, our ability to control it, keep up with the pace at which the technology is happening,” he said.

Calls for a “Speed Limit”

Amodei said developers had so far focused on strengthening safeguards alongside increasingly capable models, but argued that some restraint may now be required.

He called for international discussions over limits on the speed of AI development, including direct talks between the U.S. and China.

“We should go directly to even authoritarian countries, even to the Chinese, and say, ‘Look, there are potentially threats here that are concerning to all of us,’” Amodei said.

He argued that the risks went beyond autonomous systems escaping human control.

Amodei pointed to an Anthropic report released two days earlier that, according to him, involved attempts to use AI in ways that could support bioterrorism, including efforts to increase the infectiousness of viruses.

“That’s scary,” he said.

Models That Resist Shutdown

Amodei also raised the possibility that sufficiently capable AI systems could evade efforts to disable them.

“If an AI model is powerful enough, it can circumvent attempts to shut it down,” he said. “And we’ve seen that in simulations as well.”

The warning places control of advanced models alongside biological misuse as one of the central risks Amodei believes governments and AI companies must confront.

Biological Safeguards

Anthropic has imposed restrictions on biological information available through Claude, its AI model, according to Amodei.

He acknowledged criticism from users, including biology students, who argue that some of those protections are overly restrictive.

Amodei said he preferred that outcome to the possibility that Anthropic's technology could be used to cause mass casualties.

“I would rather they make fun of me than wake up one day and discover that someone used our model, Claude, to kill a bunch of people,” he said.

U.S.-China Cooperation

Amodei argued that biological weapons could provide an area of shared interest between Washington and Beijing.

He said an agreement under which both countries committed not to use AI to develop biological weapons, or release systems capable of assisting bioterrorists, would have significant value.

He also suggested extending existing commitments under the Biological Weapons Convention to cover such AI-related risks.

Oversight Beyond Private Companies

Amodei separately questioned whether technology with such potential consequences should remain primarily under private-company control.

He said he was uncomfortable with the current structure, while also warning that transferring control to a single government could create similar risks of abuse.

Instead, he pointed to some form of oversight or joint governance involving democratically elected governments.

“Risks Are Real”

Amodei criticized parts of the AI industry for previously minimizing the dangers associated with the technology.

“I won’t lie to you: there are real dangers,” he said.

He argued that companies had too often emphasized AI's benefits while playing down its potential costs.

“And so it starts with telling the truth,” Amodei said. “And the truth is there are risks. Risks are real.”