Anthropic says it blocked possible attempts to use AI to develop bioweapons

Anthropic says it has blocked multiple accounts that used the company's artificial intelligence models in ways that could support the development of biological weapons.

In a report published Thursday, the AI company known for its Claude model said it considers biological misuse one of the "most serious risks" to artificial intelligence models.

The new report outlines five real-life case studies in which users "circumvented controls" that block users from specific regions and "engaged in other efforts to obfuscate the purpose of their research to evade our safeguards." The examples include possible gain-of-function research and involve both infectious diseases, such as bird flu, and novel venoms and toxins.

Looking over 30 days of activity, Anthropic said, it identified about 35 "distinct research efforts" with potentially concerning activity. However, it acknowledged that it could not be certain whether the actors "intended harm" or were conducting research for legitimate scientific purposes. Anthropic said individuals in the case studies were "working scientists" but did not identify the research institutions or countries where the activities took place.

Drug development is considered one of the most promising and optimistic applications of AI; leaders in the industry often point to its potential to cure cancer and other diseases. But the same features that make AI powerful for discovering new materials and medications to help people could also make it useful for developing bioweapons in the hands of bad actors.

Anthropic said that it wants its models to be useful for scientific research, but the latest models have become capable of much more complex work than earlier versions — and the potential for misuse rises with the potential benefits. It added that bad actors could try to use the beneficial biological uses of AI to "maintain a kind of 'plausible deniability'" about potentially harmful research.

"As our models become increasingly capable, approaching or exceeding expert performance at challenging scientific tasks, we expect their impact will only increase, both in beneficial and potentially harmful contexts," the Anthropic report said. "As Al models become more widely used, providers will continue to acquire threat-relevant visibility into real-world use that even governments and intergovernmental organizations lack."

The company said that recent models, including Claude Fable 5, have "stronger safeguards that restrict access to a wide range of dual-use biological research queries."

Anthropic said that there's evidence that the safeguards it has in place are working but that even more robust safety systems will be necessary as the AI models and possible threats become more sophisticated. The company also noted in the report that it disrupted efforts to use its models for surveillance operations, scams and the development of conventional weapons like drones and missiles.

The disclosure follows a growing chorus of warnings from AI employees who've recently left their companies over safety concerns. Some shared concerns that the technology is moving too quickly and will one day outpace humans' ability to control it — and called for governments to consider how to slow down development if things get out of hand.

"If you extrapolate into the future, the level of capabilities of these AIs … they could cause extreme havoc," Jacob Coxon, a former Anthropic employee, told CNN's Anderson Cooper on Wednesday.

In a resignation thread on X this week, the 27-year-old AI researcher said "the people building AI earnestly believe that it could kill us all by the end of the decade."

Speaking Wednesday, Coxon said it could hack critical infrastructure — or build "extinction-level bioweapons."

"There's a lot of ways that the AI could actuate itself in the world," he said.

In July, nearly 1,400 AI company employees signed an open letter urging the US government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And this week, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them.

For now, AI companies are largely policing themselves. The Trump administration has worked to undermine state AI regulations, and Congress has so far been unwilling to rein in the technology.

"We believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models," Anthropic said in a statement Wednesday.

CNN's Clare Duffy contributed to this report.

For more CNN news and newsletters create an account at CNN.com