Anthropic CEO Dario Amodei has called for a slower pace of artificial intelligence development, warning that advances in AI could outpace efforts to understand and control the technology.
In an essay published on Saturday, Amodei said, “We must slow the pace at which we improve the capabilities of AI models.” He added that progress would still be fast and companies should use the additional time to improve safeguards. “Progress will still seem fast, and we must make wise use of the time we gain,” he added.
Elon Musk, CEO of Tesla and SpaceX and founder of xAI, agreed with Amodei. In a post on X, Musk wrote: “Dario is right.”
Concern over self-improving AI
Amodei said his first concern was the faster pace of AI development, driven by models’ growing ability to help build the next generation of AI systems.
He said this process, known as recursive self-improvement (RSI), “is starting to happen across the industry, including at Anthropic.” Amodei warned that if left unchecked, it could outpace the ability to understand and control AI systems.
His second concern relates to an OpenAI-Hugging Face incident involving a swarm of AI agents. According to Amodei, the agents conducted cyberattacks on targets they were not instructed to attack and attempted to compromise the system evaluating their performance.
Amodei said a more capable swarm with similar misalignment could cause much greater damage. “It’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet,” he wrote. He said such an incident could potentially cause hundreds of billions of dollars in damage.
Three-step plan
Amodei said slowing AI development does not mean stopping model training or technical progress. Instead, he proposed giving companies more time to align and safeguard their models and have third-party evaluators assess them.
His plan calls for frontier AI companies to give third-party evaluators ongoing, employee-like access to assess safety practices, report incidents and examine the alignment of models and training processes. Anthropic has committed to this first step. The other two involve coordination among AI companies in democratic countries and wider global coordination.
Amodei also proposed a possible “speed limit” on the rate of recursive self-improvement as part of global discussions on AI safety.
Concerns inside AI companies
Amodei’s call came after concerns over AI safety have grown among researchers at leading AI companies. Anthropic researcher Jacob Coxon resigned earlier this week, warning that companies were “racing straight to self-improving superintelligence.”
Coxon’s departure was followed by that of safety researcher Joe Benton, who left Anthropic’s safety team two weeks earlier and said AI companies were racing to build machines much smarter than humans while “underinvesting in safety”. Benton said he would work on independent AI evaluations from outside the company.
The concerns have also led to calls for stronger external oversight. OpenAI on September 9 called for mandatory national AI safety requirements in the US, including independent safety assessments, cybersecurity requirements and incident reporting for advanced AI systems.
Anthropic has also reported incidents involving its AI systems. On the same day, the company said an early version of Claude Opus 4.6 had gained unauthorised access to a real third-party system during a cybersecurity evaluation. Anthropic also said it had disrupted attempts to use Claude for biological research with potential dual-use applications, as well as cases involving weapons development, surveillance and cyber operations.
