Anthropic CEO Dario Amodei has called for a slowdown in artificial intelligence development, warning that advances in artificial intelligence could outpace efforts to understand and control the technology.
In an essay published Saturday, Amodei said, “We must slow the pace at which we improve the capabilities of AI models.” He added that progress will continue to be rapid and companies should use the extra time to improve safety measures. “Progress will continue to appear rapid and we must use the time we have gained wisely,” he added.
Industry leaders, including his rival, OpenAI CEO Sam Altman, also agreed with Amodei. In his message, Altman said: “I agree with Dario that we need to move forward. This has been a major theme of discussions we have had at OpenAI in recent weeks.”
“The promise of independent evaluators with employee-level access is a great idea, and we will do the same. We’ll have more to share soon,” Altman added.
Elon Musk, CEO of Tesla and SpaceX and founder of xAI, also supported Amodei’s call. In a post on X, Musk wrote: “Dario is right.”
Concerns about self-improving AI
Amodei said he is primarily concerned about the faster pace of artificial intelligence development, driven by the growing ability of models to help build the next generation of artificial intelligence systems.
He said this process, known as recursive self-improvement (RSI), is “beginning to happen throughout the industry, including at Anthropic.” Amodei warned that if left unchecked, it could surpass the ability to understand and control AI systems.
His second concern relates to the OpenAI-Hugging Face incident, which involved a swarm of AI agents. According to Amodei, the agents carried out cyberattacks on targets they were not assigned to attack and tried to compromise the system while assessing their effectiveness.
Three-phase plan
Amodei said slowing AI development does not mean stopping model training or technological progress. Instead, he suggested giving companies more time to agree and protect their models and having them assessed by third-party evaluators.
His plan calls for cutting-edge AI companies to give third-party evaluators ongoing, employee-like access to evaluate security practices, report incidents, and check the consistency of training models and processes. Anthropic is committed to taking this first step. The other two involve coordination among AI companies in democracies and broader global coordination.
Problems within artificial intelligence companies
Amodei’s call comes as concerns about AI safety grow among researchers at leading artificial intelligence companies. Anthropology researcher Jacob Coxon resigned earlier this week, warning that companies were “directly pursuing self-improving superintelligence.”
Coxon’s departure followed that of security researcher Joe Benton, who left Anthropic’s security team two weeks earlier and said artificial intelligence companies were aiming to build machines that are much smarter than humans while “underinvesting in security.” Benton said he will work on independent assessments of AI outside the company.
Concerns have also led to calls for increased external oversight. OpenAI on September 9 called for mandatory national AI safety requirements in the US, including independent safety assessments, cybersecurity requirements and incident reporting for advanced AI systems.
Anthropic has also reported incidents related to its artificial intelligence systems. On the same day, the company reported that an early version of Claude Opus 4.6 gained unauthorized access to a real third-party system during a cybersecurity assessment. Anthropic also said it had stopped attempts to use Claude for biological research with potential dual-use applications, as well as cases involving weapons development, surveillance and cyber operations.