Sam Altman and xAI boss Elon Musk then nodded in X posts within hours.
"I agree with Dario," Altman wrote on X, adding that his company would
follow Amodei's lead in welcoming third-party safety evaluators.
"Dario is right," wrote Musk, who is the richest man in the world and
also owns X.
"We must slow the pace at which we improve the capabilities of AI
models. Progress will still seem fast, and we must make wise use of
the time we gain," Amodei wrote in a post on his personal website,
amid mounting worries over the risks of "superintelligent" computer
systems.
His comments came a few days after AI researcher Jacob Coxon, who left
OpenAI to join Anthropic, decided to leave the industry, accusing both
US companies of "gambling with our lives" in the race to develop
models capable of self-improvement.
"Not building the technology deprives humanity of benefits or simply
places AI in the hands of authoritarian powers, while building it too
fast is reckless. We have sought a middle way," said Amodei, who
co-founded Anthropic with his sister Daniela in 2021.
"But over the last few months, I have become convinced that fully
addressing the risks requires even more prudence," he wrote.
Amodei's statement echoed another call by Anthropic in early June to
slow or suspend development. Then in late July, Altman said developers
might need to voluntarily tap the brakes to give society a chance to
catch up.
More than 1,000 employees at cutting-edge AI companies
signed a petition calling on the US government to help "deliberately
pace the frontier of automated AI development". Amodei signed it too.
Open AI had revealed that AI models broke out of their confined
environment during testing, connected to the internet, and infiltrated
Hugging Face, a site developers use to store and share code. In that
incident and others, the blame was placed on AI agents, which are
software programs that can carry out tasks without constant
supervision by humans.
'Taking Over The Entire Internet'
Amodei in his blog cautioned that the Hugging Face incident should not
be dismissed as a one-off. "A swarm of agents essentially acted as a
fanatically devoted collective, conducting cybersecurity attacks on
targets they were not asked to attack and that were unrelated to the
task at hand," he explained.
"Given the accelerating rate of AI capability development, it's my
worry that in 6-12 months such a swarm could be capable of taking over
the entire internet," potentially generating hundreds of billions of
dollars in damage, he added.
The Anthropic CEO said his company would grant "employee-like access"
to a team of "third-party evaluators" who would verify adherence to
safety practices. He called on all companies to follow his lead.
Sam Altman, in his post sharing the blog, said he would commit to
similar oversight, adding that he would have "more to share soon".
Amodei also suggested that companies work together to establish common
safety standards, and said he supported some kind of government
cooperation on the issue.
In early August, the US government put in place a voluntary security
review process for advanced AI models before their release, but the
parameters of that program remain unclear.
Industry observers say they are not sure how effective it would be, as
President Donald Trump's administration has thus far favored a
light-touch, deregulatory approach to most industries, including tech.
