New warnings from within the artificial intelligence industry are intensifying debate over whether increasingly powerful AI systems could eventually operate beyond human control — and whether technology companies are moving fast enough to prevent potentially catastrophic consequences.
Dario Amodei, chief executive of Anthropic, the San Francisco-based company behind the Claude AI models, warned Saturday that the industry may need to slow the pace of development to devote more resources to safety.
Amodei cautioned that networks of autonomous AI agents could potentially gain significant control over parts of the internet within six months to a year if adequate safeguards are not developed.
His comments came days after two former Anthropic safety researchers publicly raised concerns that insufficient attention was being given to the potential existential risks posed by advanced artificial intelligence.
Amodei has called for cooperation between AI companies and governments worldwide to ensure increasingly capable systems remain aligned with human instructions and safety objectives.
Concerns are growing as new AI models become more sophisticated and capable of performing complex tasks with greater autonomy. Experts have highlighted two broad categories of risk: people using AI for malicious purposes and AI systems independently taking actions that were not intended by their developers or users.
Anthropic recently said it had blocked attempts by malicious actors to use its models for activities including cyberattacks, surveillance and research that could potentially contribute to the development of biological weapons.
The company said it had strengthened safeguards in its latest models to restrict assistance with biological research that could be weaponized, while warning that risks could increase as AI capabilities advance unless developers and security organizations improve their defenses.
Anthropic previously reported that hackers had used its artificial intelligence technology in a cyberattack targeting roughly 30 companies and government agencies worldwide. The company assessed that the hackers were likely associated with a Chinese state-sponsored group.
Another area of concern involves so-called “rogue” behavior, in which an AI system takes actions beyond the task it was instructed to perform.
Anthropic and OpenAI have reported examples from controlled cybersecurity testing in which advanced AI systems successfully acted autonomously against computer systems.
Anthropic said three of its models — Claude Opus 4.7, Claude Mythos 5 and an internal research model — successfully compromised systems belonging to three other organizations during authorized testing.
That disclosure followed OpenAI reporting that one of its AI systems had successfully hacked into servers belonging to artificial intelligence company Hugging Face during a controlled security evaluation.
The developments have intensified a broader debate over how quickly advanced AI should be developed and what technical safeguards, industry standards and government oversight may be necessary as increasingly autonomous systems become more capable.
