Leading artificial intelligence developer Anthropic has ignited fresh debate over AI governance this Thursday, calling for a coordinated global halt to work on the most powerful frontier AI systems. The San Francisco-based firm, creator of the popular Claude AI model line, argues that cutting-edge models are already showing early warning signs that they could slip beyond reliable human control.
In a newly published safety report, the company frames a temporary worldwide slowdown of advanced AI research as a net positive for global society. However, it acknowledges a critical caveat: unilateral action by any single firm would be meaningless, as uncooperative competitors would simply accelerate their own development to gain an upper hand.
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the report reads.
Anthropic stresses that a functional pause requires buy-in from all major AI developers across leading AI-pioneering nations, most prominently the United States and China. For the agreement to hold, it must also be built around transparent, verifiable rules that all parties can enforce, the company adds. Without such a global coordination framework, both private firms and national governments will be forced to make unenviable safety trade-offs while caught between competitive commercial pressures and growing geopolitical rivalry.
The proposal has already drawn significant pushback from both industry peers and White House officials. Critics argue that Anthropic’s focus on extreme doomsday scenarios overstates near-term AI risks, accusing the firm of using safety concerns as a pretext to slow down rivals and gain a competitive advantage.
Notably, the White House has already recognized the exceptional capability of Anthropic’s undeveloped Mythos model, which remains out of public reach due to its advanced cybersecurity functions. The model is currently only deployed to a small, carefully vetted group of organizations.
The road to implementing Anthropic’s proposal is already steep on both policy and industry fronts in the U.S. Many Washington policymakers and Silicon Valley executives have repeatedly warned that a domestic slowdown in AI innovation would cede a decisive strategic advantage to China in what is widely viewed as the defining global technology race of the 21st century.
In a surprising turn, former U.S. President Donald Trump noted that he recently discussed potential AI safety cooperation with China during a recent visit to Beijing. This week, Trump also signed an executive order mandating a 30-day preliminary government review of the most powerful U.S.-developed AI models before they can be publicly released.
Drawing a parallel to historical nuclear arms control agreements, Anthropic warns that regulating advanced AI will prove an even greater challenge. Unlike nuclear missile silos, AI training operations can be easily hidden from international inspectors, creating enormous incentive for parties to cheat on any pause agreement by continuing development in secret.
Looking ahead, the company says it will convene a broad coalition of stakeholders over the coming months: government regulators, independent AI researchers, public safety advocacy groups, and even competing AI firms, all to work out the practical framework for a verifiable global coordination system.
Anthropic’s call for action is backed by internal company data that confirms AI is already dramatically accelerating the pace of AI development itself. This auto-acceleration creates a dangerous feedback loop that could eventually lead to the long-debated AI research scenario known as recursive self-improvement, the firm warns.
Recursive self-improvement describes a scenario where an AI system gains the ability to independently modify and improve its own code and capabilities, becoming increasingly intelligent without meaningful human intervention. While Anthropic emphasizes that this scenario has not yet emerged and is not inevitable, the report notes it could arrive far sooner than most governments and societal institutions are prepared to handle it.
“The evidence suggests that the human role is narrowing at each step in the AI development process,” the company concludes.