For years, warnings from the highest ranks of the artificial intelligence industry about potential existential threats to humanity have circulated, but for much of that time, these alarms were dismissed as science fiction. When the world’s first global AI safety summit convened at Bletchley Park in November 2023, the conversation centered on the most catastrophic risks posed by frontier AI models, with many critics and observers arguing that the only real harms we faced were far more ordinary: mass labor displacement, academic dishonesty, and everyday privacy violations. Three years on, however, that skepticism has eroded, and a growing chorus of influential industry leaders are doubling down on urgent calls to rein in the breakneck speed of AI advancement.
On Saturday, Dario Amodei, the chief executive of leading AI firm Anthropic, publicly urged the global tech community to slow the pace of frontier AI development. His call received immediate backing from two of the most prominent figures in the industry: Sam Altman, head of rival giant OpenAI, and Elon Musk, founder of xAI and one of the earliest public voices warning of unregulated AI risk. The following day, former Anthropic AI researcher Jacob Coxon, who left the company over safety concerns, told the BBC that current employees building next-generation AI systems are “genuinely frightened” about the long-term future of humanity if development continues at its current rate.
While a slowdown might sound like a straightforward solution to growing risk, the geopolitical, economic, and structural barriers to implementing such a measure are enormous. At its core, the issue mirrors the decades-long deadlock of the Cold War nuclear disarmament movement: no nation or company wants to be the first to step back, for fear of being outpaced by competitors. The United States has long framed AI development as a high-stakes geopolitical race with China, and on Sunday, former President Donald Trump made that position explicit, stating that the U.S. currently holds a lead over China and declaring “Whoever wins AI, wins.” China, likewise, has made advancing its domestic AI industry a top national priority, meaning a unilateral pause by Western companies would simply cede the global lead to competitors, leaving them permanently behind.
Beyond geopolitical rivalry, there is also the unanswered question of how a slowdown would actually be enforced. There is no existing global regulatory body with the authority to police frontier AI development, and any plan would require unprecedented levels of transparency from private tech companies — a level of trust that many argue the tech sector has never earned. Amodei has put forward a three-point framework to advance his call, including independent third-party monitoring of AI model development during training, coordinated industry-wide standards, and binding global regulation. Still, many industry observers and analysts remain skeptical that such a plan can work in practice.
Ed Zitron, chief executive of EZ Primary Research, argues that proponents of a slowdown have failed to define exactly what a reduction in pace would look like in tangible terms. “Nobody has given a substantive explanation of what ‘slowdown’ means,” Zitron explained. He added that halting cutting-edge AI model training would leave Western firms vulnerable to Chinese competitors, noting that while profit margins might improve for companies that pause, their technology would stagnate while rival labs advance. “Right now we are very thin on what a ‘slowdown’ means,” he said, adding that the push for an abrupt slowdown could even trigger a sudden bursting of the AI investment bubble before clear guardrails are in place.
In the United Kingdom, where the AI sector has been positioned as a core driver of future economic growth, a widespread slowdown raises major concerns. The government has already outlined plans to expand AI use across the National Health Service to improve patient outcomes, and the sector delivered a much-needed boost to UK GDP over the past summer. Policymakers across the political spectrum have embraced wider AI adoption in workplaces, schools, and daily life, with one former government adviser noting there is “no plan B” for economic growth — meaning a sudden pullback could derail years of economic strategy.
Adding to the uncertainty, the AI industry is currently burning through billions in investor and corporate capital while consuming massive amounts of energy and natural resources, with comparatively little revenue generated to date. Multiple recent surveys have found that many early corporate adopters of cutting-edge AI are disappointed with the return on their investments, and many economists predict that a market correction — a so-called “AI bubble burst” — is coming, with only a small handful of current giants surviving to become the most powerful mega-corporations in global history. That concentration of power, even if the sector stabilizes, carries its own unique set of regulatory and social risks.
The recent decision by OpenAI to delay its planned initial public offering has been interpreted two ways: some see it as a landmark moment of corporate responsibility, as the company prioritizes public safety over short-term investor payouts. Others argue it is a pragmatic business move: going public while the company’s core technology is widely perceived as an existential threat to humanity would make it impossible to secure the lucrative valuation OpenAI was targeting.
Alexander Voica, a senior leader at UK-based AI firm Synthesia, notes that the core challenge facing regulators and developers alike is that no one can predict with certainty how the technology will evolve. “We know that these systems are getting more powerful, but we don’t know where and how they’re going to be used, and we haven’t figured out essentially a way of taking full advantage of their potential,” Voica explained. He warned that rushing to implement strict regulation and a forced slowdown before key questions about the technology are answered could backfire, stifling innovation that could deliver widespread public benefit.
Critics of the current unregulated development model point out that the entire AI boom is underpinned by trillions in investor cash, and the primary driver of rapid advancement is corporate profit, not public good. “I’m not worried about the existential risks of AI, I’m worried about the corporate greed of the companies that are creating it,” said Sasha Luccioni, founder of Sustainable AI. Leading AI researcher Dame Wendy Hall, a computer scientist who advises the United Nations on AI policy, argues that the current crisis stems from a failure of responsible governance by the companies developing the technology, not an inherent risk in the technology itself. She compared the current situation to a farmer who allows a dangerous bull to escape its fence, only to blame the bull for the destruction it causes. “Of course it’s not the bull’s fault — it’s the farmer,” Hall said. “Clearly, the fences weren’t robust enough, and that is exactly what we are seeing with AI guardrails right now.”
Still, the debate over regulation has its own fringe divides, with some observers arguing that the push for strict rules is a politically motivated attempt to put the entire industry out of business. Parker Thayer, an investigative researcher at the conservative-leaning Capital Research Center, framed the push for strict regulation as an effort to “regulate AI into oblivion” in a recent social media post that was viewed nearly eight million times. While Thayer’s view is extreme and unproven, it demonstrates that there is no widespread consensus on whether regulation is even the right path forward for the sector.
Whatever the ultimate outcome of the current debate, the high-profile public split over safety and the growing focus on existential risk has already done lasting reputational damage to the leading firms at the forefront of the industry. As Dame Wendy Hall put it: “Would you invest in a company that says it’s going to bring about human extinction?”
