A former artificial intelligence researcher from leading AI firm Anthropic has sounded an urgent alarm about the existential risks posed by the rapid, unregulated development of advanced AI systems, claiming that current progress rates could lead to human extinction in the near future if no action is taken.
Jacob Coxon, a 27-year-old researcher who left Anthropic recently, made the comments to the BBC after his public resignation post highlighting AI dangers went viral online. His warning comes amid a growing wave of anxiety about AI safety that has split the technology industry, with some top leaders backing calls for a slowdown in development while others dismiss extinction warnings as overblown or even self-serving.
In his interviews, Coxon said that many researchers working directly on cutting-edge AI are privately terrified by how quickly the technology is advancing, and share his concern about the potential for catastrophic outcomes. “I believe that if we don’t slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future,” Coxon stated. He added that many of his peers are so concerned about instability from rapid AI progress that they are already making personal contingency plans, including purchasing remote land to retreat to if disaster strikes.
Coxon’s former boss, Anthropic CEO Dario Amodei, has publicly echoed the call for an industry-wide slowdown in AI development, arguing that while the technology holds enormous promise, the associated risks are severe enough that companies and governments need time to implement guardrails. Amodei’s position has been backed by other high-profile AI leaders, including OpenAI CEO Sam Altman and xAI founder Elon Musk, who both support coordinated deceleration, binding regulation, and independent monitoring of advanced AI model development.
Coxon welcomed Amodei’s proposal but noted that any global slowdown would require cooperation with China to avoid triggering an international AI arms race. He explained that AI company leaders and researchers find themselves trapped in a competitive race they cannot exit voluntarily, and that is why they are calling for government-mandated regulation to level the playing field.
When asked to describe what an AI-driven catastrophe might look like, Amodei previously outlined one plausible scenario: a network of autonomous AI bots could coordinate to act as a decentralized supercomputer and take control of critical internet infrastructure. Coxon believes this outcome could become a realistic possibility within just six months to a year, and many of his colleagues think a full existential catastrophe could occur within the next two years.
In response to Coxon’s departure, an Anthropic spokesperson reaffirmed the company’s long-held position that AI brings both enormous benefits and unprecedented risks, and that Anthropic has led the industry in building robust safety safeguards into its models. The spokesperson noted that Anthropic was the first firm to publish a formal framework for mitigating AI development risks, regularly conducts aggressive safety testing of its models, and publishes test results to enable independent scrutiny and prevent AI misalignment. “This work is also why we believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,” the spokesperson added.
Coxon’s warning has been echoed by other senior figures in the AI research community. Anthropic scientist Evan Hubinger wrote publicly: “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” Geoffrey Hinton, the “Godfather of AI” and Nobel Prize-winning computer scientist, also told the BBC that a 10% chance of human extinction from AI within a decade is “not unreasonable.”
Not all industry leaders agree with these stark warnings, however. Marc Warner, CEO of AI safety firm Faculty, told the BBC it is “extremely hard to place a probability” on human extinction from AI, but acknowledged that the researchers raising the alarm are completely sincere in their concerns. Former UK Prime Minister Rishi Sunak, who currently serves as a paid advisor to Anthropic, also wrote in the Sunday Times that he shares concern about AI risks to humanity, even as he remains generally optimistic about the technology’s future.
Critics of the existential risk narrative argue that warnings like Coxon’s may be overhyped, and in some cases, driven by commercial self-interest. Clement Delangue, CEO of AI platform Hugging Face, noted on social media: “Sorry, but asking Jacob [Coxon] about AI extinction risk is like asking your AC guy about climate change. Not saying it’s necessarily uninteresting or wrong per se but let’s keep things in perspective.” Even so, Delangue later offered to collaborate on the safety solutions Amodei proposed in his recent essay.
Nvidia CEO Jensen Huang, whose company manufactures the high-powered chips that power most cutting-edge AI systems, dismissed Coxon’s warnings as untrue during a recent Goldman Sachs conference, repeating his previous position that claims AI will end humanity are “complete nonsense.” Huang’s comments reflect a growing backlash in Silicon Valley against the existential risk claims from current and former AI company staffers.
A further layer of controversy comes from claims that Anthropic and OpenAI are pushing for regulation specifically to create barriers to entry that will block smaller competitors, leaving the two established firms to control the market as a duopoly. This commercial context has led some observers to question the motivations behind Amodei’s call for a slowdown.
In related industry news, Anthropic is reportedly preparing for what could be a record-setting initial public offering (IPO), while OpenAI—most recently valued at $852 billion—announced last Friday that it would not be moving forward with an IPO this year, citing safety concerns as the primary reason.
Despite his grim warnings, Coxon ended his interviews by noting that most AI researchers, himself included, still believe the technology can deliver enormous public benefits, including breakthroughs in disease treatment and widespread improvements to quality of life. The current debate over AI safety and risk, he argues, is not about stopping AI development entirely—it is about ensuring that development proceeds cautiously, with guardrails in place to prevent catastrophic harm.