Anthropic researcher Jacob Coxon has resigned from the AI company, warning that the race to build increasingly powerful systems is moving faster than safeguards can keep up. Coxon announced his resignation on Tuesday, 8 September.
The former Anthropic researcher warned that competition between leading AI labs was pushing companies towards developing increasingly powerful systems, accusing them of 'gambling with our lives'.
Coxon, who spent three years conducting pre-training research at OpenAI and Anthropic, said he was leaving the AI industry altogether because he no longer believed individual companies could responsibly manage the risks associated with increasingly capable systems.
Why Jacob Coxon Quit Anthropic
Coxon's concerns centre on the possibility of self-improving artificial intelligence: hypothetical systems capable of improving their own abilities and eventually outperforming humans across a wide range of tasks.
Coxon told The Wall Street Journal that some of the most aggressive scenarios could leave AI development 'out of control' by the end of 2027, reflecting his own assessment rather than an established industry forecast.
His comments reflect his assessment of where AI development could be heading. They do not establish that self-improving superintelligence will emerge on that timetable, nor that catastrophic outcomes are inevitable.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
Coxon Says AI Competition Is Undermining Safety
Coxon's criticism does not suggest that Anthropic has abandoned AI safety research.
Instead, he argues that competition between companies such as Anthropic and OpenAI creates pressure to continue developing more capable models, even when researchers recognise potentially serious risks.
Coxon said that, in his experience, the risks were 'well understood' at Anthropic, while claiming that many at OpenAI had not fully internalised what he considers the civilisation-level stakes.
He has called for stronger coordination between AI developers and suggested that preventing an uncontrolled global race could ultimately require measures including a temporary halt to further capability improvements.
That proposal remains Coxon's position rather than an agreed industry standard.
Anthropic Researcher Echoes Broader AI Risk Concerns
Coxon's resignation gained further attention after Anthropic Alignment Science Lead Evan Hubinger publicly expressed similar concerns about advanced AI systems.
Hubinger has said his personal estimate places the probability of AI causing human extinction within the next decade at more than 10%. He has also said Anthropic is making a genuine effort to address alignment risks but does not yet have a clear solution for controlling a future superintelligent system.
That figure represents Hubinger's personal risk assessment, not a demonstrated probability or an official Anthropic forecast.
Experts remain sharply divided over both the probability of such outcomes and how quickly superintelligent systems could emerge. Some AI safety specialists believe rapid improvements in capability justify preparing for potentially catastrophic outcomes, while sceptics argue that predictions of imminent superintelligence remain highly uncertain.
The disagreement highlights one of the central challenges in the AI safety debate: the potential consequences could be enormous, but their probability and timing remain difficult to measure.
What Coxon's Resignation Means for the AI Safety Debate
Coxon's departure does not prove that AI is approaching an uncontrollable or extinction-level threshold.
What it does provide is another insider account suggesting that some researchers building frontier systems believe the risks are serious enough to reconsider their own participation in the industry.
His warning also raises a broader policy question over whether voluntary safeguards implemented by individual companies are sufficient when those same firms face significant commercial and competitive pressure to release more capable models.
For policymakers and the wider technology industry, the unresolved issue is whether stronger regulation or coordinated limits should be introduced before more powerful AI systems emerge, rather than after their consequences become clear.
For now, Coxon's resignation adds another prominent insider voice to an increasingly public debate over whether companies developing advanced AI should be responsible for setting their own limits, or whether governments and international bodies should play a greater role in determining how far that race should go.
Neither Anthropic nor OpenAI had publicly responded to Coxon's claims at the time of publication.
Frequently Asked Questions
- Who is Jacob Coxon?Jacob Coxon is a former researcher at Anthropic who resigned due to concerns about AI safety.
- Why did Jacob Coxon resign from Anthropic?He resigned due to concerns that the race to develop powerful AI systems is outpacing safety measures.
- What are the concerns raised by Jacob Coxon about AI?Coxon is concerned about the risks of self-improving AI systems and the competitive pressure to develop more capable models without adequate safety measures.
