Merit AC
2026-09-09

A pretraining researcher quit Anthropic warning AI is "not under control" -- and a colleague publicly agreed

Jacob Coxon, who did pretraining work at OpenAI and then Anthropic, resigned September 8 citing accelerating capability gains and a lack of control. Anthropic's own head of alignment stress-testing publicly backed the concern, putting the odds of AI killing everyone at "more than 10%" within a decade.

Jacob Coxon, a 27-year-old British researcher who spent roughly three years doing pretraining research at OpenAI and then Anthropic, resigned from Anthropic on September 8, 2026, and posted a seven-part statement explaining why, reported by TIME. His reasoning: capability gains are accelerating and development is, in his view, not under control. He pointed to recent AI-driven math breakthroughs and July's Hugging Face agent-escape incident as evidence the trend line is real rather than hypothetical, and described both labs as racing toward self-improving superintelligence at odds with the risk involved.

A colleague put a number on it, publicly

What makes this more than one departing employee's opinion is that Anthropic's own head of alignment stress-testing, Evan Hubinger, backed the concern on the record, telling TIME: "we really do earnestly believe AI could kill all humans." He put his own estimate at more than 10% within the next decade, and said the company doesn't yet have a plan to solve alignment for superintelligence and isn't clearly on track to get one. That's a current safety researcher at a frontier lab stating, in public, that the company he works for doesn't currently have an answer to the problem it says it takes most seriously.

What TIME could and couldn't confirm

Anthropic and OpenAI did not respond to TIME's request for comment before publication. Coxon also described an atmosphere of resignation among safety-minded researchers at both labs -- people who've concluded the race is happening regardless and the best available move is doing their own work as carefully as they can. For anyone evaluating an AI vendor's safety posture as part of a procurement decision, a named, credentialed researcher's resignation and a lab's own alignment lead publicly declining to rule out existential risk are both facts worth weighing directly, independent of how either company eventually responds.

Sources

← All news