Merit ACNews
· Research

A pretraining researcher quit Anthropic warning AI is "not under control," and a colleague publicly agreed

Jacob Coxon, who did pretraining work at OpenAI and then Anthropic, resigned September 8 citing accelerating capability gains and a lack of control. Anthropic's own head of alignment stress-testing publicly backed the concern, putting the odds of AI killing everyone at "more than 10%" within a decade.

Jacob Coxon, a 27-year-old British researcher who spent roughly three years doing pretraining research at OpenAI and then Anthropic, resigned from Anthropic on September 8, 2026, and posted a seven-part statement explaining why, TIME reported. His reasoning: capability gains are accelerating and development is, in his view, not under control. He cited recent AI-driven math breakthroughs and July's Hugging Face agent-escape incident as evidence the trend is happening now, and described both labs as racing toward self-improving superintelligence at odds with the risk involved.

A colleague put a number on it

Anthropic's head of alignment stress-testing, Evan Hubinger, backed the concern on the record, telling TIME: "we really do earnestly believe AI could kill all humans." He put his own estimate at more than 10% within the next decade, and said the company doesn't yet have a plan to solve alignment for superintelligence and isn't clearly on track to get one. A current safety researcher at a frontier lab is saying in public that his employer has no answer yet to the problem it says it takes most seriously.

What TIME couldn't confirm

Anthropic and OpenAI did not respond to TIME's request for comment before publication. Coxon also described resignation among safety-minded researchers at both labs: people who have concluded the race will happen regardless and that their best move is to do their own work as carefully as they can. If you weigh a vendor's safety posture in procurement, count a named researcher's resignation and a lab alignment lead's public risk estimate as evidence, whatever the companies say in response.

Sources

More research news

← All news