A Top AI Researcher Just Quit Over Extinction Fears. Here’s What He Saw.

You probably worry about AI taking your job. You probably think the biggest risk of artificial intelligence is a flooded market of unemployed copywriters and displaced programmers.

You are aiming your anxiety at a mosquito while ignoring the approaching asteroid.

Last week, a 27-year-old math prodigy named Jacob Coxon did something radical. He resigned from his job as a core pre-training researcher at Anthropic. Before that, he was a core contributor to GPT-4o and GPT-4.5 at OpenAI. He was as deep inside the engine room of the AI revolution as you can get.

He didn’t quit to join a competitor. He quit to warn us that the industry is barreling toward a self-improving superintelligence, and that high-level executives privately believe there is a greater than 10% chance of human extinction within this decade.

The people closest to the fire aren’t running away because they hate the warmth; they’re running because they can see the storm coming.

When I read his warning, I felt a chilling sense of déjà vu. It reminded me of an essay I read eleven years ago, in 2015, by Tim Urban titled “The AI Revolution.” It was a beautifully terrifying piece that reduced the entire future of human civilization to a single, brutal binary: extinction or immortality.

Urban’s core thesis was that human brains are terrible at understanding exponential growth. We think linearly. We expect tomorrow to be slightly better than today. But technological progress doesn’t work like that. It compounds. Urban coined a term for the amount of progress required to completely melt an ancient person’s brain: the Die Progress Unit (DPU).

For thousands of years, it took centuries to accumulate one DPU. Then it took decades. Now? We are racking up DPUs in a matter of weeks. In 2022, ChatGPT emerged. In 2023, GPT-4 brought multimodal capabilities. By last week, AI models were autonomously opening Blender, Unity, and Excel, learning to navigate unfamiliar software through trial and error for hours on end. In the academic world, AI is already solving 80-year-old mathematical conjectures and formalizing Fermat’s Last Theorem in 11 days—a task previously estimated to take a human a decade.

The world is becoming science fiction in real-time. But that’s not what keeps Jacob Coxon up at night.

The real terror is the “self-improving superintelligence.” In 2015, Tim Urban asked a hypothetical question: What happens when AI starts helping humans research stronger AI? In 2026, we have our answer. OpenAI recently released GPT-5.3 Codex, explicitly stating it was the first model to play a critical role in its own creation.

We aren’t just building a tool anymore. We are planting a seed, and we’ve already lost control of the roots.

The flywheel is spinning. A smarter AI creates more economic value. More economic value attracts more capital. More capital buys more GPUs. More GPUs train a stronger AI. And that stronger AI is now actively used to design the next generation. Add cutthroat corporate competition and geopolitical arms races into the mix, and you have a river that is accelerating rapidly toward a waterfall. Nobody—no single individual, no government, no AI lab—is holding the steering wheel.

This is the great blind spot in our public discourse. We debate copyright laws, market shares, and job displacement. We think the risk is a rogue lab acting irresponsibly. But the real existential risk is the automatic, self-reinforcing dynamic of an intelligence explosion. Even if every researcher wanted to stop, they can’t. If you pause, your competitor surges ahead. The momentum is baked into the system, entirely independent of human intention.

Tim Urban illustrated this by pointing out how we underestimate intelligence. We think of the gap between an average person and Einstein as vast. But on a biological scale, the cognitive difference between a chimpanzee and a human is just a tiny sliver. Yet, that sliver is the difference between poking a stick into an ant hill and splitting the atom. If a superintelligence stands just one rung above us on the cognitive ladder, it might view us the way we view ants. It couldn’t even explain its simplest concepts to us because we lack the cognitive hardware to receive them.

This is why the end of Urban’s essay is so haunting. Earth has seen millions of species, and almost all of them eventually go extinct. Extinction is a gravitational attractor; once you fall in, you don’t come back. But superintelligence might be the first thing to open up a different attractor: Species Immortality.

Humanity walks a balance beam. Sooner or later, an asteroid, a plague, or a self-inflicted technological wound will knock us off. Superintelligence is the ultimate jolt to that beam. It will push us in one of two directions.

While the public debates whether programmers will lose their jobs, the actual stakes are whether our species becomes gods or dust.

I understand why researchers stay in the lab, even those who share Jacob’s fears. The promise is simply too massive. AI is already accelerating drug discovery, decoding the human genome, and inventing new materials. If the curve continues, it might cure aging. It might cure death itself. It might take us to Alpha Centauri. The temptation to reach for the ultimate prize is too intoxicating to resist, even if the odds of annihilation are 10% or higher.

You don’t need to work in AI to be a stakeholder in this gamble. Every time you use ChatGPT, every prompt you type, feeds the flywheel. We are all passengers on this boat, watching the river accelerate beneath us.

I don’t know if we are taking humanity’s greatest step or pulling the final black ball out of the jar. I just know that the dashed line Tim Urban drew in the fog eleven years ago is no longer hypothetical. We are standing on it. And the water is rising fast.