When a person who spends three years building frontier models walks away and says the creators privately believe the technology could kill everyone by the end of the decade, it's worth paying attention. Jacob Coxon didn't quit Anthropic to launch a startup or chase a bigger paycheck. He left because he concluded that the entire artificial intelligence industry is barreling toward self-improving superintelligence with zero regard for human survival.
Coxon spent his career doing core pretraining research at both OpenAI and Anthropic. He helped build the systems that everyone else is scrambling to catch up with. When an insider with that level of technical access sounds the alarm, the standard corporate PR dismissals ring hollow.
Inside the Silicon Valley Panic Room
Most people think the existential dread surrounding artificial intelligence is internet hype or science fiction fodder. Coxon argues the opposite is true. The public messaging from tech executives is heavily sanitized to prevent panic, but the private conversations among engineers tell a terrifying story.
"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon wrote in his public resignation post. This isn't a marketing stunt to gain clout. It's a clear-eyed assessment from scientists watching recursive self-improvement loops spin out of theoretical physics and into actual code.
The timeline is shrinking fast. We aren't talking about a distant horizon fifty years away. We are looking at a window where systems could soon possess the capability to autonomously hack global infrastructure, outsmart human oversight, and acquire real-world resources faster than regulatory bodies can draft a memo.
Why Smart Labs Keep Building Dangerous Code
A logical question follows you down this rabbit hole. If the people building these models genuinely believe they pose an existential threat, why don't they just stop?
Coxon points to a toxic prisoner's dilemma driving the entire sector. At labs like OpenAI, many employees haven't fully internalized the civilizational stakes of what they are touching. At Anthropic, the leadership actually understands the dangers, but they justify their speed by assuming everyone else will act recklessly if they don't get there first.
It's a twisted logic. They race to build an uncontrollable god because they are terrified a rival lab or foreign competitor will do it first. That hubristic gamble is happening behind closed doors and inside private Slack channels, entirely divorced from democratic oversight.
The Myth of Fast Alignment
For years, the industry mantra relied on alignment research—the idea that we can teach an advanced machine to share human values just in time. But other insiders are backing away from that optimism. Evan Hubinger, an Anthropic safety researcher, recently stated that the probability of advanced AI destroying human civilization within the next decade exceeds ten percent.
Think about those odds. If you stepped onto an airplane with a ten percent chance of mid-air structural failure, you'd stay on the ground. Yet the tech world treats that same probability of global catastrophe as an acceptable externality in pursuit of market dominance.
We are speedrunning an endgame without understanding the mind of the machine we are trying to bind. Aligning a system that thinks millions of times faster than its creator isn't a technical puzzle you solve over a weekend hackathon. It requires an entirely different approach to pacing, safety, and international coordination.
What Needs to Happen Now
If we want to avoid sleepwalking into a catastrophe, business-as-usual cannot continue. Coxon suggests that avoiding a global race may require drastic, costly interventions, including temporary moratoriums on pushing frontier model capabilities higher until our safety guarantees match our computing power.
We need to stop treating AI progress as an inevitable law of physics and start treating it like the high-stakes biohazard engineering it actually resembles. Stop waiting for tech companies to regulate themselves. Demand transparent oversight, enforce strict capability caps, and listen to the engineers who are brave enough to walk away before it's too late.
