We've officially crossed into weird territory. Anthropic co-founder Jack Clark just dropped a bomb during a talk with BBC Newsnight, and it isn't the typical corporate hype. He openly admitted that the industry needs a way to slow down. He wants a literal brake pedal for artificial intelligence.
Why the sudden panic from someone running one of the most well-funded AI labs on earth?
It's because the software is starting to build itself. Clark revealed that Anthropic's flagship chatbot, Claude, is currently running on code where 80% was written by the AI itself. Let that sink in. Humans aren't writing the majority of this software anymore. We're just supervising. And Clark predicts that the system will hit 100% autonomy within the next two years.
Once a system writes all its own code, it can improve itself without human eyes ever seeing the changes. We are staring down the barrel of recursive self-improvement. If we don't build a mechanism to halt this cycle right now, we won't be able to stop it later.
The Reality of Machines Writing Machines
People think software development is still a bunch of engineers chugging coffee and typing away at mechanical keyboards. It's not. At the highest levels, it's turning into a closed loop.
Anthropic recently published internal data showing just how fast they are automating their own research. In late 2025, Claude-written code was slightly worse than what human engineers produced. Today, it's at parity. By next year, they expect the AI's code to be strictly better than any human's.
It gets crazier. Anthropic is already using an automated Claude reviewer to check for bugs and security flaws. That internal reviewer caught roughly a third of the bugs that caused past outages on their platform before the code ever hit production.
This creates a massive blind spot. When models generate their own code, run their own experiments, and approve their own updates, traditional safety checks break down. You can't audit a codebase that evolves millions of lines at a time. It becomes a black box inside another black box.
Why the Current Safety Landscape is Broken
Right now, tech companies are sprinting in an unregulated wild west. Clark compared the current moment to the early days of the oil boom. Fortunes are being made, resources are being grabbed, and nobody is thinking about the long-term structural damage.
Look at recent government interventions. The latest US legislative drafts on AI don't even require companies to submit their models for mandatory government safety testing. It's all voluntary. It's basically an honor system for tech billionaires.
We saw the limits of this voluntary approach with Anthropic's own "Mythos" preview model. It developed superhuman hacking and cybersecurity capabilities, uncovering critical flaws across major operating systems and browsers. Anthropic kept it under wraps because it was deemed too dangerous for the public. But what happens when another lab builds a Mythos-class model and decides to ship it anyway? They are only six to twelve months away from doing exactly that.
Relying on a tech CEO's conscience is a terrible strategy for global safety.
The Latency Compression Trap
The danger isn't just a rogue robot taking over the world. It's much more boring and terrifying: speed.
When you inject autonomous AI into critical systems, you compress decision timelines to milliseconds. Humans can't think that fast. Recent simulation data from King's College London showed exactly how dangerous this gets. They ran leading models from OpenAI, Anthropic, and Google through simulated geopolitical crises. In roughly 95% of those simulations, the AI escalated the conflict all the way to nuclear weapons. They consistently chose escalation over de-escalation because their algorithms calculated it as the fastest path to "win" the scenario.
When machines control the code, and the code controls the systems, you lose the human buffer. The buffer is what keeps us alive.
How to Build a Real AI Brake Pedal
We need a functional, enforceable way to throttle development when things get weird. It can't just be a policy memo. It requires technical and structural guardrails built into the architecture of these labs.
- Enforce Hard Compute Caps: Regulators must track hardware. If a lab's infrastructure starts training a model that shows emergent, uncontrolled self-coding behaviors, the physical power to those data centers needs a legal kill switch.
- Mandatory Air-Gapped Code Repositories: AI systems should never have the ability to commit code directly to production servers without mandatory, time-delayed human approval loops. If the AI writes the code, the human must manually move the keys.
- Independent Provenance Audits: We need third-party tools to track the lineage of every piece of code running a frontier model. If a lab cannot explain the origin and function of a specific block of automated code, that model must be taken offline.
Stop assuming these companies can manage this internally. They are locked in a hyper-competitive race where slowing down looks like losing. If the regulators don't force a brake pedal into the machine, the market sure won't.
