Summary

Tom Bilyeu interviews Connor Leahy, CEO of ControlAI, for eighty-five minutes. Leahy is not a commentator: he is a computer scientist who built some of the first open-source large language models and led EleutherAI, and he now argues that building superintelligence should be a crime. What makes this worth a full entry is that he does the work most people making this argument skip — he defines what he wants banned, distinguishes it from AI generally, names the two laws he would pass, and answers the enforcement question directly. Bilyeu pushes back throughout rather than nodding along, which is what makes the argument testable.

• Leahy's central distinction, and the one he says is deliberately blurred by lobbyists: superintelligence "is not a tool, it's not a weapon, it's an adversary", and it is a different technology from AI in general. His analogy — you can buy uranium ore on Amazon and keep it on your desk harmlessly, while weapons-grade enriched uranium is illegal. The line exists and can be drawn.
• His definition is behavioural rather than technical: any system, single agent or swarm, that can out-compete humanity at all relevant tasks. Every business you start loses to an AI business, every trade loses to an AI fund, every campaign loses to an AI-advised opponent, every military campaign loses to more autonomous systems. At that point economic, military and political power sits with non-human entities.
• Autonomy is the criterion, not capability. A human rubber-stamping decisions they do not understand is not oversight — he expects human CEOs to persist for years as legal representation while approving what AI executives decide.
• He argues the milestone that matters is the point of no return, not extinction: the point past which we no longer control the future and cannot get it back, even if people live on for decades. He says we are close, does not know how close, and that this is itself the argument for stopping.
• On why nobody can guarantee safety: "the people at OpenAI do not know what is going on inside of their AIs." He cites Dario Amodei as putting understanding of what happens inside these models at roughly 3%. His description of the technology matches OpenAI's own chief scientist in DK-93 almost word for word — grown rather than written, billions of numbers rather than readable code.
• The historical comparison he draws is Leo Szilard, who did the maths on the bomb and went straight to the government rather than founding a startup to sell weapons. His charge against the AI field is that its founders did the opposite — DeepMind, Anthropic and OpenAI went to raise money rather than to build institutions. He notes nuclear governance required inventing entirely new forms of law; the IAEA had no precedent in any legal code.
• The two laws he would pass, stated concretely. First, criminalise the creation of superintelligence — including the attempt, as with attempted murder or attempted construction of a nuclear weapon. Second, regulate the precursors: ability to self-reproduce, task-horizon length, agentic autonomy, with registration and oversight of large frontier experiments. He notes frontier training runs cost billions, so this would touch a handful of companies rather than 99% of the industry.
• His metaphor for sequencing: driving at 120mph through thick fog knowing there is a cliff somewhere but not where. "Before we argue about the speed, let's first pull over."
• On US-China, he rejects the racing frame outright: the only defensible position is that nobody may build superintelligence anywhere, including one's own companies. If China builds it, it kills American citizens; the reverse holds equally. He believes most people arguing to race with China simply do not believe superintelligence is possible and think AI will always be an app on a phone.
• On enforcement, he does not dodge: all law rests on credible deterrence, and well-designed deterrence never has to be used. He would exhaust sanctions and diplomacy first but reserves the right to self-defence — while conceding deterrence only works against rational actors.
• On open source he refuses the ideological answer despite having built open-source LLMs himself: should the F-35 blueprints be open? Once weights that are superintelligent or close to it are distributed, recall is not feasible — a point that lands directly on the 6GB distilled-model scenario raised in DK-93.
• A timeline claim that inverts the usual worry: he expects superintelligence within a couple of years, arriving before mass job displacement becomes the dominant problem. "I kind of wish we were in a world where superintelligence is 20 years away. Then job displacement would become a huge problem."
• Useful correction on Bill Gates: Leahy points out Gates signed the 2023 Center for AI Safety extinction statement alongside Sam Altman, Dario Amodei and Geoffrey Hinton, so recent coverage framing this as Gates changing his mind is wrong. His criticism is that Gates stopped at job displacement rather than going to extinction risk.
• On probability language, citing CIA research: analysts saying "likely" meant anywhere between 30% and 90%, which is why he insists on putting numbers on gut feelings. The number is not a mathematical claim, it is a way of saying what the word actually meant.

Why it matters

This is the most fully worked-out version of the stop-building position the knowledge base holds, and it is the one that answers the objection the Moonshots panel used to dismiss it. In DK-93 Salim Ismail says "I see no mechanism by which we can slow this down, like zero" — Leahy names the mechanism: criminalise the attempt, regulate the precursors, back it with the same deterrence that already stops people building nuclear weapons. Whether that would work is arguable; that it is a mechanism is not.

Three entries filed in two days now describe the same technology from three positions — the panel who think slowing down is impossible and undesirable (DK-93), the insider who thinks the risk is real but leaves rather than campaigns (DK-92), and the campaigner with draft legislation (this one). They agree on the facts and on the description of the technology, including OpenAI's own chief scientist's phrase that AI is grown rather than designed. They disagree entirely about what follows. That is the shape the Handbook's safety section should take.

His sharpest contribution is the definitional one. Most of this argument is conducted about "AI", which lets both sides talk past each other — the uranium ore against the enriched uranium. Anyone writing about regulation should have to say which of the two they mean.

The claim to keep flagged as unverified is the 3% figure attributed to Dario Amodei on model interpretability. It is quoted second-hand and not sourced in the interview.

ZqiS0tLAKGE-transcript.txt