Indexing the archive…
Your Universe of Digital Possibilities
Measure something you care about with something you can count. While you only watch the number, it tracks the truth. Make it a target, and the harder you push the further it drifts from what it was measuring.
A proxy U is the value V plus an independent error X, both normal. Pick by U and you get back only this fraction of the pick in V — the rest of the selection was error. Galton’s regression toward the mean, read as a law of metrics.
The expected largest of n standard normals, exactly by the order-statistic density, and its slow asymptote. For a Gaussian proxy the true value of the best of n is σV²/σU times it — it keeps rising, ever more slowly.
Gao, Schulman & Hilton’s fitted form for the gold reward against the distance d a best-of-n policy moves from the initial one (d is the square root of its KL divergence, which for best of n is known in closed form). It rises, peaks at d = α/2β, and falls.
The eighth prophecy is a number that stops meaning what it meant. A forecast published to The Bar changed who went; a metric published as a target changes what gets done. The mechanism here is the plainest one: select hard on a noisy measure and you select the noise. With ordinary noise the truth still rises, only less; with heavy-tailed noise it rises and then falls — the over-optimisation hump the machine-learning literature rediscovered in reward models. Next, The Score lets the people being measured move.