Goodhart’s Law and Why Measurement is Hard
The other day, I was failing to teach my 3-year-old son about measurement. He wanted to figure out if something would fit in an envelope, and I was “helping” by showing him how to measure the width of the envelope, then comparing it to the width of the paper he was trying to insert. It turns out that this is a trickier concept than I had assumed; the ability to understand even simple measurements requires a fair amount of cognitive maturity. 3-year-old kids can compare directly, but the concept of using a measure to compare indirectly is more difficult. I finally let him try to fit the paper in the envelope to see it wouldn’t fit. As with many other cognitive skills, the fact that it’s a counter-intuitive learned skill for children means that adults don’t do it intuitively either. So why are measures used? There are lots of good reasons, and I think a useful heuristic for understanding where to use them is to look for the triad of intuition, trust, and complexity. Measurement replaces
Highlights Tour This post is part of the highlights tour, a series of historically significant ribbonfarm posts The other day, I was failing to teach my 3-year-old son about measurement. He wanted to figure out if something would fit in an envelope, and I was “helping” by showing him how to measure the width of the envelope, then comparing it to the width of the paper he was trying to insert. It turns out that this is a trickier concept than I had assumed; the ability to understand even simple measurements requires a fair amount of cognitive maturity . 3-year-old kids can compare directly, but
Explore this link on the map →saved by
related reading
- Goodhart's law - Wikipediaen.wikipedia.org
- Biases: An Introductionreadthesequences.com
- Goodhart's Law Isn't as Useful as You Might Think - Commoncogcommoncog.com
- Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blogsohl-dickstein.github.io
- How to Measure Anything — LessWronglesswrong.com
- The fallacy of “what gets measured gets managed” - Ness Labsnesslabs.com
- Gears in understanding — LessWronglesswrong.com
- Is Anything Worth Maximizing?. How metrics shape markets, how we’re… | by Joe Edelman | The School for Social Design | Mediummedium.com
- You Are Not Measuring What You Think You Are Measuring — LessWronglesswrong.com
- The Limits of Dataissues.org
- Metric fixation - Wikipediaen.wikipedia.org
- Plato's Dashboardsferd.ca