Goodhart’s Law, the Cobra Effect, and Dunning-Kreuger • Posts by EIM | Evidence In Motion

If the test itself becomes a training tool, then are you simply training to the test and does the test actually still work?

Goodhart’s Law, the Cobra Effect, and Dunning-Kreuger November 19, 2019 • Sports It happened again. I got caught in the black hole of web-surfing and the downward trundling of “ oh, what ‘ s that? That sounds interesting,” which led to a 4-hour adventure into interesting concepts and underlying psychologic phenomena. Enjoy! Let ‘ s start with a thought-provoking idea. Have you ever considered how expectations or targets might influence our motives, decision-making, and/or actions? Well, Goodhart ‘ s Law is exactly that: “ When a measure becomes a target

related reading

• Goodhart's law - Wikipediaen.wikipedia.org
• Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blogsohl-dickstein.github.io
• Goodhart's Law Isn't as Useful as You Might Think - Commoncogcommoncog.com
• Goodhart's Law and Why Measurement is Hard — Ribbonfarmribbonfarm.com
• Dunning–Kruger effect - Wikipediaen.wikipedia.org
• The Algernon Argument · Gwern.netgwern.net
• Goodhart's Law — AI Alignment Forumalignmentforum.org
• Barnum effect - Wikipediaen.wikipedia.org
• Practice As The Bar For Truth - Commoncogcommoncog.com
• How to Measure Anything — LessWronglesswrong.com
• Hawthorne effect - Wikipediaen.wikipedia.org
• Gwern's Most Important Writing (in essays, tweets, book reviews, and other forms) - Alexey Guzeyguzey.com