Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blog
This blog is intended to be a place to share ideas and results that are too weird, incomplete, or off-topic to turn into an academic paper, but that I think may be important. Let me know what you think! Contact links to the left.
Increased efficiency can sometimes, counterintuitively, lead to worse outcomes. This is true almost everywhere. We will name this phenomenon the strong version of [Goodhart's law](https://en.wikipedia.org/wiki/Goodhart%27s_law). As one example, more efficient centralized tracking of student progress by standardized testing seems like such a good idea that well-intentioned laws [mandate it](https://en.wikipedia.org/wiki/No_Child_Left_Behind_Act). However, testing also incentivizes schools to focus more on teaching students to test well, and less on teaching broadly useful skills. As a result, i
saved by
related reading
- Goodhart's law - Wikipediaen.wikipedia.org
- Scaling Laws for Reward Model Overoptimizationarxiv.org
- Lil'Loglilianweng.github.io
- The hot mess theory of AI misalignment: More intelligent agents behave less coherently | Jascha’s blogsohl-dickstein.github.io
- Brain dump on the diversity of AI risk | Jascha’s blogsohl-dickstein.github.io
- Just Ask for Generalization | Eric Jangevjang.com
- Evolution as Backstop for Reinforcement Learning · Gwern.netgwern.net
- Scaling Laws, Carefully | Lil'Loglilianweng.github.io
- Overfitting in Machine Learning: What It Is and How to Prevent Itelitedatascience.com
- 4 – The Overfitting Iceberg – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
- Goodhart's Law — AI Alignment Forumalignmentforum.org
- Troubles with the Bias-Variance tradeoffejenner.com