fast.ai - The problem with metrics is a big problem for AI
fast.ai · 1,978 words · saved by 1 readers
Unthinkingly optimizing metrics can lead to a variety of grave harms, and what most current AI approaches do is to optimize metrics.
The problem with metrics is a big problem for AI – fast.ai Update: This post was expanded into a paper, The Problem with Metrics is a Fundamental Problem for AI , by Rachel Thomas and David Uminsky, which was accepted to the Ethics of Data Science Conference 2020 . The paper version includes more grounding in previous academic work and a framework towards mitigating these harms. Goodhart’s Law states that “When a measure becomes a target, it ceases to be a good measure.” At their heart, what most current AI approaches do is to optimize metrics. The practice of optimizing metrics is not new nor
related reading
- Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blogsohl-dickstein.github.io
- What failure looks like — LessWronglesswrong.com
- On measuring AInikilravi.substack.com
- The Artificiality of Alignmentjoinreboot.org
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- What failure looks like — AI Alignment Forumalignmentforum.org
- Existential Risk from AI: An Exposition for Mathematiciansalkjash.github.io
- Scaling Laws Do Not Scalearxiv.org
- Current AIs seem pretty misaligned to meblog.redwoodresearch.org
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- AI Safety Seems Hard to Measurecold-takes.com
- Why are AI agents lying, cheating and coordinating?yoshuabengio.org