✳flâneur — a map of the web's best reading

fast.ai - The problem with metrics is a big problem for AI

fast.ai · 1,978 words · saved by 1 readers

Unthinkingly optimizing metrics can lead to a variety of grave harms, and what most current AI approaches do is to optimize metrics.

The problem with metrics is a big problem for AI – fast.ai Update: This post was expanded into a paper, The Problem with Metrics is a Fundamental Problem for AI , by Rachel Thomas and David Uminsky, which was accepted to the Ethics of Data Science Conference 2020 . The paper version includes more grounding in previous academic work and a framework towards mitigating these harms. Goodhart’s Law states that “When a measure becomes a target, it ceases to be a good measure.” At their heart, what most current AI approaches do is to optimize metrics. The practice of optimizing metrics is not new nor

Explore this link on the map →

related reading