Minimum description length

Minimum description length (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data.

Minimum description length - Wikipedia Jump to content From Wikipedia, the free encyclopedia Model selection principle Minimum description length ( MDL ) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor . The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data. MDL has its origins mo

saved by

• Ann He
• Lydia Nottingham
• Evie Hu

related reading

• Solomonoff's theory of inductive inference - Wikipediaen.wikipedia.org
• Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
• Compression and Intelligencegreene.sh
• Minimax - Wikipediaen.wikipedia.org
• Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs — LessWronglesswrong.com
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• [2301.12987] The Optimal Choice of Hypothesis Is the Weakest, Not the Shortestarxiv.org
• "Occam"-style Bounds for Long Programsbactra.org
• Manifold learning: what, how, and whyarxiv.org
• Bayesian programming - Wikipediaen.wikipedia.org
• https://arxiv.org/pdf/1811.12808arxiv.org
• The necessity of machine learning theory in mitigating AI riskmishabelkin.substack.com