Minimum description length
Minimum description length (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data.
Minimum description length - Wikipedia Jump to content From Wikipedia, the free encyclopedia Model selection principle Minimum description length ( MDL ) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor . The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data. MDL has its origins mo
saved by
related reading
- Solomonoff's theory of inductive inference - Wikipediaen.wikipedia.org
- Compression and Intelligencegreene.sh
- Scaling Laws, Carefully | Lil'Loglilianweng.github.io
- [2309.10668] Language Modeling Is Compressionarxiv.org
- Gregory Gundersengregorygundersen.com
- Compression is predictionngrok.com
- [2301.12987] The Optimal Choice of Hypothesis Is the Weakest, Not the Shortestarxiv.org
- MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
- deeplearningbook.org/contents/ml.htmldeeplearningbook.org
- Occam’s Razorreadthesequences.com
- Linear Regressionmlu-explain.github.io
- Rediscovery of Interiority in MLdanburfoot.net