A Maximum Entropy Intuition for Fundamental Statistical Distributions | long intuition
longintuition.com · 3,149 words · saved by 1 readers
Imagine you wake up tomorrow in an empty white room, a la The Matrix. You don’t remember how you got there. Anything can happen.
Imagine you wake up tomorrow in an empty white room, a la The Matrix. You don’t remember how you got there. Anything can happen. If Keanu Reeves appears, you’d probably think this is related in some way to the Matrix. If Keanu appears and then Laurence Fishburne (Morpheus) appears, you’d think—okay, this is almost definitely related to the Matrix. On the other hand, if Obama and Clinton appear in your white room, in your head, you’d think, okay, the Matrix-related possibilities are less likely; the politics-related possibilities are more likely, whatever those are. For the first few seconds in
saved by
related reading
- Visual Information Theory -- colah's blogcolah.github.io
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- Maximum Entropy Distributions | Bounded Rationalitybjlkeng.github.io
- [2206.07867] A visual introduction to information theoryarxiv.org
- Maximum Entropy Methods (MaxEnt)bactra.org
- Principle of maximum entropy - Wikipediaen.wikipedia.org
- How much do you believe your results? — LessWronglesswrong.com
- Visual Information Theory -- colah's blogcolah.github.io
- Understanding Momentsgregorygundersen.com
- Information entropy (Foundations of information theory: Part 2) - Matthew N. Bernsteinmbernste.github.io
- Introduction to abstract entropy — LessWronglesswrong.com
- Natural Latents: The Math — LessWronglesswrong.com