Visual Information Theory -- colah's blog
I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this.
Visual Information Theory -- colah's blog Visual Information Theory Posted on October 14, 2015 I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this. Information theory gives us precise language for describing a lot of things. How uncertain am I? How much does knowing the answer to question A tell me about the answer to question B? How similar is one set of beliefs to another? I’ve had informal versions of these ideas since I was a young child, b
Explore this link on the map →related reading
- Visual Information Theory -- colah's blogcolah.github.io
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- Information entropy (Foundations of information theory: Part 2) - Matthew N. Bernsteinmbernste.github.io
- book.pdfinference.org.uk
- blog.alexalemi.com Why KL?blog.alexalemi.com
- [physics/0004057] 1 Introductionar5iv.labs.arxiv.org
- Cross-entropy | Anıl Zenginoğluanilzen.github.io
- A Gentle Introduction to Cross-Entropy for Machine Learning - MachineLearningMastery.commachinelearningmastery.com
- A Maximum Entropy Intuition for Fundamental Statistical Distributions | long intuitionlongintuition.com
- Conditional entropy - Wikipediaen.wikipedia.org