Visual Information Theory -- colah's blog
I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this.
Visual Information Theory -- colah's blog Visual Information Theory Posted on October 14, 2015 I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this. Information theory gives us precise language for describing a lot of things. How uncertain am I? How much does knowing the answer to question A tell me about the answer to question B? How similar is one set of beliefs to another? I’ve had informal versions of these ideas since I was a young child, b
related reading
- Visual Information Theory -- colah's blogcolah.github.io
- [2206.07867] A visual introduction to information theoryarxiv.org
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- Visualizing Algorithmsbost.ocks.org
- Reinventing Entropy | Compression is Intelligence Part 1youtube.com
- www.deeplearningbook.orgdeeplearningbook.org
- A Maximum Entropy Intuition for Fundamental Statistical Distributions | long intuitionlongintuition.com
- Why Information Theory is Important - Computerphileyoutube.com
- blog.alexalemi.com Why KL?blog.alexalemi.com
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- Sending Samples Without Bits-Backjoschu.net
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org