Visual Information Theory -- colah's blog
I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this.
Visual Information Theory -- colah's blog Visual Information Theory Posted on October 14, 2015 I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this. Information theory gives us precise language for describing a lot of things. How uncertain am I? How much does knowing the answer to question A tell me about the answer to question B? How similar is one set of beliefs to another? I’ve had informal versions of these ideas since I was a young child, b
saved by
- Sam Brook
- Winnie Xu
- Anson Yu
- Klaudia U.
- Ellen Xu
- Justin Wang
- Viraj Cz
- Freeman Jiang
- Marley Xiong
- Raffi Hotter
- Aaron Pham
- Alex Becker
related reading
- Visual Information Theory -- colah's blogcolah.github.io
- [2206.07867] A visual introduction to information theoryarxiv.org
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- book.pdfinference.org.uk
- How Claude Shannon Invented the Future | Quanta Magazinequantamagazine.org
- notes.pdfcourses.maths.ox.ac.uk
- Information entropy (Foundations of information theory: Part 2) - Matthew N. Bernsteinmbernste.github.io
- blog.alexalemi.com Why KL?blog.alexalemi.com
- Biology as Information Dynamicsmath.ucr.edu
- Gibbs' inequality - Wikipediaen.wikipedia.org