Visual Information Theory -- colah's blog
I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this.
Visual Information Theory -- colah's blog Visual Information Theory Posted on October 14, 2015 I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into a concrete concept. Information theory is a prime example of this. Information theory gives us precise language for describing a lot of things. How uncertain am I? How much does knowing the answer to question A tell me about the answer to question B? How similar is one set of beliefs to another? I’ve had informal versions of these ideas since I was a young child, b
related reading
- Visual Information Theory -- colah's blogcolah.github.io
- [2206.07867] A visual introduction to information theoryarxiv.org
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- notes.pdfcourses.maths.ox.ac.uk
- book.pdfinference.org.uk
- Information entropy (Foundations of information theory: Part 2) - Matthew N. Bernsteinmbernste.github.io
- blog.alexalemi.com Why KL?blog.alexalemi.com
- How Claude Shannon Invented the Future | Quanta Magazinequantamagazine.org
- www.deeplearningbook.orgdeeplearningbook.org
- [physics/0004057] 1 Introductionar5iv.labs.arxiv.org