Lewis Smith - A gentle introduction to information geometry

The following post is a slightly modified version of some notes I made for an internal reading group at Oxford. I actually gave the tutorial ages ago, but I finally got around to cleaning it up and putting it online. In brief, information geometry is the study of the geometrical structure of families of probability distributions. The field thus brings together statistics, information theory and differential geometry, revealing some fascinating and unexpected connections between them. This tutorial attempts to fit a flavour of a fairly technical and abstract field into a brief tutorial that assumes no knowledge of differential geometry. Therefore, I have chosen intuition over rigour whenever I felt it appropriate, for which I make no apologies, and only introduced the absolute minimum of background. I skip over a lot of very interesting things to try to get to the information metric as quickly as possible. There is a list of further reading in the conclusion for those who are interested

Posted on September 27, 2019 The following post is a slightly modified version of some notes I made for an internal reading group at Oxford. I actually gave the tutorial ages ago, but I finally got around to cleaning it up and putting it online. Introduction In brief, information geometry is the study of the geometrical structure of families of probability distributions. The field thus brings together statistics, information theory and differential geometry, revealing some fascinating and unexpected connections between them. This tutorial attempts to fit a flavour of a fairly technical and abs

saved by

• Ashwin Reddy

related reading

• An Elementary Introduction to Information Geometryfranknielsen.github.io
• Visual Information Theory -- colah's blogcolah.github.io
• Kullback–Leibler divergence - Wikipediaen.wikipedia.org
• An Introduction to Fisher Information – Awni Hannun – Writing About Machine Learningawni.github.io
• gstom.pdfmath.umd.edu
• Short Notes on Divergence Measuresdanilorezende.com
• Information Geometry (Part 16) | Azimuthjohncarlosbaez.wordpress.com
• Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
• The Fast Track – Sheafificationsheafification.com
• Differential geometry of MLresearch.fal.ai
• Napkin.pdfvenhance.github.io
• Koszul lecture related to geometric and analytic mechanics, Souriau’s Lie group thermodynamics and information geometry | Information Geometry | Springer Nature Linklink.springer.com