Short Notes on Divergence Measures

N/A

# link_2521brwbqgz.pdf ## Metadata - PDFFormatVersion=1.4 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Title=Short Notes on Divergence Measures - Author=Danilo Jimenez Rezende - Creator=TeXmacs 1.99.5 - Producer=TeXmacs 1.99.5 + Hummus ## Contents ### Page 1 Short Notes on Divergence MeasuresDanilo Jimenez Rezende1. IntroThe problem of density estimation is at the foundation of basic sciences, statistics and infor- mation theory. Not surprisingly, this problem receives substantial attention from the ML and DL c

saved by

• Eliezer de Souza da Silva
• Fernando Silva

related reading

• Kullback–Leibler divergence - Wikipediaen.wikipedia.org
• Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
• Approximating KL Divergencejoschu.net
• blog.alexalemi.com Why KL?blog.alexalemi.com
• Visual Information Theory -- colah's blogcolah.github.io
• Andy Jonesandrewcharlesjones.github.io
• Anas Aldadi on X: "We know VAEs try to match *Density values* using KL-Divergence, while score-matching try to match *Density gradients* using Fisher-Divergence but do you know that if you take KL and add Gaussian noise to it, the derivative of KL with respect to noise level is actually F-Div https://t.co/e6k6hhUhFT" / Xx.com
• An Introduction to Fisher Information – Awni Hannun – Writing About Machine Learningawni.github.io
• Exponential families from a single KL identityarxiv.org
• Lewis Smith - A gentle introduction to information geometryrobots.ox.ac.uk
• blog.alexalemi.com KL is All You Needblog.alexalemi.com
• Pinsker's inequality - Wikipediaen.wikipedia.org