flâneur

Six (and a half) intuitions for KL divergence - LessWrong

lesswrong.com · 7,569 words · saved by 15 readers

KL-divergence is a topic which crops up in a ton of different places in information theory and machine learning, so it's important to understand well…

x Six (and a half) intuitions for KL divergence — LessWrong Information theory Probability & Statistics Machine Learning (ML) World Modeling Frontpage 185 Six (and a half) intuitions for KL divergence by CallumMcDougall 12th Oct 2022 Linkpost for www.perfectlynormal.co.uk 12 min read 27 185 KL-divergence is a topic which crops up in a ton of different places in information theory and machine learning, so it's important to understand well. Unfortunately, it has some properties which seem confusing at a first pass (e.g. it isn't symmetric like we would expect from most distance measures, and it

saved by

related reading