Approximating KL Divergence

joschu.net · 1,073 words · saved by 11 readers

This post is about Monte-Carlo approximations of KL divergence. 𝐾 𝐿 [ 𝑞 , 𝑝 ] = ∑ 𝑥 𝑞 ( 𝑥 ) log ⁡ 𝑞 ( 𝑥 ) 𝑝 ( 𝑥 ) = 𝐸 𝑥 ∼ 𝑞 [ log ⁡ 𝑞 ( 𝑥 ) 𝑝 ( 𝑥 ) ] KL[q,p]= x ∑ q(x)log p(x) q(x) =E x∼q [log p(x) q(x) ] It explains a trick I’ve used in various code, where I approximate K L [ 𝑞 , 𝑝 ] KL[q,p] as a sample average of 1 2 ( log ⁡ 𝑝 ( 𝑥 ) − log ⁡ 𝑞 ( 𝑥 ) ) 2 2 1 (logp(x)−logq(x)) 2 , for samples 𝑥 x from 𝑞 q, rather the more standard log ⁡ 𝑞 ( 𝑥 ) 𝑝 ( 𝑥 ) log p(x) q(x) . This post will explain why this expression is a good (though biased) estimator of KL, and how to make it unbiased while preserving its low variance. Our options for computing KL depend on what kind of access we have to 𝑝 p and 𝑞 q. Here, we’ll be assuming that we can compute the probabilities (or probability densities) 𝑝 ( 𝑥 ) p(x) and 𝑞 ( 𝑥 ) q(x) for any 𝑥 x, but we can’t calculate the sum over 𝑥 x analytically. Why wouldn’t we be able to calculate

Approximating KL Divergence John Schulman's Homepage Approximating KL Divergence Posted on 2020/03/07 ← back to blog index \gdef\ratio{\tfrac{p(x)}{q(x)}} \gdef\iratio{\tfrac{q(x)}{p(x)}} \gdef\half{\tfrac{1}{2}} \gdef{\klqp}{\mathrm{KL}[q,p]} \gdef{\klpq}{\mathrm{KL}[p,q]} This post is about Monte-Carlo approximations of KL divergence. KL[q, p] = \sum_x q(x) \log \iratio = E_{ x \sim q}[\log \iratio ] It explains a trick I’ve used in various code, where I approximate \klqp as a sample average of \half (\log p(x) - \log q(x))^2 , for samples x from q , rather the more standard \log \frac{q(x)}

Approximating KL Divergence

saved by

related reading