Approximating KL Divergence

This post is about Monte-Carlo approximations of KL divergence. 𝐾 𝐿 [ 𝑞 , 𝑝 ] = ∑ 𝑥 𝑞 ( 𝑥 ) log ⁡ 𝑞 ( 𝑥 ) 𝑝 ( 𝑥 ) = 𝐸 𝑥 ∼ 𝑞 [ log ⁡ 𝑞 ( 𝑥 ) 𝑝 ( 𝑥 ) ] KL[q,p]= x ∑ ​ q(x)log p(x) q(x) ​ =E x∼q ​ [log p(x) q(x) ​ ] It explains a trick I’ve used in various code, where I approximate K L [ 𝑞 , 𝑝 ] KL[q,p] as a sample average of 1 2 ( log ⁡ 𝑝 ( 𝑥 ) − log ⁡ 𝑞 ( 𝑥 ) ) 2 2 1 ​ (logp(x)−logq(x)) 2 , for samples 𝑥 x from 𝑞 q, rather the more standard log ⁡ 𝑞 ( 𝑥 ) 𝑝 ( 𝑥 ) log p(x) q(x) ​ . This post will explain why this expression is a good (though biased) estimator of KL, and how to make it unbiased while preserving its low variance. Our options for computing KL depend on what kind of access we have to 𝑝 p and 𝑞 q. Here, we’ll be assuming that we can compute the probabilities (or probability densities) 𝑝 ( 𝑥 ) p(x) and 𝑞 ( 𝑥 ) q(x) for any 𝑥 x, but we can’t calculate the sum over 𝑥 x analytically. Why wouldn’t we be able to calculate

saved by

• stefano viel