Andy Jones

The “log-derivative trick” is really just a simple application of the chain rule. However, it allows us to rewrite expectations in a way that is amenable to Monte Carlo approximation. Suppose we have a function p ( x ; θ ) 𝑝 ( 𝑥 ; 𝜃 ) (in this context we’ll mostly think of p 𝑝 as a probability density), and we’d like to take the gradient of its logarithm with respect to θ 𝜃 , By a simple application of the chain rule, we have which, rearranging, implies that In many statistical applications, we want to estimate the gradient of an expectation of a function f 𝑓 : To learn more about a few applications where this gradient estimation problem shows up, as well as more modern methods for solving it, I’d recommend this review by Shakir Mohamed et al. Unfortunately, we cannot directly approximate this expression with naive Monte Carlo methods. This is because the expression isn’t in general an expectation. Expanding the expectation we have: ∇ θ E p ( x ; θ ) [ f ( x ) ] = ∇ θ ∫ p

saved by

• Pranav Jadhav