Rao-Blackwellization and discrete parameters in Stan | Statistical Modeling, Causal Inference, and Social Science

I’m reading a really dense and beautifully written survey of Monte Carlo gradient estimation for machine learning by Shakir Mohamed, Mihaela Rosca, Michael Figurnov, and Andriy Mnih. There are great explanations of everything including variance reduction techniques like coupling, control variates, and Rao-Blackwellization. The latter’s the topic of today’s post, as it relates directly to current Stan practices. Expecations of interest In Bayesian inference, parameter estimates and event probabilities and predictions can all be formulated as expectations of functions of parameters conditioned on observed data. In symbols, that’s $latex \displaystyle \mathbb{E}[f(\Theta) \mid Y = y] = \int f(\theta) \cdot p(\theta \mid y) \, \textrm{d}\theta$ for a model with parameter vector $latex \Theta$ and data $latex Y = y.$ In this post and most writing about probability theory, random variables are capitalized and bound variables are not. Partitioning variables Suppose we have two random variable

saved by

• Leo Ware