The Reparameterization Trick
gregorygundersen.com · 3,119 words · saved by 1 readers
Gregory Gundersen is a quantitative researcher in New York.
The Reparameterization Trick --> Home Blog RSS The Reparameterization Trick A common explanation for the reparameterization trick with variational autoencoders is that we cannot backpropagate through a stochastic node. I provide a more formal justification. Published 29 April 2018 Informal justifications In Auto-Encoding Variational Bayes , (Kingma & Welling, 2013) , Kingma presents an unbiased, differentiable, and scalable estimator for the ELBO in variational inference. A key idea behind this estimator is the reparameterization trick . But why do we need this trick in the first place? When f
related reading
- Tutorial - What is a variational autoencoder? – Jaan Lı 李jaan.io
- [1906.02691] An Introduction to Variational Autoencodersarxiv.org
- Gregory Gundersengregorygundersen.com
- Approximating KL Divergencejoschu.net
- 1312.6114.pdfarxiv.org
- Bayesian Neural Networkscs.toronto.edu
- ELBO — What & Whyyunfanj.com
- Policy Gradients Part 1: The REINFORCE Estimatorfa.bianp.net
- RescaledKL.tmdanilorezende.com
- Coordinate Ascent Mean-field Variational Inference (Univariate Gaussian Example)suzyahyah.github.io
- NeurIPS-2021-understanding-end-to-end-model-based-reinforcement-learning-methods-as-implicit-parameterization-Supplemental.pdflis.csail.mit.edu
- Log-derivative trickandrewcharlesjones.github.io