The Reparameterization Trick

Gregory Gundersen is a quantitative researcher in New York.

The Reparameterization Trick --> Home Blog RSS The Reparameterization Trick A common explanation for the reparameterization trick with variational autoencoders is that we cannot backpropagate through a stochastic node. I provide a more formal justification. Published 29 April 2018 Informal justifications In Auto-Encoding Variational Bayes , (Kingma & Welling, 2013) , Kingma presents an unbiased, differentiable, and scalable estimator for the ELBO in variational inference. A key idea behind this estimator is the reparameterization trick . But why do we need this trick in the first place? When f

related reading

• Mediumandacdemir-94699.medium.com
• Tutorial - What is a variational autoencoder? – Jaan Lı 李jaan.io
• Variational autoencoder - Wikipediaen.wikipedia.org
• Bayesian Neural Networkscs.toronto.edu
• Difference between AutoEncoder (AE) and Variational AutoEncoder (VAE) | Towards Data Sciencetowardsdatascience.com
• Approximating KL Divergencejoschu.net
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• Diffusion Models as a kind of VAE | Angus Turnerangusturner.github.io
• From Autoencoder to Beta-VAE | Lil'Loglilianweng.github.io
• Variational autoencoders.jeremyjordan.me
• Eric Jang: A Beginner's Guide to Variational Methods: Mean-Field Approximationblog.evjang.com
• Anas Aldadi on X: "We know VAEs try to match *Density values* using KL-Divergence, while score-matching try to match *Density gradients* using Fisher-Divergence but do you know that if you take KL and add Gaussian noise to it, the derivative of KL with respect to noise level is actually F-Div https://t.co/e6k6hhUhFT" / Xx.com