A Simplified Overview of Langevin Dynamics | Roy Friedman
An overview of Langevin dynamics (or sampling), with a focus on building up intuition for how it works, when it works, and what can be done to make it work when it doesn't.
Langevin dynamics (or sampling) is one of the most popular Markov chain Monte Carlo (MCMC) methods out there. It is used for countless tasks that require sampling from a distribution, and is even really simple to use especially since automatic differentiation is easily accessible. While straightforward to implement, I think that it’s hard to build an intuition for what to expect without seeing toy examples. In this post, I want to try and build the intuition for the sampling procedure itself, the stationary distribution that will be reached, and how to mitigate possible problems with the…
saved by
related reading
- An Introduction to Flow Matching and Diffusion Modelsdiffusion.csail.mit.edu
- Why Momentum Really Worksdistill.pub
- Approximating KL Divergencejoschu.net
- Diffusion models from scratchchenyang.co
- Learning the integral of a diffusion model – Sander Dielemansander.ai
- A Gallery of Methods Beyond RL — Part I: Sampling Methodsshengyu-feng.github.io
- Sampling: Two Basic Algorithmsgregorygundersen.com
- What are Diffusion Models? | Lil'Loglilianweng.github.io
- Step-by-Step Diffusion: An Elementary Tutorialarxiv.org
- An optimization perspective on log-concave sampling and beyond | Sinho Chewichewisinho.github.io
- [2505.02621] Mirror Mean-Field Langevin Dynamicsarxiv.org
- Generative Modeling by Estimating Gradients of the Data Distribution | Yang Songyang-song.github.io