Anas Aldadi on X: "We know VAEs try to match *Density values* using KL-Divergence, while score-matching try to match *Density gradients* using Fisher-Divergence but do you know that if you take KL and add Gaussian noise to it, the derivative of KL with respect to noise level is actually F-Div https://t.co/e6k6hhUhFT" / X

To view keyboard shortcuts, press question mark View keyboard shortcuts Home Explore 1 Notifications Chat Grok Bookmarks Creator Studio Premium 50% off Profile More Post Goober @oailyor Post See new posts Conversation Alexia Jolicoeur-Martineau reposted Anas Aldadi @leetm5n We know VAEs try to match *Density values* using KL-Divergence, while score-matching try to match *Density gradients* using Fisher-Divergence but do you know that if you take KL and add Gaussian noise to it, the derivative of KL with respect to noise level is actually F-Div 3:08 PM · May 3, 2026 · 21.4K Views 3 23 219 220 Relevant View quotes Post your reply Reply Anas Aldadi @leetm5n · May 3 Why this is important? while we think diffusion is doing variational inference while in truth it is minimizing Fisher-Div implicitly. [first image] bec, DDPMs take kl between 2 guassians (true reverse - learned reverse) [second image] and the mean of noised gaussian related... 1 1 11 1.3K Anas Aldadi @leetm5n · May 3 and

@leetm5n: We know VAEs try to match *Density values* using KL-Divergence, while score-matching try to match *Density gradients* using Fisher-Divergence but do you know that if you take KL and add Gaussian noise to it, the derivative of KL with respect to noise level is actually F-Div @leetm5n: Why this is important? while we think diffusion is doing variational inference while in truth it is minimizing Fisher-Div implicitly. [first image] bec, DDPMs take kl between 2 guassians (true reverse - learned reverse) [second image] and the mean of noised gaussian related... @leetm5n: and the mea

saved by

• Roy Liao

related reading

• What are Diffusion Models? | Lil'Loglilianweng.github.io
• Approximating KL Divergencejoschu.net
• Diffusion Models as a kind of VAE | Angus Turnerangusturner.github.io
• Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
• Sliced Score Matching: A Scalable Approach to Density and Score Estimation | Yang Songyang-song.github.io
• Kullback–Leibler divergence - Wikipediaen.wikipedia.org
• Short Notes on Divergence Measuresdanilorezende.com
• Generative Modeling by Estimating Gradients of the Data Distribution | Yang Songyang-song.github.io
• The Principles of Diffusion Modelsarxiv.org
• The Principles of Diffusion Modelsarxiv.org
• Andy Jonesandrewcharlesjones.github.io
• Stable Diffusion I — Mathematics Behind It | by Void | Mediummedium.com