Diffusion is spectral autoregression – Sander Dieleman

A deep dive into spectral analysis of diffusion models of images, revealing how they implicitly perform a form of autoregression in the frequency domain.

A bit of signal processing swiftly reveals that diffusion models and autoregressive models aren’t all that different: diffusion models of images perform approximate autoregression in the frequency domain! This blog post is also available as a Python notebook in Google Colab , with the code used to produce all the plots and animations. Last year, I wrote a blog post describing various different perspectives on diffusion . The idea was to highlight a number of connections between diffusion models and other classes of models and concepts. In recent months, I have given a few talks where I discuss

saved by

• Pranav
• Freeman Jiang
• Dron Hazra
• Thi Vũ
• Jacob G-W
• Sudarsh K
• Yudhister Joel Kumar
• Dhruv Sheth
• Kenneth Sun
• Alexiy Buynitsky
• Florent Tavernier
• Ishan Mukherjee

related reading

• Diffusion is not necessarily Spectral Autoregression | Fabian Falckfabianfalck.com
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• Diffusion Meets Flow Matchingdiffusionflow.github.io
• Generating music in the waveform domain – Sander Dielemansander.ai
• Diffusion language models – Sander Dielemansander.ai
• What are Diffusion Models?lilianweng.github.io
• [2404.02905] Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictionarxiv.org
• Generative modelling in latent space – Sander Dielemansander.ai
• Diffusion models from scratchchenyang.co
• Is Your Conditional Diffusion Model Actually Denoising?arxiv.org
• Diffusion model - Wikipediaen.wikipedia.org
• [M2L 2025] 5.2 Diffusion models - Sander Dielemanyoutube.com