Dynamical regimes of diffusion models | Nature Communications

Diffusion methods are widely used for generating data in AI applications. Here, authors show that optimally trained diffusion models exhibit three dynamical regimes: starting from pure noise, they reach a regime where the main data class is sealed, and finally collapse onto one training point.

Introduction Machine learning has recently witnessed thrilling advancements, especially in the realm of generative models. At the forefront of this progress are diffusion models (DMs), which have emerged as powerful tools for modeling complex data distributions and generating new realistic samples. They have become the state of the art in generating images, videos, audio or 3D scenes1,2,3,4,5,6,7,8. Although the practical success of generative DMs is widely recognized, their full theoretical understanding remains an open challenge. Rigorous results assessing their convergence on…

saved by

• Vihan Tiwari

related reading

• What are Diffusion Models? | Lil'Loglilianweng.github.io
• What are Diffusion Models?lilianweng.github.io
• Diffusion model - Wikipediaen.wikipedia.org
• ⭐️ Diffusion Modelsandrewkchan.dev
• Diffusion models from scratchchenyang.co
• The Principles of Diffusion Modelsarxiv.org
• The Principles of Diffusion Modelsarxiv.org
• [2010.02502] Denoising Diffusion Implicit Modelsarxiv.org
• Diffusion is spectral autoregression – Sander Dielemansander.ai
• Yang Songyang-song.net
• Towards demystifying the creativity of diffusion modelsresearch.google
• Step-by-Step Diffusion: An Elementary Tutorialarxiv.org