Towards demystifying the creativity of diffusion models

We show that a diffusion model’s creativity (its ability to generate novel data, rather than just memorize its training set) is a mathematical consequence of neural networks learning a "smoothed" version of the score function, driving the model to interpolate between training data points along the hidden data manifold. Diffusion models are currently one of the most powerful types of tools for generative tasks that require complex and local structures, such as image generation and molecular discovery. They’ve shown an exciting capability to generalize beyond their training data and, in this sense, exhibit “creativity”. For instance, after being trained with datasets of actual images, they can transform random noise samples into novel, high-quality images. While this creative capability is impressive, it raises an intriguing question: where does it come from? Understanding the answer to this question is an important step towards demystifying the “black-box” nature of diffusion-based gene

Diffusion models are currently one of the most powerful types of tools for generative tasks that require complex and local structures, such as image generation and molecular discovery. They’ve shown an exciting capability to generalize beyond their training data and, in this sense, exhibit “creativity”. For instance, after being trained with datasets of actual images, they can transform random noise samples into novel, high-quality images. While this creative capability is impressive, it raises an intriguing question: where does it come from? Understanding the answer to this question is an…

saved by

• Benedict Neo

related reading

• What are Diffusion Models?lilianweng.github.io
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• ⭐️ Diffusion Modelsandrewkchan.dev
• Diffusion model - Wikipediaen.wikipedia.org
• Bare-bones Diffusion Modelsmadebyoll.in
• Diffusion models from scratchchenyang.co
• Yang Songyang-song.net
• Using Artificial Intelligence to Augment Human Intelligencedistill.pub
• Diffusion Models in AI – Everything You Need to Know – Unite.AIunite.ai
• Dynamical regimes of diffusion modelsnature.com
• [2006.11239] Denoising Diffusion Probabilistic Modelsarxiv.org
• The Illustrated Stable Diffusion – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io