Towards demystifying the creativity of diffusion models

We show that a diffusion model’s creativity (its ability to generate novel data, rather than just memorize its training set) is a mathematical consequence of neural networks learning a "smoothed" version of the score function, driving the model to interpolate between training data points along the hidden data manifold. Diffusion models are currently one of the most powerful types of tools for generative tasks that require complex and local structures, such as image generation and molecular discovery. They’ve shown an exciting capability to generalize beyond their training data and, in this sense, exhibit “creativity”. For instance, after being trained with datasets of actual images, they can transform random noise samples into novel, high-quality images. While this creative capability is impressive, it raises an intriguing question: where does it come from? Understanding the answer to this question is an important step towards demystifying the “black-box” nature of diffusion-based gene

Diffusion models are currently one of the most powerful types of tools for generative tasks that require complex and local structures, such as image generation and molecular discovery. They’ve shown an exciting capability to generalize beyond their training data and, in this sense, exhibit “creativity”. For instance, after being trained with datasets of actual images, they can transform random noise samples into novel, high-quality images. While this creative capability is impressive, it raises an intriguing question: where does it come from? Understanding the answer to this question is an…

saved by

• Benedict Neo

related reading

• Diffusion models from scratchchenyang.co
• What are Diffusion Models?lilianweng.github.io
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• ⭐️ Diffusion Modelsandrewkchan.dev
• [2006.11239] Denoising Diffusion Probabilistic Modelsarxiv.org
• Diffusion is spectral autoregression – Sander Dielemansander.ai
• Elucidating the Design Space of Diffusion-Based Generative Models | PDFarxiv.org
• Dynamical regimes of diffusion modelsnature.com
• Interpretable Diffusion via Information Decompositionarxiv.org
• [2010.02502] Denoising Diffusion Implicit Modelsarxiv.org
• Learning the integral of a diffusion model – Sander Dielemansander.ai
• https://arxiv.org/pdf/2006.11239arxiv.org