[2302.07121] Universal Guidance for Diffusion Models
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack
View PDF HTML (experimental) Abstract:Typical diffusion models are trained to accept a particular form of conditioning, most commonly text, and cannot be conditioned on other modalities without retraining. In this work, we propose a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain any use-specific components. We show that our algorithm successfully generates quality images with guidance functions including segmentation, face recognition, object detection, and classifier signals. Code is available at this…
saved by
related reading
- openaccess.thecvf.com/content/ICCV2023/papers/Li_Your_Diffusion_Model_is_Secretly_a_Zero-Shot_Classifier_ICCV_2023_paper.pdfopenaccess.thecvf.com
- Total Selfie: Generating Full-Body Selfiesarxiv.org
- Scalable Diffusion Models with Transformersarxiv.org
- What are Diffusion Models?lilianweng.github.io
- [2310.08586] PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmarxiv.org
- [2312.03079] LooseControl: Lifting ControlNet for Generalized Depth Conditioningarxiv.org
- [2406.11717] Refusal in Language Models Is Mediated by a Single Directionarxiv.org
- [2006.11239] Denoising Diffusion Probabilistic Modelsarxiv.org
- Diffusion models from scratchchenyang.co
- Large Language Diffusion Modelsarxiv.org
- https://arxiv.org/pdf/2006.11239arxiv.org
- Diffusion will be everywherevarunneal.github.io