Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models
ArXiv Preprint Source Code Github Trained Concept Sliders Demo Colab: Sliders Inference Huggingface Demo Artists spend significant time crafting prompts and finding seeds to generate a desired image with text-to-image models. However, they need more nuanced, fine-grained control over attribute strengths like eye size or lighting in their generated images. Modifying the prompt disrupts overall structure. Artists require expressive control that maintains coherence. To enable precise editing without changing structure, we present Concept Sliders that are plug-and-play low rank adaptors applied on top of pretrained models. By using simple text descriptions or a small set of paired images, we train concept sliders to represent the direction of desired attributes. At generation time, these sliders can be used to control the strength of the concept in the image, enabling nuanced tweaking. The ability to precisely modulate semantic concepts during image generation and editing unlocks new f
Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models Rohit Gandikota *1 , Joanna Materzyńska 2 , Tingrui Zhou 3 , Antonio Torralba 2 , David Bau 1 1 Northeastern University , 2 MIT CSAIL , 3 Independent Contributor European Conference on Computer Vision (ECCV 2024) Update! See our SliderSpace that automatically extracts 100s of sliders from a single prompt. No additional supervision required! SliderSpace: Decomposing the Visual Capabilities of Diffusion Models ArXiv Preprint Source Code Github Trained
Explore this link on the map →related reading
- What are Diffusion Models? | Lil'Loglilianweng.github.io
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversiontextual-inversion.github.io
- The Illustrated Stable Diffusion – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- ⭐️ Diffusion Modelsandrewkchan.dev
- Editing Text in Images with AI | Towards Data Sciencetowardsdatascience.com
- Replicate - Run AI with an APIreplicate.com
- How does Stable Diffusion work?stable-diffusion-art.com
- Large Language Diffusion Modelsarxiv.org
- Diffusion models from scratchchenyang.co
- Yang Songyang-song.net
- ComfyUI_Workflows/image_conditioning/README.md at main · cubiq/ComfyUI_Workflows · GitHubgithub.com
- Bare-bones Diffusion Modelsmadebyoll.in