UNITE | Project Page

UNITE: End-to-End Training for Unified Tokenization and Latent Denoising Shivam Duggal*, Xingjian Bai*, Zongze Wu, Richard Zhang, Eli Shechtman, Antonio Torralba, Phillip Isola, William T. Freeman Training state-of-the-art latent diffusion models requires complex staging: a tokenizer must first be trained before a diffusion model can operate in its frozen latent space. We propose UNITE — an arc

UNITE: End-to-End Training for Unified Tokenization and Latent Denoising Shivam Duggal*, Xingjian Bai*, Zongze Wu, Richard Zhang, Eli Shechtman, Antonio Torralba, Phillip Isola, William T. Freeman Training state-of-the-art latent diffusion models requires complex staging: a tokenizer must first be trained before a diffusion model can operate in its frozen latent space. We propose UNITE — an architecture for unified tokenization and latent diffusion. A single Generative Encoder serves as both image tokenizer and latent generator via weight sharing, trained in a single stage that jointly…

related reading

• What are Diffusion Models? | Lil'Loglilianweng.github.io
• Unified Multimodal Models as Auto-Encodersarxiv.org
• Fast Training of Diffusion Models with Masked Transformers | PDFarxiv.org
• Together AI | The AI Native Cloudtogether.ai
• What are Diffusion Models?lilianweng.github.io
• [2006.11239] Denoising Diffusion Probabilistic Modelsarxiv.org
• [2605.05331] ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parametersarxiv.org
• Scalable Diffusion Models with Transformersopenaccess.thecvf.com
• https://arxiv.org/pdf/2006.11239arxiv.org
• Generative modelling in latent space – Sander Dielemansander.ai
• An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversiontextual-inversion.github.io
• Introducing Un-0: Generating Images with Coupled Oscillatorsunconv.ai