Fast-dLLM
Chengyue Wu1,2, Hao Zhang2, Shuchen Xue2, Shizhe Diao2, Yonggan Fu2, Zhijian Liu2, Pavlo Molchanov2, Ping Luo1, Song Han2,3, Enze Xie2 1HKU, 2NVIDIA, 3MIT Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks. However, their inherent sequential decoding limits inference efficiency. In this work, we propose Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that transforms pretrained AR models into diffusion-style decoders for parallel text generation. Our approach introduces a novel decoding recipe incorporating the block diffusion mechanism and complementary attention mask, which together enable blockwise bidirectional context modeling while preserving the original AR training objectives and performance. To further enhance inference speed, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations and a sub-block level cache that suppo
Fast-dLLM v2 Fast-dLLM v2: Efficient Block-Diffusion Large Language Model Accepted by ICLR 2026 Chengyue Wu 1,2 , Hao Zhang 2 , Shuchen Xue 2 , Shizhe Diao 2 , Yonggan Fu 2 , Zhijian Liu 2 , Pavlo Molchanov 2 , Ping Luo 1 , Song Han 2,3 , Enze Xie 2 1 HKU, 2 NVIDIA, 3 MIT --> --> Paper Code Model Fast-dVLM Your browser does not support the video tag. Realtime throughput comparison between Fast-dLLM v2 and Qwen2.5-7B-Instruct. About Fast-dLLM v2 Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent
Explore this link on the map →related reading
- Large Language Diffusion Modelsarxiv.org
- Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusionarxiv.org
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Speculative Decoding - philkravphilkrav.com
- Optimizing inference · Hugging Facehuggingface.co
- GenAI Handbookgenai-handbook.github.io
- Esoteric Language Modelsarxiv.org
- 2506.17298arxiv.org
- How to make LLMs go fastvgel.me
- vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention | vLLM Blogblog.vllm.ai
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com