Model Runner V2: A Modular and Faster Core for vLLM | vLLM Blog
We are excited to announce Model Runner V2 (MRV2), a ground-up re-implementation of the vLLM model runner. MRV2 delivers a cleaner, more modular, and more effic
Table of Contents We are excited to announce Model Runner V2 (MRV2) , a ground-up re-implementation of the vLLM model runner. MRV2 delivers a cleaner, more modular, and more efficient execution core—with no API changes . The goal is simple: better code and better performance. Like the vLLM V1 release last year, this is an architectural upgrade driven by hard-earned lessons from vLLM's large user base and feedback from the community. We revisited persistent batching, async scheduling, input preparation, and sampling, then rebuilt the model runner around three core principles: Be modular. Isolat
saved by
related reading
- Defeating Nondeterminism in LLM Inference - Thinking Machines Labthinkingmachines.ai
- Composer2.pdfcursor.com
- Inside vLLM: Anatomy of a High-Throughput LLM Inference Systemvllm.ai
- 2025 LLM Year in Review – karpathykarpathy.bearblog.dev
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Introduction to vLLM and PagedAttentionblog.runpod.io
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- GLM-5.2: Built for Long-Horizon Tasksz.ai
- vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention | vLLM Blogblog.vllm.ai
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- The State of LLM Serving in 2026: Ollama, SGLang, TensorRT, Triton, and vLLM | Canteenthecanteenapp.com
- structured decoding, a guide for the impatientaarnphm.xyz