M* (M-star): A Modular, Extensible, Serving System for Multimodal Models
mstar.stanford.edu · 3,359 words · saved by 1 readers
Composite models broke the single-loop assumption behind LLM serving. The Walk Graph fixes it.
Inference is no longer a single loop LLM serving systems like vLLM and SGLang are built on one assumption: that inference is a single autoregressive loop — prefill the prompt, then decode one token at a time until the model stops. The newest multimodal models break that assumption. Five families make it concrete: UMMs — BAGEL SpeechLMs — Orpheus Omni — Qwen3-Omni VLAs — π0.5 World models — V-JEPA 2 They are composite: built from structurally distinct components — vision encoders, transformer backbones, diffusion and flow heads, audio codecs, action and world-model predictors — wired…
saved by
related reading
- M*: A Modular, Extensible, Serving System for Multimodal Modelsai.stanford.edu
- LLM Engineer's Almanac - Advisormodal.com
- GLM-5.3-Flash: Frontier Intelligence, Flash Costz.ai
- Inkling: Our Open-Weights Model - Thinking Machines Labthinkingmachines.ai
- Composer2.pdfcursor.com
- 2403.09611.pdfarxiv.org
- vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention | vLLM Blogblog.vllm.ai
- Model Runner V2: A Modular and Faster Core for vLLM | vLLM Blogvllm.ai
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Inside vLLM: Anatomy of a High-Throughput LLM Inference Systemvllm.ai
- Machine Learning System Resources | std::bodun::blogbodunhu.com
- laguna-m1-xs2-technical-report.pdfpoolside.ai