M* (M-star): A Modular, Extensible, Serving System for Multimodal Models
mstar.stanford.edu · 3,359 words · saved by 1 readers
Composite models broke the single-loop assumption behind LLM serving. The Walk Graph fixes it.
Inference is no longer a single loop LLM serving systems like vLLM and SGLang are built on one assumption: that inference is a single autoregressive loop — prefill the prompt, then decode one token at a time until the model stops. The newest multimodal models break that assumption. Five families make it concrete: UMMs — BAGEL SpeechLMs — Orpheus Omni — Qwen3-Omni VLAs — π0.5 World models — V-JEPA 2 They are composite: built from structurally distinct components — vision encoders, transformer backbones, diffusion and flow heads, audio codecs, action and world-model predictors — wired…
saved by
related reading
- M*: A Modular, Extensible, Serving System for Multimodal Modelsai.stanford.edu
- GLM-5.3-Flash: Frontier Intelligence, Flash Costz.ai
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- Inside vLLM: Anatomy of a High-Throughput LLM Inference Systemvllm.ai
- Better MoE model inference with warp decode · Cursorcursor.com
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Mamba: The Easy Wayjackcook.com
- LLM Engineer's Almanac - Workloads | Modalmodal.com
- Can LLMs Be Computers?percepta.ai
- The State of LLM Serving in 2026: Ollama, SGLang, TensorRT, Triton, and vLLM | Canteenthecanteenapp.com