M*: A Modular, Extensible, Serving System for Multimodal Models | SAIL Blog
ai.stanford.edu · 3,526 words · saved by 1 readers
Stanford University · University of Washington · Correspondence: atindra@cs.stanford.edu
Atindra Jha, Naomi Sagan, Keisuke Kamahori, Xikai(Noah) Meng, Luke Zettlemoyer, Olivia Hsu, Jure Leskovec, Baris Kasikci, Stephanie Wang June 15, 2026 Stanford University · University of Washington · Correspondence: atindra@cs.stanford.edu Read the paper (arXiv) · Code (GitHub) · Docs Today’s models no longer fit the mold of autoregressive token generation, but the systems supporting LLM inference have not kept up. These models have composite architectures best captured by dataflow graphs. Requests are just walks on these graphs. M* is designed to fit this paradigm and maximize…
saved by
related reading
- M*: One Serving System for Any-to-Any Multimodal Modelsmstar.stanford.edu
- Interaction Models: A Scalable Approach to Human-AI Collaboration - Thinking Machines Labthinkingmachines.ai
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- 2403.09611.pdfarxiv.org
- Better MoE model inference with warp decode · Cursorcursor.com
- Mamba: The Easy Wayjackcook.com
- LLM Engineer's Almanac - Workloads | Modalmodal.com
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- [2605.15156] MeMo: Memory as a Modelarxiv.org
- MMHal Bencharxiv.org
- Fast Inference from Transformers via Speculative Decodingarxiv.org
- Inside vLLM: Anatomy of a High-Throughput LLM Inference Systemvllm.ai