How trillion-parameter AI models work and how to serve them - YouTube
youtube.com · 10,630 words · saved by 2 readers
Follow along using the transcript. Follow along using the transcript.
How do you build and serve a trillion-parameter AI model? A deep dive using Kimi K3 as the case study, from Stable LatentMoE and Attention Residuals to hybrid KDA and MLA attention, plus how they're served in production. Philip, author of Inference Engineering, talks with Ali and Bryce from Baseten's model performance team. They cover the three problems trillion-parameter models like Kimi K3 solve at this scale: expert sparsity with Stable LatentMoE, million-token context with Attention Residuals, and depth with hybrid KDA and MLA attention. Ali walks through the history of attention on…
saved by
related reading
- How Kimi Trains 896 Experts Without Collapseyoutube.com
- ali (@waterloo_intern) on Xx.com
- What's New in Inference Engineering — Philip Kiely, Basetenyoutube.com
- Why Inference Matters 12x More Than Training | YC Paper Clubyoutube.com
- Stanford CS25: Transformers United V6 I Serving Transformers: Lessons from the Trenchesyoutube.com
- How GPT, Claude, and Gemini are actually trained and served – Reiner Popeyoutube.com
- Recursive Language Models — Alex Zhang, MIT PhDyoutube.com
- Big Boss (@0xBADB01E) on Xx.com
- How to Parallelize a Transformer for Training — an explorable explanationezyang.github.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Quantization from the ground upngrok.com
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io