Mixture-of-Kittens: our open-source MoE megakernel for NVL72s · Cursor

We're open-sourcing Mixture-of-Kittens, a deterministic MoE training megakernel for NVL72s that fuses communication and computation into a single kernel.

Today, we're open-sourcing Mixture-of-Kittens (MoK), our production MoE training megakernel for NVL72s. As we have scaled the training and inference of Composer, our agentic coding model, the mixture-of-experts layer has consistently remained the major bottleneck. Depending on the workload and training configuration, it can consume more than half of end-to-end training time. MoK addresses that bottleneck by fusing all MoE communication and computation into a single, fully deterministic kernel. It now powers Composer training across tens of thousands of GPUs. You can try MoK and explore…

saved by

• Devin Plumb
• Chris Shi
• Arya G

related reading

• DeepSeek-V3: A Large-Scale MoE Pretraining Benchmark for MLPerf Training v6.0mlcommons.org
• Better MoE model inference with warp decode · Cursorcursor.com
• Mixture of Experts Quantile Balancing: Validated at 32B-A5B (1e22 FLOPs) Scaleopenathena.ai
• Composer2.pdfcursor.com
• Transformer Inference Arithmetic | kipply's blogkipp.ly
• 1.5x faster MoE training with custom MXFP8 kernels · Cursorcursor.com
• Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
• Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
• Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing Tensor Programsarxiv.org
• KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu
• Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)blog.kog.ai
• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co