Papers Explained 577: MAI-Thinking-1 | by Ritvik Rastogi | Medium
MAI-Thinking-1 is a large reasoning model with 35B active/1T parameters, trained from scratch, on 30T tokens of exclusively clean, non-synthetic, human-generated data, with no distillation from third-party or open-source models. Its development follows three principles: The process includes successive improvements in architecture, data, and reinforcement learning optimized for sustained, log-linear gains. MAI-Base-1 uses a decoder-only Transformer. Each layer features RMSNorm at both the input and output, right before residual addition. The model does not use any biases. Input and output embedding weights are tied. The model uses the o200k_base tokenizer (vocabulary size: 200,019) for better integration with tools, even though some in-house trained tokenizers showed performance improvement. The model implements periodic attention, following Gemma 3: five local attention layers are paired with one global attention layer. Group-query attention is used with 8 KV heads and 128 per-head dim