Teaching a Language Model Arithmetic with Reinforcement Learning - Sami Khan

Teaching a Language Model Arithmetic with Reinforcement Learning

Teaching a Language Model Arithmetic with Reinforcement Learning - Sami Khan Teaching a Language Model Arithmetic with Reinforcement Learning January 2026 My experience training a model on the Countdown Numbers Game — and observing it learn to cheat. I recently got early access to Prime Intellect 's hosted training platform (shoutout @willccbb ☺) and spent some time training language models with reinforcement learning on the Countdown Numbers Game — the classic UK TV show puzzle where you reach a target number using six source numbers and basic arithmetic. This post walks through what I built,

related reading

• DeepSeek-R1arxiv.org
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Training Math Reasoning Model with Reinforcement Learning - NVIDIA ADLRresearch.nvidia.com
• As Rocks May Think | Eric Jangevjang.com
• State of RL for reasoning LLMs | A. Weersaweers.de
• Composer2.pdfcursor.com
• The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
• The State of Reinforcement Learning for LLM Reasoningsebastianraschka.com
• The upcoming GPT-3 moment for RL | Mechanize, Inc.mechanize.work
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• GRPO Trainer · Hugging Facehuggingface.co
• GRPO++: Tricks for Making RL Actually Workcameronrwolfe.substack.com