Teaching a Language Model Arithmetic with Reinforcement Learning - Sami Khan

Teaching a Language Model Arithmetic with Reinforcement Learning

Teaching a Language Model Arithmetic with Reinforcement Learning - Sami Khan Teaching a Language Model Arithmetic with Reinforcement Learning January 2026 My experience training a model on the Countdown Numbers Game — and observing it learn to cheat. I recently got early access to Prime Intellect 's hosted training platform (shoutout @willccbb ☺) and spent some time training language models with reinforcement learning on the Countdown Numbers Game — the classic UK TV show puzzle where you reach a target number using six source numbers and basic arithmetic. This post walks through what I built,

related reading

• DeepSeek-R1arxiv.org
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Training Math Reasoning Model with Reinforcement Learning - NVIDIA ADLRresearch.nvidia.com
• State of RL for reasoning LLMs | A. Weersaweers.de
• Composer2.pdfcursor.com
• The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• The State of Reinforcement Learning for LLM Reasoningsebastianraschka.com
• Understanding R1-Zero-Like Training: A Critical Perspectivearxiv.org
• RL Scaling Laws for LLMs - by Cameron R. Wolfe, Ph.D.cameronrwolfe.substack.com
• Language Models can Solve Computer Tasksarxiv.org
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io