Teaching a Language Model Arithmetic with Reinforcement Learning - Sami Khan

Teaching a Language Model Arithmetic with Reinforcement Learning

Teaching a Language Model Arithmetic with Reinforcement Learning - Sami Khan Teaching a Language Model Arithmetic with Reinforcement Learning January 2026 My experience training a model on the Countdown Numbers Game — and observing it learn to cheat. I recently got early access to Prime Intellect 's hosted training platform (shoutout @willccbb ☺) and spent some time training language models with reinforcement learning on the Countdown Numbers Game — the classic UK TV show puzzle where you reach a target number using six source numbers and basic arithmetic. This post walks through what I built,

related reading

• combinatorial reasoning environments for LLMs and RLdjdumpling.github.io
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• RL at 1T Scale: prime-rl Performance Deep Diveprimeintellect.ai
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• State of RL for reasoning LLMs | A. Weersaweers.de
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
• My first Vibe RL experiencesriraam.me
• Training Math Reasoning Model with Reinforcement Learning - NVIDIA ADLRresearch.nvidia.com
• The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
• Reinforcement Learning for Knowledge Awareness – kalomaze's kalomazing blogkalomaze.bearblog.dev
• Let's Verify Step by Steparxiv.org
• Optimizing LLM Test-Time Compute Involves Solving a Meta-RL Problem – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu