Base Models Know How to Reason, Thinking Models Learn When

What do thinking language models learn during training that their base models lack? We first present an unsupervised method that discovers a model’s reasoning behaviors by training small Sparse Autoencoders on sentence-level activations of reasoning traces, yielding interpretable reasoning taxonomies. Building on this, we introduce constructive model diffing, which aims to reconstruct the base-to-fine-tuned difference from interpretable components: reasoning mechanisms (category vectors that can induce a reasoning behavior in the base model) and reasoning heuristics (a classifier determining when a mechanism should fire). Across nine base/thinking pairs (four RL-trained, four SFT-distilled, one mixed), two independent findings agree: category vectors in the base model converge to far lower loss for taxonomies derived from purely RL-trained models, and hybrid models recover roughly 76% of the RL base-to-thinking gap but only 11% of the SFT gap. This indicates RL primarily teaches heuris

Constantin Venhoff Affiliation: University of Oxford, UK Affiliation: MATS Correspondence to: constantin@robots.ox.ac.uk Philip Torr Affiliation: University of Oxford, UK Arthur Conmy Neel Nanda Abstract What do thinking language models learn during training that their base models lack? We first present an unsupervised method that discovers a model’s reasoning behaviors by training small Sparse Autoencoders on sentence-level activations of reasoning traces, yielding interpretable reasoning taxonomies. Building on this, we introduce constructive model diffing, which aims to…

saved by

• Chris Shi

related reading

• Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelsarxiv.org
• [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
• Training Language Models to Explain Their Own Computationsarxiv.org
• Sophie Wangsophielwang.com
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Language Models can Solve Computer Tasksarxiv.org
• Training Language Models to Explain Their Own Computationsarxiv.org
• Reasoning Models Reason Well, Until They Don'tarxiv.org