Vestigial reasoning in RL — LessWrong

TL;DR: I claim that many reasoning patterns that appear in chains-of-thought are not actually used by the model to come to its answer, and can be more accurately thought of as historical artifacts of training. This can be true even for CoTs that are apparently "faithful" to the true reasons for the model's answer. Epistemic status: I'm pretty confident that the model described here is more accurate than my previous understanding. However, I wouldn't be very surprised if parts of this post are significantly wrong or misleading. Further experiments would be helpful for validating some of these hypotheses. Thanks to @Andy Arditi and @David Lindner for giving feedback on a draft of this post. Until recently, I assumed that RL training would cause reasoning models to make their chains-of-thought as efficient as possible, so that every token is directly useful to the model. However, I now believe that by default,[1] reasoning models' CoTs will often include many "useless" tokens that don't h

x Vestigial reasoning in RL — LessWrong AI Frontpage 69 Vestigial reasoning in RL by Caleb Biddulph 13th Apr 2025 11 min read 8 69 TL;DR: I claim that many reasoning patterns that appear in chains-of-thought are not actually used by the model to come to its answer, and can be more accurately thought of as historical artifacts of training. This can be true even for CoTs that are apparently "faithful" to the true reasons for the model's answer. Epistemic status: I'm pretty confident that the model described here is more accurate than my previous understanding. However, I wouldn't be very surpris

related reading

• DeepSeek-R1arxiv.org
• State of RL for reasoning LLMs | A. Weersaweers.de
• [2510.27338] Reasoning Models Sometimes Output Illegible Chains of Thoughtarxiv.org
• Thought Branches: Interpreting LLM Reasoning Requires Resamplingarxiv.org
• o1 and Reasoning | AndoLogsblog.ando.ai
• [2505.05410] Reasoning Models Don't Always Say What They Thinkarxiv.org
• The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
• Reasoning models don't always say what they think \ Anthropicanthropic.com
• the case for CoT unfaithfulness is overstated — LessWronglesswrong.com
• [2510.24941] Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtarxiv.org
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationarxiv.org
• Measuring Faithfulness in Chain-of-Thought Reasoning \ Anthropicanthropic.com