[2510.27484] Thought Branches: Interpreting LLM Reasoning Requires Resampling

Abstract:Most work interpreting reasoning models studies only a single chain-of-thought (CoT), yet these models define distributions over many possible CoTs. We argue that studying a single sample is inadequate for understanding causal influence and the underlying computation. Though fully specifying this distribution is intractable, it can be understood by sampling. We present case studies using resampling to investigate model decisions. First, when a model states a reason for its action, does that reason actually cause the action? In "agentic misalignment" scenarios, we resample specific sentences to measure their downstream effects. Self-preservation sentences have small causal impact, suggesting they do not meaningfully drive blackmail. Second, are artificial edits to CoT sufficient for steering reasoning? These are common in literature, yet take the model off-policy. Resampling and selecting a completion with the desired property is a principled on-policy alternative. We find off-policy interventions yield small and unstable effects compared to resampling in decision-making tasks. Third, how do we understand the effect of removing a reasoning step when the model may repeat it post-edit? We introduce a resilience metric that repeatedly resamples to prevent similar content from reappearing downstream. Critical planning statements resist removal but have large effects when eliminated. Fourth, since CoT is sometimes "unfaithful", can our methods teach us anything in these settings? Adapting causal mediation analysis, we find that hints that have a causal effect on the output without being explicitly mentioned exert a subtle and cumulative influence on the CoT that persists even if the hint is removed. Overall, studying distributions via resampling enables reliable causal analysis, clearer narratives of model reasoning, and principled CoT interventions.

[2510.27484] Thought Branches: Interpreting LLM Reasoning Requires Resampling Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2510.27484 (cs) [Submitted on 31 Oct 2025 ( v1 ), last revised 13 Apr 2026 (this version, v2)] Title: Thought Branches: Interpreting LLM Reasoning Requires Resampling Authors: Uzay Macar , Paul C. Bogdan , Senthooran Rajamanoharan , Neel Nanda View a PDF of the paper titled Thought Branches: Interpreting LLM Reasoning Requires Resampli

related reading

• Thought Branches: Interpreting LLM Reasoning Requires Resamplingarxiv.org
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• [2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyarxiv.org
• [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
• [2510.24941] Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtarxiv.org
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Forking Fast: Efficiently Estimating Uncertainty Dynamics in Text Generationarxiv.org
• [2510.09714] All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Languagearxiv.org
• Reasoning Models Reason Well, Until They Don'tarxiv.org
• [2603.05706] Reasoning Models Struggle to Control their Chains of Thoughtarxiv.org