[2510.24941] Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought

Abstract:Large language models can generate long chain-of-thought (CoT) reasoning, but it remains unclear whether the verbalized steps reflect the models' internal thinking. In this work, we propose a True Thinking Score (TTS) to quantify the causal contribution of each step in CoT to the model's final prediction. Our experiments show that LLMs often interleave between true-thinking steps (which are genuinely used to compute the final output) and decorative-thinking steps (which give the appearance of reasoning but have minimal causal influence). We reveal that only a small subset of the total reasoning steps causally drive the model's prediction: e.g., on AIME, only an average of 2.3% of reasoning steps in CoT have a TTS >= 0.7 (range: 0-1) for Qwen-2.5. Furthermore, we find that LLMs can be steered to internally follow or disregard specific steps in their verbalized CoT using the identified TrueThinking direction. We highlight that self-verification steps in CoT (i.e., aha moments) can be decorative, while steering along the TrueThinking direction can force internal reasoning over these steps. Overall, our work reveals that LLMs often verbalize reasoning steps without performing them internally, challenging the efficiency of LLM reasoning and the trustworthiness of CoT.

[2510.24941] Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought --> Computer Science > Machine Learning arXiv:2510.24941 (cs) [Submitted on 28 Oct 2025 ( v1 ), last revised 26 May 2026 (this version, v4)] Title: Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought Authors: Jiachen Zhao , Yiyou Sun , Weiyan Shi , Dawn Song View a PDF of the paper titled Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought, by Jiachen Zhao and 3 other authors View PDF HTML (experimental) A

saved by

• Ishaan Panigrahi

related reading

• Transformers Provably Learn to Internalize Chain-of-Thoughtarxiv.org
• Quantifying the Necessity of Chain of Thought through Opaque Serial Deptharxiv.org
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• Measuring Faithfulness in Chain-of-Thought Reasoning \ Anthropicanthropic.com
• Thought Anchors: Which LLM Reasoning Steps Matter? — LessWronglesswrong.com
• As Rocks May Think | Eric Jangevjang.com
• Chain of Draft: Thinking Faster by Writing Lessarxiv.org
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• Reasoning as Trajectoriesslhleosun.github.io
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org