✳flâneur — a map of the web's best reading

[2510.24941] Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought

arxiv.org · 730 words · saved by 1 readers

Abstract:Large language models can generate long chain-of-thought (CoT) reasoning, but it remains unclear whether the verbalized steps reflect the models' internal thinking. In this work, we propose a True Thinking Score (TTS) to quantify the causal contribution of each step in CoT to the model's final prediction. Our experiments show that LLMs often interleave between true-thinking steps (which are genuinely used to compute the final output) and decorative-thinking steps (which give the appearance of reasoning but have minimal causal influence). We reveal that only a small subset of the total reasoning steps causally drive the model's prediction: e.g., on AIME, only an average of 2.3% of reasoning steps in CoT have a TTS >= 0.7 (range: 0-1) for Qwen-2.5. Furthermore, we find that LLMs can be steered to internally follow or disregard specific steps in their verbalized CoT using the identified TrueThinking direction. We highlight that self-verification steps in CoT (i.e., aha moments) can be decorative, while steering along the TrueThinking direction can force internal reasoning over these steps. Overall, our work reveals that LLMs often verbalize reasoning steps without performing them internally, challenging the efficiency of LLM reasoning and the trustworthiness of CoT.

[2510.24941] Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought --> Computer Science > Machine Learning arXiv:2510.24941 (cs) [Submitted on 28 Oct 2025 ( v1 ), last revised 26 May 2026 (this version, v4)] Title: Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought Authors: Jiachen Zhao , Yiyou Sun , Weiyan Shi , Dawn Song View a PDF of the paper titled Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought, by Jiachen Zhao and 3 other authors View PDF HTML (experimental) A

Explore this link on the map →

saved by

related reading