[2507.06203] A Survey on Latent Reasoning

Abstract:Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: this https URL.

[2507.06203] A Survey on Latent Reasoning Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Computation and Language arXiv:2507.06203 (cs) [Submitted on 8 Jul 2025 ( v1 ), last revised 10 Jul 2025 (this version, v2)] Title: A Survey on Latent Reasoning Authors: Rui-Jie Zhu , Tianhao Peng , Tianhao Cheng , Xingwei Qu , Jinfa Huang , Dawei Zhu , Hao Wang , Kaiwen Xue , Xuanliang Zhang , Yong Shan , Tianle Cai , Taylor Kergan , Assel Kembay , Andrew Smith , Chenghua Lin , Binh Nguyen ,

related reading

• Unlocking the Working Memory of Large Language Models for Latent Reasoningarxiv.org
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Worries about latent reasoning in LLMs — EA Forumforum.effectivealtruism.org
• [2504.10615] Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Modelsarxiv.org
• DeepSeek-R1arxiv.org
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• Announcing ReasoningLens — Visualizing and Diagnosing LLM Reasoning at a Glancehuggingface.co
• Do Large Language Models (LLMs) reason? | Shapedshaped.ai
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• Thought Anchors: Which LLM Reasoning Steps Matter? — LessWronglesswrong.com