Thinking to recall: How reasoning unlocks parametric knowledge in LLMs

We study the counterintuitive phenomenon where reasoning helps language models recall simple facts, even when no complex step-by-step solutions are required. We show that this phenomenon is driven by two mechanisms: (1) using generated reasoning tokens to perform latent computation, and (2) generating related facts to prime correct answer recall. It is well-established that allowing large language models (LLMs) to generate step-by-step reasoning traces, commonly known as chain-of-thought (CoT), enhances performance on complex tasks. When a model solves difficult math equations, writes software, or answers multi-hop factual questions, breaking the problem down into manageable logical steps is highly effective. However, the utility of this approach remains unclear for simple, single-hop factual questions. For instance, consider a query like: "What year was Mary Engle Pennington inducted into the National Inventors Hall of Fame?" An LLM either has the fact stored in its parametric memory

It is well-established that allowing large language models (LLMs) to generate step-by-step reasoning traces, commonly known as chain-of-thought (CoT), enhances performance on complex tasks. When a model solves difficult math equations, writes software, or answers multi-hop factual questions, breaking the problem down into manageable logical steps is highly effective. However, the utility of this approach remains unclear for simple, single-hop factual questions. For instance, consider a query like: "What year was Mary Engle Pennington inducted into the National Inventors Hall of Fame?" An…

saved by

• Davis Brown

related reading

• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Unlocking the Working Memory of Large Language Models for Latent Reasoningarxiv.org
• [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• Base Models Know How to Reason, Thinking Models Learn Whenarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• [2205.11916] Large Language Models are Zero-Shot Reasonersarxiv.org
• Language Models Perform Reasoning via Chain of Thoughtai.googleblog.com
• [2502.19402] General Reasoning Requires Learning to Reason from the Get-goar5iv.labs.arxiv.org
• [2203.14465] STaR: Bootstrapping Reasoning With Reasoningarxiv.org