Unlocking the Working Memory of Large Language Models for Latent Reasoning

To improve the reasoning capabilities of large language models, test-time compute is typically scaled by generating intermediate tokens before the final answer. However, this couples reasoning to autoregressive generation and thereby conflates internal computation with external communication. In contrast, human cognition can use working memory to hold and manipulate information internally without the need to externalize intermediate thoughts. Drawing on this principle, we introduce Reasoning in Memory (RiM), a latent reasoning method that replaces the autoregressive generation of reasoning steps with memory blocks. These memory blocks are fixed sequences of special tokens that unlock the working-memory capacity of large language models. Since they are fixed rather than generated, they can be processed in a single forward pass, enabling compute-efficient latent reasoning. To operationalize these memory blocks, we employ a two-stage curriculum. First, we ground them by predicting explici

Unlocking the Working Memory of Large Language Models for Latent Reasoning Lukas Aichberger 1 Sepp Hochreiter 1,2 1 ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria 2 NXAI GmbH, Linz, Austria {aichberger, hochreit}@ml.jku.at Abstract To improve the reasoning capabilities of large language models, test-time compute is typically scaled by generating intermediate tokens before the final answer. However, this couples reasoning to autoregressive generation and thereby conflates internal computation with external communication. In contrast, hum

saved by

• Henry Zimmerman

related reading

• DeepSeek-R1arxiv.org
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Explore | alphaXivalphaxiv.org
• [2507.06203] A Survey on Latent Reasoningarxiv.org
• [2504.10615] Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Modelsarxiv.org
• Reasoning as Trajectoriesslhleosun.github.io
• [2502.19402] General Reasoning Requires Learning to Reason from the Get-goar5iv.labs.arxiv.org
• Do Large Language Models (LLMs) reason? | Shapedshaped.ai
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• The State of LLM Reasoning Model Inferencesebastianraschka.com