Do Large Language Models (LLMs) reason? | Shaped Blog

In recent years, Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP), enabling significant advancements in language understanding, text generation, and more. With the help of memorization and compositionality capabilities, LLMs can perform various tasks like never before. They're already at the core of several products used by millions of people, such as Google's search engine, Github's Copilot, and OpenAI's ChatGPT! Title image from Xavier Amatriain (2023) Despite their groundbreaking capabilities, people argue whether these language models are still missing fundamental parts that make up general intelligence. The question is whether these language models, when the parameters are scaled up, could match human-level intelligence. Or are these language models just understanding the statistics of language, so that they can pattern-match output well enough to mimic understanding? In the recent paper: “Augmented Language Models: a Survey” from Met

Do Large Language Models (LLMs) reason? In recent years, Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP), enabling significant advancements in language understanding, text generation, and more. With the help of memorization and compositionality capabilities, LLMs can perform various tasks like never before. They're already at the core of several products used by millions of people, such as Google's search engine, Github's Copilot, and OpenAI's ChatGPT! Feb 21, 2023 | 10 min read by Nina Shenker Tauris Title image from Xavier Amatriain (2023) Desp

related reading

• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• As Rocks May Think | Eric Jangevjang.com
• DeepSeek-R1arxiv.org
• Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• Is AI Reasoning Right for the Wrong Reasons? | Quanta Magazinequantamagazine.org
• the-illusion-of-thinking.pdfml-site.cdn-apple.com
• [2412.06769] Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
• [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
• [2205.11916] Large Language Models are Zero-Shot Reasonersarxiv.org
• The Role of Deductive and Inductive Reasoning in Large Language Models - ACL Anthologyaclanthology.org
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitymachinelearning.apple.com
• Unlocking the Working Memory of Large Language Models for Latent Reasoningarxiv.org