The Normal Blog - Supersizing Transformers: Going Beyond RAG with Extended minds for LLMs

Today’s popularized large language models are optimized for the task of producing sequences of tokens that look like they could’ve been present in the training corpus. This is quite distinct from the ways in which LLMs are wielded in such user interfaces as ChatGPT or Perplexity.ai, where users expect the model to perform complex reasoning tasks and faithfully retrieve factual, topical information. If we hope to use the model as a general reasoning agent and not as a stochastic parrot, we need to provide it with any relevant data at inference time, rather than rely on (1) the salient data having appeared in the training corpus and (2) the model being able to recall said data. Further, surfacing references or citations that highlight which content the model used during its generation is crucial for building applications that truly augment human workflows. This has prompted much development on methods colloquially referred to as “retrieval”1. Or, methods that help LLMs make use of pertin

Supersizing Transformers: Beyond RAG with Extended minds for LLMs - Normal Computing Let's talk Menu About Research Writing Solutions - EDA Solutions - ASICS Careers Partner with Us Partner with Us 1 Partner with Us © 2026 Normal Computing Corp. ‍ Sign up for the latest news & Insights Submit Thank you! Your submission has been received. Oops! Something went wrong while submitting the form. LET’S TALK LET’S TALK 1 LET’S TALK ‍ Privacy Policy | Terms & Conditions 10.24.2023 Supersizing Transformers: Beyond RAG with Extended minds for LLMs Phoebe Klett, Thomas Ahle Today’s popularized large lang

related reading

• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• Patterns for Building LLM-based Systems & Productseugeneyan.com
• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• [2605.15156] MeMo: Memory as a Modelarxiv.org
• Extending Context is Hard | kaiokendevkaiokendev.github.io
• Retrieval-Augmented Generation for Large Language Models: A Surveyarxiv.org
• GenAI Handbookgenai-handbook.github.io
• Explore | alphaXivalphaxiv.org
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• Long Context RAG Performance of LLMs | Databricks Blogdatabricks.com
• Reimagining LLM Memory: Using Context as Training Data Unlocks Models That Learn at Test-Time | NVIDIA Technical Blogdeveloper.nvidia.com
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org