2112.04426

N/A

Improving language models by retrieving from trillions of tokens Sebastian Borgeaud† , Arthur Mensch† , Jordan Hoffmann† , Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela…

related reading

• Improving language models by retrieving from trillions of tokensarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• 1706.03762arxiv.org
• [2002.08909] REALM: Retrieval-Augmented Language Model Pre-Trainingarxiv.org
• [2512.23675] End-to-End Test-Time Training for Long Contextarxiv.org
• Large Language Diffusion Modelsarxiv.org
• 2212.03827arxiv.org
• 2026.mellm-1.2.pdfaclanthology.org
• [2506.06266] Cartridges: Lightweight and general-purpose long context representations via self-studyarxiv.org
• [2108.12409] Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationarxiv.org