Physics of LMs 3.1

N/A

Physics of Language Models: Part 3.1, Knowledge Storage and Extraction Zeyuan Allen-Zhu Yuanzhi Li zeyuanallenzhu@meta.com Yuanzhi.Li@mbzuai.ac.ae Meta / FAIR Labs Mohamed bin Zayed University of AI arXiv:2309.14316v3 [cs.CL] 16 Jul 2024…

saved by

• Kevin Wang

related reading

• [2002.08909] REALM: Retrieval-Augmented Language Model Pre-Trainingarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• Patterns for Building LLM-based Systems & Productseugeneyan.com
• [2605.15156] MeMo: Memory as a Modelarxiv.org
• MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• Can a Language Model Learn Facts Continually in Its Weights?labs.baseten.co
• Language Models as Knowledge Bases?aclanthology.org
• 2212.03827arxiv.org
• Do language models possess knowledge (soundness)? - HackMDhackmd.io
• Unfamiliar Finetuning Examples Control How Language Models Hallucinatearxiv.org
• [2602.05910] Chunky Post-Training: Data Driven Failures of Generalizationarxiv.org