On the Theoretical Limitations of Embedding-Based Retrieval

N/A

On the Theoretical Limitations of Embedding-Based Retrieval Orion Weller*,1,2 , Michael Boratko1 , Iftekhar Naim1 and Jinhyuk Lee1 1 Google DeepMind, 2 Johns Hopkins University Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-following, coding, and more. These new benchmarks…

related reading

• On the Theoretical Limitations of Embedding-Based Retrievalarxiv.org
• [2508.21038] On the Theoretical Limitations of Embedding-Based Retrievalarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• 1301.3781arxiv.org
• [2603.07146] Fine-Grained Table Retrieval Through the Lens of Complex Queriesarxiv.org
• [2205.13147] Matryoshka Representation Learningarxiv.org
• [2002.08909] REALM: Retrieval-Augmented Language Model Pre-Trainingarxiv.org
• [2212.10496] Precise Zero-Shot Dense Retrieval without Relevance Labelsarxiv.org
• Beyond embeddings: Navigating the shift to completions-only RAGsensible.so
• [2108.12409] Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationarxiv.org
• You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectoriesarxiv.org
• [2506.06266] Cartridges: Lightweight and general-purpose long context representations via self-studyarxiv.org