Retrieval Augmented Generation Research: 2017-2024

I first wrote about Retrieval Augmented Generation 14 months ago on this blog. A lot has happened in this field since my initial blog post. This post seeks to give an overview of the research in this field and explain the difference between frozen, advanced, and fully dynamic RAG. This article draws inspiration from the excellent lecture "Stanford CS25: V3 I Retrieval Augmented Language Models" by Douwe Kiela, who, along with Patrick Lewis, Ethan Perez, et al., invented RAG in May 2020. The idea of enabling computers to extract information from a knowledge base to assist in language tasks goes back decades, with early question-answering systems from the 1960s and IBM's Watson Jeopardy system having similar conceptual underpinnings. To understand the origins of the first RAG-like systems in 2017 and its invention in 2021 we have to understand the underlying retrieval technology. Sparse vectors are called sparse because they are sparsely populated with information (a lot of values are ze

I first wrote about Retrieval Augmented Generation 14 months ago on this blog. A lot has happened in this field since my initial blog post. This post seeks to give an overview of the research in this field and explain the difference between frozen, advanced, and fully dynamic RAG. This article draws inspiration from the excellent lecture "Stanford CS25: V3 I Retrieval Augmented Language Models" by Douwe Kiela, who, along with Patrick Lewis, Ethan Perez, et al., invented RAG in May 2020. The idea of enabling computers to extract information from a knowledge base to assist in language tasks…

related reading

• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• Retrieval Augmented Generation: Streamlining the creation of intelligent natural language processing modelsai.facebook.com
• Retrieval-Augmented Generation for Large Language Models: A Surveyarxiv.org
• Code a simple RAG from scratchhuggingface.co
• Advanced RAG Techniques: What They Are & How to Use Themfalkordb.com
• Advanced Retriever Techniques to Improve Your RAGs | Towards Data Sciencetowardsdatascience.com
• Retrieval-augmented generation - Wikipediaen.wikipedia.org
• Mediumpub.towardsai.net
• What is Retrieval Augmented Generation (RAG)? | Databricksdatabricks.com
• Towards Data Sciencetowardsdatascience.com
• Rerankers and Two-Stage Retrieval | Pineconepinecone.io
• RAG Architecture Deep Divelinkedin.com