Retrieval Augmented Generation Research: 2017-2024

I first wrote about Retrieval Augmented Generation 14 months ago on this blog. A lot has happened in this field since my initial blog post. This post seeks to give an overview of the research in this field and explain the difference between frozen, advanced, and fully dynamic RAG. This article draws inspiration from the excellent lecture "Stanford CS25: V3 I Retrieval Augmented Language Models" by Douwe Kiela, who, along with Patrick Lewis, Ethan Perez, et al., invented RAG in May 2020. The idea of enabling computers to extract information from a knowledge base to assist in language tasks goes back decades, with early question-answering systems from the 1960s and IBM's Watson Jeopardy system having similar conceptual underpinnings. To understand the origins of the first RAG-like systems in 2017 and its invention in 2021 we have to understand the underlying retrieval technology. Sparse vectors are called sparse because they are sparsely populated with information (a lot of values are ze

I first wrote about Retrieval Augmented Generation 14 months ago on this blog. A lot has happened in this field since my initial blog post. This post seeks to give an overview of the research in this field and explain the difference between frozen, advanced, and fully dynamic RAG. This article draws inspiration from the excellent lecture "Stanford CS25: V3 I Retrieval Augmented Language Models" by Douwe Kiela, who, along with Patrick Lewis, Ethan Perez, et al., invented RAG in May 2020. The idea of enabling computers to extract information from a knowledge base to assist in language tasks goes