Vinija's Notes • NLP • Retrieval Augmented Generation
A potential solution could be to combine the strengths of both databases: indexing parsed entity relationships with vector representations in a graph database for more flexible information retrieval. It remains to be seen if such a hybrid model exists. The image below (source) displays the high-level working of RAG. Prompting uses pre-selected static contexts provided in the prompt.
Overview Motivation Lexical Retrieval Semantic Retrieval Hybrid Retrieval (Lexical + Semantic) The Retrieval Augmented Generation (RAG) Pipeline Benefits of RAG RAG vs. Fine-tuning Ensemble of RAG Choosing a Vector DB using a Feature Matrix Building a RAG pipeline Ingestion Chunking Figuring out the ideal chunk size Retriever Ensembling and Reranking Embeddings Naive Chunking vs. Late Chunking vs. Late Interaction (ColBERT and ColPali) Overview Naive/Vanilla Chunking What is Naive/Vanilla Chunking? Example Advantages and Limitations Late Chunking What is Late Chunking? How Late Chunking Works
Explore this link on the map →related reading
- Aman's AI Journal • Primers • Retrieval Augmented Generationaman.ai
- Retrieval-Augmented Generation for Large Language Models: A Surveyarxiv.org
- Patterns for Building LLM-based Systems & Productseugeneyan.com
- Advanced Retriever Techniques to Improve Your RAGs | Towards Data Sciencetowardsdatascience.com
- Advanced RAG Techniques: What They Are & How to Use Themfalkordb.com
- Rerankers and Two-Stage Retrieval | Pineconepinecone.io
- What is Retrieval Augmented Generation (RAG)? | Databricksdatabricks.com
- Mediumpub.towardsai.net
- RAG Architecture Deep Divelinkedin.com
- Better RAG 1: Basicsolickel.com
- Better RAG 3: The text is your friendolickel.com
- Towards Data Sciencetowardsdatascience.com