Better RAG 1: Basics

Large Language Models have proven themselves to be wonderful few-shot learners, almost general-purpose intelligences that can learn and execute new tasks, process complex information and serve as digital assistants. However, their limited input/output window (known as context) has been the biggest hurdle in making them truly intelligent. Beyond the trillions of words used in pretraining - an expensive and time-consuming process - we are often limited to thousands of words (sometimes less than a few pages of writing) in which we can provide new information and instructions, to generate an output. If we can connect the corpus of human data with the increasing but limited contexts of LLMs, we can create systems that can learn and adapt on the fly - the same way humans can. We can build agents with persistent long-term memories, that can remember and make use of our organizational and personal data. A lot of subproblems exist in this space, but one of the simplest is that of Q&A: Given a q

Better RAG 1: Basics Hrishi Olickel Better RAG 1: Basics Getting to retrieval-augmented research, a series of everything I've learned building RAG pipelines Large Language Models have proven themselves to be wonderful few-shot learners , almost general-purpose intelligences that can learn and execute new tasks, process complex information and serve as digital assistants. However, their limited input/output window (known as context) has been the biggest hurdle in making them truly intelligent. Beyond the trillions of words used in pretraining - an expensive and time-consuming process - we are o

related reading

• Patterns for Building LLM-based Systems & Productseugeneyan.com
• Better RAG 3: The text is your friendolickel.com
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• Retrieval-Augmented Generation for Large Language Models: A Surveyarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• Better RAG 2: Single-shot is not good enougholickel.com
• Retrieval Augmented Generation: Streamlining the creation of intelligent natural language processing modelsai.facebook.com
• Rerankers and Two-Stage Retrieval | Pineconepinecone.io
• Code a simple RAG from scratchhuggingface.co
• Towards Data Sciencetowardsdatascience.com
• Productizing Large Language Modelsblog.replit.com