Better RAG 1: Basics

olickel.com · 2,298 words · saved by 1 readers

Large Language Models have proven themselves to be wonderful few-shot learners, almost general-purpose intelligences that can learn and execute new tasks, process complex information and serve as digital assistants. However, their limited input/output window (known as context) has been the biggest hurdle in making them truly intelligent. Beyond the trillions of words used in pretraining - an expensive and time-consuming process - we are often limited to thousands of words (sometimes less than a few pages of writing) in which we can provide new information and instructions, to generate an output. If we can connect the corpus of human data with the increasing but limited contexts of LLMs, we can create systems that can learn and adapt on the fly - the same way humans can. We can build agents with persistent long-term memories, that can remember and make use of our organizational and personal data. A lot of subproblems exist in this space, but one of the simplest is that of Q&A: Given a q

Better RAG 1: Basics Hrishi Olickel Better RAG 1: Basics Getting to retrieval-augmented research, a series of everything I've learned building RAG pipelines Large Language Models have proven themselves to be wonderful few-shot learners , almost general-purpose intelligences that can learn and execute new tasks, process complex information and serve as digital assistants. However, their limited input/output window (known as context) has been the biggest hurdle in making them truly intelligent. Beyond the trillions of words used in pretraining - an expensive and time-consuming process - we are o

Better RAG 1: Basics

related reading