flâneur

LangChain ParentDocumentRetriever: Strike a Balance between large vs small chunks | ClusteredBytes

clusteredbytes.pages.dev · 891 words · saved by 1 readers

Looking at LangChain ParentDocumentRetriever to strike a balance between large vs small document chunks while retrieving related documents.

Table of Contents Small vs Large Document chunks ParentDocumentRetriever Two splitters instead of one Storing both larger and smaller chunks Create the ParentDocumentRetriever Adding the documents Retrieving relevant documents Small vs Large Document chunks # For Retrieval Augmented Generation (RAG), we split the raw text into multiple chunks and embed them using an embedding model (e.g. OpenAI Embeddings). Then we use a retriever to retrieve the most relevant chunks to user's query and feed them chunks to the LLM alongside the input. But what should be the ideal length of each chunk? What's t

saved by

related reading