✳flâneur — a map of the web's best reading

Using Document Layout Structure for Efficient RAG

ambikasukla.substack.com · 868 words · saved by 1 readers

The remarkable capabilities of large language models (LLMs) open up incredible opportunities for analyzing documents at scale. However, making the text in documents, especially large PDFs available for LLM has been a challenge due to the amount of text the LLM can analyze at a time (a.k.a LLM context window). There are broadly two approaches to solve the problem: Convert documents to raw text and break down the text into small chunks of texts, usually less than the LLM context window size (approx 2000-3000 words). There is a growing body of research in making LLMs work with much larger contexts that scale up to several hundred pages of text. Both the options have severe drawbacks, which I will discuss below. At this point, if you’d like to go straight the solution, here’s the github link. Let’s look at naive chunking. Documents express their meaning through their layout: Headings and nested subheadings represent a theme that describes all the text underneath. The picture below illustra

Using Document Layout Structure for Efficient RAG Chunking documents by their logical layout structure improves LLM performance on large documents. Ambika Sukla Oct 19, 2023 38 11 4 Share The remarkable capabilities of large language models (LLMs) open up incredible opportunities for analyzing documents at scale. However, making the text in documents, especially large PDFs available for LLM has been a challenge due to the amount of text the LLM can analyze at a time (a.k.a LLM context window). There are broadly two approaches to solve the problem: Convert documents to raw text and break down t

Explore this link on the map →

related reading