flâneur

The Ground Truth Weekly - Effective Context Length and Block Diffusion

thegroundtruth.substack.com · 1,031 words · saved by 1 readers

No breaking news or top stories. I will be sharing the less known new developments or tools that are really interesting or noteworthy. I will also add my personal thoughts or experience with it where possible. Context window (context length) has become a widely known concept among AI adopters. It means how much text LLMs can see and process, before it starts forgetting the earliest part of the text. With a 64k context window, you can fit 10 pages of pdf files (500 words *10 pages * 0.75 = 3750 tokens) into the LLM without any issue. But to fit Shakespeare’s complete works (884,647 words), you need 884647 * 0.75 = about 660k context window. Newer models generally offer larger context window (Claude 3.5 Sonnet at 200K), and some models offer much larger context window (Gemini 2.0 Flash at 1M). However, you should not dump everything into the context window just because the model supports it. A recent study has shown that LLMs suffer “performance degrades significantly as context length i

Welcome to the first issue of The Ground Truth Weekly. There are already a lot of AI news in the world, so I am going in a different direction. No breaking news or top stories. I will be sharing the less known new developments or tools that are really interesting or noteworthy. I will also add my personal thoughts or experience with it where possible. Context window (context length) has become a widely known concept among AI adopters. It means how much text LLMs can see and process, before it starts forgetting the earliest part of the text. With a 64k context window, you can fit 10 pages…

related reading