The Ground Truth Weekly - Effective Context Length and Block Diffusion

No breaking news or top stories. I will be sharing the less known new developments or tools that are really interesting or noteworthy. I will also add my personal thoughts or experience with it where possible. Context window (context length) has become a widely known concept among AI adopters. It means how much text LLMs can see and process, before it starts forgetting the earliest part of the text. With a 64k context window, you can fit 10 pages of pdf files (500 words *10 pages * 0.75 = 3750 tokens) into the LLM without any issue. But to fit Shakespeare’s complete works (884,647 words), you need 884647 * 0.75 = about 660k context window. Newer models generally offer larger context window (Claude 3.5 Sonnet at 200K), and some models offer much larger context window (Gemini 2.0 Flash at 1M). However, you should not dump everything into the context window just because the model supports it. A recent study has shown that LLMs suffer “performance degrades significantly as context length i

No breaking news or top stories. I will be sharing the less known new developments or tools that are really interesting or noteworthy. I will also add my personal thoughts or experience with it where possible. Context window (context length) has become a widely known concept among AI adopters. It means how much text LLMs can see and process, before it starts forgetting the earliest part of the text. With a 64k context window, you can fit 10 pages of pdf files (500 words *10 pages * 0.75 = 3750 tokens) into the LLM without any issue. But to fit Shakespeare’s complete works (884,647 words), you