✳flâneur — a map of the web's best reading

How to Fix Your Context | Drew Breunig

dbreunig.com · 2,279 words · saved by 1 readers

Following up on our earlier post, “How Long Contexts Fail”, let’s run through the ways we can mitigate or avoid these failures entirely. But before we do, let’s briefly recap some of the ways long contexts can fail: Everything here is about information management. Everything in the context influences the response. We’re back to the old programming adage of, “Garbage in, garbage out.” Thankfully, there’s plenty of options for dealing with the issues above. Retrieval-Augmented Generation (RAG) is the act of selectively adding relevant information to help the LLM generate a better response. So much has been written about RAG, we’re not going to cover it today beyond saying: it’s very much alive. Every time a model ups the context window ante, a new “RAG is Dead” debate is born. The last significant event was when Llama 4 Scout landed with a 10 million token window. At that size it’s really tempting to think, “Screw it, throw it all in,” and call it a day. But, as we covered last time: if

Jun 26, 2025 AGENTS LLM AI PROMPTING CONTEXT MANAGEMENT How to Fix Your Context Mitigating & Avoiding Context Failures Following up on our earlier post, “ How Long Contexts Fail ”, let’s run through the ways we can mitigate or avoid these failures entirely. But before we do, let’s briefly recap some of the ways long contexts can fail: Context Poisoning: When a hallucination or other error makes it into the context, where it is repeatedly referenced. Context Distraction: When a context grows so long that the model over-focuses on the context, neglecting what it learned during training. Context

Explore this link on the map →

related reading