Recursive Language Models: the paradigm of 2026

How we plan to manage extremely long contexts

Recursive Language Models: the paradigm of 2026 How we plan to manage extremely long contexts LLM agents have become significantly more useful over the course of this year. They are now capable of implementing complex changes in large codebases autonomously, often reading and editing dozens of files, searching the web, and maintaining context even over the course of multiple such complex requests. These capabilities require the use of vast numbers of tokens. But that, in turn, is difficult for current LLMs: per-token costs rise linearly with the context length, while the performance of even th

related reading

• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• Recursive Language Models ("RLMs")whynowtech.substack.com
• 2025 LLM Year in Review – karpathykarpathy.bearblog.dev
• Lil'Loglilianweng.github.io
• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• MIT CSAIL Explains: Recursive Language Modelsyoutube.com
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• Extending Context is Hard | kaiokendevkaiokendev.github.io
• How Long Contexts Faildbreunig.com
• 2025: The year in LLMssimonwillison.net
• [2506.06266] Cartridges: Lightweight and general-purpose long context representations via self-studyarxiv.org