MaroonLLM Maybe MaroonLongLM: SelfExtend LLM Context Window Without Tuning

N/A

LLM Maybe LongLM: SelfExtend LLM Context Window Without Tuning Hongye Jin 1 * Xiaotian Han 1 * Jingfeng Yang 2 Zhimeng Jiang 1 Zirui Liu 3 Chia-Yuan Chang 1 Huiyuan Chen 4 Xia Hu 3 Abstract LLMs will be unpredictable and suffer from severe perfor- It is well known that LLMs cannot generalize…

related reading

• Extending Context is Hard | kaiokendevkaiokendev.github.io
• Extending the Context of Pretrained LLMs by Dropping their Positional Embeddingspub.sakana.ai
• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• GLM-5.2: Built for Long-Horizon Tasksz.ai
• Reimagining LLM Memory: Using Context as Training Data Unlocks Models That Learn at Test-Time | NVIDIA Technical Blogdeveloper.nvidia.com
• [2512.23675] End-to-End Test-Time Training for Long Contextarxiv.org
• [2506.06266] Cartridges: Lightweight and general-purpose long context representations via self-studyarxiv.org
• StreamingLLM gives language models unlimited contextbdtechtalks.com
• [2108.12409] Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationarxiv.org
• Arcee AI | Extending AFM-4.5B to 64k Context Lengtharcee.ai
• Mediumblog.gopenai.com
• What is a context window? | IBMibm.com