Subquadratic — Efficiency is Intelligence

Subquadratic is a frontier AI research and infrastructure company building a new class of LLMs.

Subquadratic — How SSA Makes Long Context Practical Research How SSA Makes Long Context Practical Date May 5, 2026 Updated May 15, 2026 Updated with recent third-party validated benchmarks from source Note: In this paper we share third-party verified benchmarks. A comprehensive model card is coming soon! SubQ is built around SSA, Subquadratic Sparse Attention, a linearly scaling attention mechanism designed for long-context retrieval, reasoning, and software engineering workloads. The core claim is simple: the hard problems enterprise AI needs to solve are long-context problems. Codebase, cont

saved by

• Varun Goyal
• Kristen Cheung

related reading

• Subquadratic — Introducing SubQ: The First Fully Subquadratic LLMsubq.ai
• [2402.12374] Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingarxiv.org
• 2502.11089arxiv.org
• Linear Attention Fundamentals | Hailey Schoelkopfhaileyschoelkopf.github.io
• [2604.20920] Simplified Sparse Attention via Gist Tokensarxiv.org
• [2603.23516] MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokensarxiv.org
• [2506.06266] Cartridges: Lightweight and general-purpose long context representations via self-studyarxiv.org
• A short note on some aspects of long context attention | nor's blognor-blog.pages.dev
• Extending Context is Hard | kaiokendevkaiokendev.github.io
• [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessarxiv.org
• How Attention Got So Efficient [GQA/MLA/DSA]youtube.com
• [2507.04239] Scaling Context Requires Rethinking Attentionarxiv.org