Subquadratic — Introducing SubQ: The First Fully Subquadratic LLM

Subquadratic is a frontier AI research and infrastructure company building a new class of LLMs.

Subquadratic — Introducing SubQ: The First Fully Subquadratic LLM Product Introducing SubQ: The First Fully Subquadratic LLM By Justin Dangel, co-founder and CEO, Subquadratic Date May 5, 2026 Transformers defined the last decade of AI — the "T" in ChatGPT. They unlocked modern language understanding, reasoning, and much of what we now think of as AI. Yet one fundamental limitation has shaped everything built on top of them: compute requirements scale quadratically with context length. In practice, longer inputs don't reliably improve how well a model uses the information provided. As context

saved by

• Varun Goyal

related reading

• Subquadratic — How SSA Makes Long Context Practicalsubq.ai
• [2402.12374] Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingarxiv.org
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
• Mosaic LLMs: GPT-3 quality formosaicml.com
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• Linear Attention Fundamentals | Hailey Schoelkopfhaileyschoelkopf.github.io
• Can LLMs Be Computers?percepta.ai
• Sequent: Scale and Automation for Higher Confidence in Alignment — Sequentsequent.org
• AI Revolution - Transformers and Large Language Models (LLMs)blog.eladgil.com
• Automated Alignment Researchers: Using large language models to scale scalable oversight \ Anthropicanthropic.com
• Quantization from the ground upngrok.com