✳flâneur — a map of the web's best reading
Hyena Hierarchy: Towards Larger Convolutional Language Models · Hazy Research
hazyresearch.stanford.edu · 1,682 words · saved by 1 readers
Hyena is a large language model that uses long convolutions and gating to reach attention quality with lower time complexity.
Hyena Hierarchy: Towards Larger Convolutional Language Models · Hazy Research Mar 7, 2023 · 9 min read Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli* , Stefano Massaroli* , Eric Nguyen* , Dan Fu , Tri Dao , Stephen A. Baccus , Yoshua Bengio , Stefano Ermon , and Chris Ré . Hyena cubs are born with their eyes open and can walk within minutes! But also, Hyena is a new operator for large language models that uses long convolutions and gating, reaching attention quality with lower time complexity. (Hyena image generated by lexica.art) We're excited to share our latest
Explore this link on the map →saved by
related reading
- From Deep to Long Learning? · Hazy Researchhazyresearch.stanford.edu
- Post-Transformers - Hyena Hierarchy - by Alex Mackenziewhynowtech.substack.com
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- transformer_attention.pdfarxiv.org
- H3: Language Modeling with State Space Models and (Almost) No Attention · Hazy Researchhazyresearch.stanford.edu
- Mamba: The Easy Wayjackcook.com
- Subquadratic — How SSA Makes Long Context Practicalsubq.ai
- 1706.03762arxiv.org
- A short note on some aspects of long context attention | nor's blognor-blog.pages.dev
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- [2507.04239] Scaling Context Requires Rethinking Attentionarxiv.org
- H-Nets - the Past | Goomba Labgoombalab.github.io