2405.21060-Transformers are SSMs (Mamba2)
arxiv.org · 8,043 words · saved by 1 readers
N/A
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality 1 2 Tri Dao∗ and Albert Gu∗ 1 Department…
related reading
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
- 2312.00752arxiv.org
- Mamba Explainedthegradient.pub
- H3: Language Modeling with State Space Models and (Almost) No Attention · Hazy Researchhazyresearch.stanford.edu
- State Space Duality (Mamba-2) Part II - The Theory | Goomba Labgoombalab.github.io
- State Space Duality (Mamba-2) Part I - The Model | Tri Daotridao.me
- A Visual Guide to Mamba and State Space Models - Maarten Grootendorstmaartengrootendorst.com
- State Space Duality (Mamba-2) Part I - The Model | Goomba Labgoombalab.github.io
- State Space Duality (Mamba-2) Part II - The Theory | Tri Daotridao.me
- The Mamba Effect: State Space Models Taking on Transformershungleai.substack.com
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub