2405.21060-Transformers are SSMs (Mamba2)
arxiv.org · 8,043 words · saved by 1 readers
N/A
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality 1 2 Tri Dao∗ and Albert Gu∗ 1 Department…
related reading
- 2312.00752arxiv.org
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- Mamba 2 - Transformers are SSMs: Generalized Models and Efficient Algorithms Through SSS Dualityyoutube.com
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
- Mamba Explainedthegradient.pub
- The Mamba Effect: State Space Models Taking on Transformershungleai.substack.com
- Mamba No. 5 (A Little Bit Of…) | Sparse Notesjameschen.io
- State Space Duality (Mamba-2) Part II - The Theory | Goomba Labgoombalab.github.io
- State Space Duality (Mamba-2) Part I - The Model | Tri Daotridao.me
- [2111.00396] Efficiently Modeling Long Sequences with Structured State Spacesarxiv.org
- State Space Duality (Mamba-2) Part I - The Model | Goomba Labgoombalab.github.io