Mamba: The Easy Way
Today, basically any language model you can name is a Transformer model. OpenAI’s ChatGPT, Google’s Gemini, and GitHub’s Copilot are all powered by Transformers, to name a few. However, Transformers suffer from a fundamental flaw: they are powered by Attention, which scales quadratically with sequence length. Simply put, for quick exchanges (asking ChatGPT to tell a joke), this is fine. But for queries that require lots of words (asking ChatGPT to summarize a 100-page document), Transformers can become prohibitively slow.1 Many models have attempted to solve this problem, but few have done as well as Mamba. Published two months ago by Albert Gu and Tri Dao, Mamba appears to outperform similarly-sized Transformers while scaling linearly with sequence length. If you’re looking for an in-depth technical explanation of Mamba, paired with a full Triton implementation, you’re in the wrong place. Mamba: The Hard Way has already been written by the legend himself, Sasha Rush. If you haven’t he
Mamba: The Easy Way Home Blog Mamba: The Easy Way Oxford, UK - February 23, 2024 Shared on Hacker News and X Today, basically any language model you can name is a Transformer model. OpenAI’s ChatGPT , Google’s Gemini , and GitHub’s Copilot are all powered by Transformers, to name a few. However, Transformers suffer from a fundamental flaw: they are powered by Attention , which scales quadratically with sequence length. Simply put, for quick exchanges (asking ChatGPT to tell a joke), this is fine. But for queries that require lots of words (asking ChatGPT to summarize a 100-page document), Tran
saved by
related reading
- Mamba Explainedthegradient.pub
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- The Mamba Effect: State Space Models Taking on Transformershungleai.substack.com
- A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- transformer_attention.pdfarxiv.org
- 2312.00752arxiv.org
- MambaByte: Token-free Selective State Space Modelarxiv.org
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io