State Space Duality (Mamba-2) Part II - The Theory | Goomba Lab

Homepage of the Goomba AI Lab @ CMU MLD.

State Space Duality (Mamba-2) Part II - The Theory | Goomba Lab State Space Duality (Mamba-2) Part II - The Theory Part I - The Model Part II - The Theory Part III - The Algorithm Part IV - The Systems In Part I of this series, we defined the state space dual (SSD) model . In isolation, this model is relatively simple to define, and we claimed that it can be computed either as an SSM recurrence or with an attention-like pattern. If you just want to use the model, feel free to skip this post! In this post, we’ll dive into the theory behind the model. We’ll derive the SSD “duality” in two comple

related reading

• State Space Duality (Mamba-2) Part II - The Theory | Tri Daotridao.me
• State Space Duality (Mamba-2) Part I - The Model | Goomba Labgoombalab.github.io
• State Space Duality (Mamba-2) Part I - The Model | Tri Daotridao.me
• Mamba No. 5 (A Little Bit Of…) | Sparse Notesjameschen.io
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
• On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
• H3: Language Modeling with State Space Models and (Almost) No Attention · Hazy Researchhazyresearch.stanford.edu
• Linear Attention Fundamentals | Hailey Schoelkopfhaileyschoelkopf.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• A Visual Guide to Mamba and State Space Models - Maarten Grootendorstmaartengrootendorst.com
• Mamba: The Easy Wayjackcook.com