Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models · Hazy Research

In our previous post, we introduced the challenges of continuous time series and overviewed the three main deep learning paradigms for addressing them: recurrence, convolutions, and continuous-time models. We discussed their strengths and weaknesses, and summarized recent progress on connecting these families of models.

Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models · Hazy Research Jan 14, 2022 · 22 min read Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models Albert Gu , Karan Goel , Khaled Saab , and Chris Ré In our previous post, we introduced the challenges of continuous time series and overviewed the three main deep learning paradigms for addressing them: recurrence, convolutions, and continuous-time models. We discussed their strengths and weaknesses, and summarized recent progress on connecting these families of models. In this

saved by

• Rishi Kothari
• Amos You
• Yudhister Joel Kumar

related reading

• On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
• The Topological Trouble With Transformersarxiv.org
• [2111.00396] Efficiently Modeling Long Sequences with Structured State Spacesarxiv.org
• [2111.00396] Efficiently Modeling Long Sequences with Structured State Spacesarxiv.org
• H-Nets - the Past | Goomba Labgoombalab.github.io
• Simplifying S4 · Hazy Researchhazyresearch.stanford.edu
• Mamba No. 5 (A Little Bit Of…) | Sparse Notesjameschen.io
• Understanding LSTM Networks -- colah's blogcolah.github.io
• H3: Language Modeling with State Space Models and (Almost) No Attention · Hazy Researchhazyresearch.stanford.edu
• A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
• 2312.00752arxiv.org
• Sequence Modeling with CTCdistill.pub