Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models · Hazy Research

In our previous post, we introduced the challenges of continuous time series and overviewed the three main deep learning paradigms for addressing them: recurrence, convolutions, and continuous-time models. We discussed their strengths and weaknesses, and summarized recent progress on connecting these families of models.

Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models · Hazy Research Jan 14, 2022 · 22 min read Structured State Spaces: Combining Continuous-Time, Recurrent, and Convolutional Models Albert Gu , Karan Goel , Khaled Saab , and Chris Ré In our previous post, we introduced the challenges of continuous time series and overviewed the three main deep learning paradigms for addressing them: recurrence, convolutions, and continuous-time models. We discussed their strengths and weaknesses, and summarized recent progress on connecting these families of models. In this

saved by

• Rishi Kothari
• Amos You
• Yudhister Joel Kumar

related reading

• On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
• The Topological Trouble With Transformersarxiv.org
• A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
• [2111.00396] Efficiently Modeling Long Sequences with Structured State Spacesarxiv.org
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• A Visual Guide to Mamba and State Space Models - Maarten Grootendorstmaartengrootendorst.com
• 2312.00752arxiv.org
• Simplifying S4 · Hazy Researchhazyresearch.stanford.edu
• David Duvenaudcs.toronto.edu
• Mamba No. 5 (A Little Bit Of…) | Sparse Notesjameschen.io
• [2111.00396] Efficiently Modeling Long Sequences with Structured State Spacesarxiv.org
• Hands-On Machine Learning with Scikit-Learn and PyTorchageron.github.io