✳flâneur — a map of the web's best reading

State Space Duality (Mamba-2) Part I - The Model | Tri Dao

tridao.me · 2,925 words · saved by 1 readers

Since the release of Mamba 6 months ago, we’ve been pleasantly surprised by the overwhelming community response. It’s been incredibly gratifying to see the line of research on efficient sequence models we’ve been pursuing for years really resonate with the machine learning community and take off more than we could have anticipated. We’ve seen an enormous amount of exciting follow-up work, from direct applications (e.g. vision , genomics , graphs , and more) to understanding (e.g. on recall abilities , in-context learning , and formal language expressivity ), and an enormous number of online blogs, tutorials, and videos. We couldn’t be more excited about the direction of this research! Yet despite its potential so far, we weren’t completely satisfied with the first version of Mamba… From a conceptual standpoint, one of the reasons we found SSMs so fascinating is how they just feel fundamental. One way this is exemplified is how they have rich ties to many major paradigms of sequence mo

State Space Duality (Mamba-2) Part I - The Model | Tri Dao State Space Duality (Mamba-2) Part I - The Model [ Paper ] [ Code ] This series is cross-posted at GoombaLab Part I - The Model Part II - The Theory Part III - The Algorithm Part IV - The Systems Since the release of Mamba 6 months ago, we’ve been pleasantly surprised by the overwhelming community response . It’s been incredibly gratifying to see the line of research on efficient sequence models we’ve been pursuing for years really resonate with the machine learning community and take off more than we could have anticipated. We’ve seen

Explore this link on the map →

related reading