Adaptive spatial-temporal information processing based on in-memory attention-inspired devices | Nature Communications

Pan et al. report an attention-inspired architecture for adaptive spatial-temporal information processing based on 0D-2D hetero-dimensional interface between MoS2 and Ag filament. Wafer-scale device array is prepared for in-memory analog computing and applied to autonomous driving edge intelligence scenarios.

Download PDF Subjects Electrical and electronic engineering Electronic devices Two-dimensional materials Abstract Spatial-temporal information perception is widely used for motion processing in dynamic scenes, but present technology requires relatively huge hardware resource consumption. The attention mechanism helps the human brain extract required information from tremendous data at a low cost. Here, we propose an attention-inspired artificial intelligence architecture based on hetero-dimensional modulations between zero-dimensional contact and two-dimensional electrostatic interfaces. An ad

saved by

• Sunain Dhingra

related reading

• AttentionLego: An Open-Source Building Block For Spatially-Scalable Large Language Model Accelerator With Processing-In-Memory Technologyarxiv.org
• MoNETA: A Mind Made from Memristors - IEEE Spectrumspectrum.ieee.org
• [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessarxiv.org
• Meta harness makes 10 times better Kimi K3 chipluoluo.ai
• [2501.00663] Titans: Learning to Memorize at Test Timearxiv.org
• A$^3$: Accelerating Attention Mechanisms in Neural Networks with Approximationarxiv.org
• Overleaf Examplearxiv.org
• The Unbearable Slowness of Beingarxiv.org
• Miruvor AI — The age of scaling is over.miruvor.ai
• DeltaNet Explained (Part I) | Songlin Yangsustcsonglin.github.io
• [2012.09852] SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningarxiv.org
• 2502.11089arxiv.org