Deeploy: Enabling Energy-Efficient Deployment of Small Language Models on Heterogeneous Microcontrollers | IEEE Journals & Magazine | IEEE Xplore
The latest evolutions in mainstream artificial intelligence (AI) have been driven by Transformers, which have taken over from recurrent neural networks (RNNs) and convolutional neural networks (CNNs) as the leading edge models for language processing and multimodal applications [1], [2]. The success of Transformers can be primarily attributed to the emergence of the foundation model (FM) paradigm: large Transformer models extensively pretrained on the datasets spanning trillions of tokens and then fine tuned with a much lower volume of labeled data to solve the domain-specific problems. Following the success of FMs in natural language processing (NLP) [1], [3], an increasing number of fields are starting to formulate and adapt FMs for high dimensional sensor data that has traditionally been challenging to process, like decoding the neural data [4], [5], or training embodied AI agents [6], [7], which may incorporate the multimodal sensor inputs. Operating directly on the sensory data an
Download PDF Download References Request Permissions Save to Alerts Abstract: With the rise of embodied foundation models (EFMs), most notably small language models (SLMs), adapting Transformers for the edge applications has become a very active fi...Show More Metadata Abstract: With the rise of embodied foundation models (EFMs), most notably small language models (SLMs), adapting Transformers for the edge applications has become a very active field of research. However, achieving the end-to-end deployment of SLMs on the microcontroller (MCU)-class chips without high-bandwidth…
saved by
related reading
- HeteroLLM: Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Acceleratorsarxiv.org
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- How To Scale Your Modeljax-ml.github.io
- Scalable MatMul-free Language Modelingarxiv.org
- Quantization and Hardware Architecture Co-Design for Matrix-Vector Multiplications of Large Language Modelsieeexplore.ieee.org
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- [2207.00032] DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalearxiv.org
- Large Language Models for Compiler Optimizationarxiv.org
- FTRANS: Energy-Efficient Acceleration of Transformers using FPGAarxiv.org
- Do Language Models Need Sleep? Offline Recurrence for Improved Online Inferencearxiv.org
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Language Model Compositionarxiv.org