Deeploy: Enabling Energy-Efficient Deployment of Small Language Models on Heterogeneous Microcontrollers | IEEE Journals & Magazine | IEEE Xplore

The latest evolutions in mainstream artificial intelligence (AI) have been driven by Transformers, which have taken over from recurrent neural networks (RNNs) and convolutional neural networks (CNNs) as the leading edge models for language processing and multimodal applications [1], [2]. The success of Transformers can be primarily attributed to the emergence of the foundation model (FM) paradigm: large Transformer models extensively pretrained on the datasets spanning trillions of tokens and then fine tuned with a much lower volume of labeled data to solve the domain-specific problems. Following the success of FMs in natural language processing (NLP) [1], [3], an increasing number of fields are starting to formulate and adapt FMs for high dimensional sensor data that has traditionally been challenging to process, like decoding the neural data [4], [5], or training embodied AI agents [6], [7], which may incorporate the multimodal sensor inputs. Operating directly on the sensory data an

saved by

• Kunal Mishra