Energon: Toward Efficient Acceleration of Transformers Using Dynamic Sparse Attention | IEEE Journals & Magazine | IEEE Xplore

Benefiting from the powerful attention mechanism, transformer models, such as Seq2seq [65], BERT [12], GPT-2 [47], XLNet [70], T5 [48], and other variants [28], [29], [31], have achieved leading-edge performance on various natural language processing (NLP) tasks, such as question-answering, text classification, machine translation, etc. Besides, transformers also show promising performance on many computer vision (CV) tasks, including image classification [13], [64], object detection [4], [78], and even video comprehension [54], [61]. In brief, transformers are becoming the de facto substitutes for traditional RNNs and CNNs in a broad range of scenarios. Despite their effectiveness, it is still challenging to deploy transformers on resource-constrained devices. The greatest difficulty comes from the attention operations, which involve complicated data movement [15], [68] and bear quadratic computational complexity concerning the input sequence length [3], [25], [46], [52]. Therefore, f

saved by

• Pavel