Accelerating Transformer Networks through Recomposing Softmax Layers | IEEE Conference Publication | IEEE Xplore

The transformer model has recently played a key role in developing deep neural networks across various domains. Since the advent of the vanilla transformer [40], several variants, such as Google’s BERT [8] and OpenAI’s GPT [34], have been proposed. The accuracy of a transformer surpasses human performance. It is easily transferred to various natural language processing (NLP) tasks, such as sequence-to-sequence, summarization, and question and answer tasks. These capabilities now mean that the transformer model is a vital operation actively used by various industries and researchers. Many more operations are required to process the transformer model compared to existing CNN models, such as ResNet [12]. Therefore, researchers have been actively trying to accelerate the transformer model using various types of hardware, such as GPU [17, 19, 25, 36, 39], FPGA [18, 31, 32], and ASIC [11, 42]. In particular, various optimized libraries, such as TensorRT [39], Faster Transformer [25], and Dee

saved by

• Pavel