quantization2 - songhan lecture

lec06 PDF File Edit View Help 1 of 1 Page 77 of 81 Edit PDF Draw Highlight Add text 118% MIT 6.5940 : TinyML and E ffi cient Deep Learning Computing https://e ffi cientml.ai If both activations and weights are binarized 66 XNOR-Net: ImageNet Classi fi cation using Binary Convolutional Neural Networks [Rastegari et al. , ECCV 2016] 1 -1 1 -1 1 1 -1 1 input weight operations memory computation = × R R + × 1 × 1 × R B + - ~32 × less ~2 × less B B xnor, popcount ~32 × less ~58 × less = -4 + popcount(1010 xnor 1101) ≪ 1 = -4 + popcount(1000) ≪ 1 = -4 + 2 = -2 y i = − n + popcount ( W i xnor x ) ≪ 1 8 -3 5 -1 5 2 0 1 = × MIT 6.5940 : TinyML and E ffi cient Deep Learning Computing https://e ffi cientml.ai Accuracy Degradation of Binarization 67 XNOR-Net: ImageNet Classi fi cation using Binary Convolutional Neural Networks [Rastegari et al. , ECCV 2016] Neural Network Quantization Bit-Width ImageNet Top-1 Accuracy W A Delta AlexNet BWN 1 32 0.2% BNN 1 1 -28.7% XNOR-Net 1 1 -12.4% GoogleNet BWN

saved by

• Sankeerth Durvasula