flâneur

Why GEMM is at the heart of deep learning « Pete Warden's blog

petewarden.com · 1,545 words · saved by 1 readers

Photo by Anthony Catalano I spend most of my time worrying about how to make deep learning with neural networks faster and more power efficient. In practice that means focusing on a function called…

Photo by Anthony Catalano I spend most of my time worrying about how to make deep learning with neural networks faster and more power efficient. In practice that means focusing on a function called GEMM. It’s part of the BLAS (Basic Linear Algebra Subprograms) library that was first created in 1979, and until I started trying to optimize neural networks I’d never heard of it. To explain why it’s so important, here’s a diagram from my friend Yangqing Jia’s thesis: This is breaking down where the time’s going for a typical deep convolutional neural network doing image recognition using Alex…

saved by

related reading