Matrix Multiplication — Triton documentation
In this tutorial, you will write a very short high-performance FP16 matrix multiplication kernel that achieves performance on par with cuBLAS or rocBLAS. You will specifically learn about: Block-level matrix multiplications. Multi-dimensional pointer arithmetic. Program re-ordering for improved L2 cache hit rate. Automatic performance tuning. Matrix multiplications are a key building block of most modern high-performance computing systems. They are notoriously hard to optimize, hence their implementation is generally done by hardware vendors themselves as part of so-called “kernel libraries” (e.g., cuBLAS). Unfortunately, these libraries are often proprietary and cannot be easily customized to accommodate the needs of modern deep learning workloads (e.g., fused activation functions). In this tutorial, you will learn how to implement efficient matrix multiplications by yourself with Triton, in a way that is easy to customize and extend. Roughly speaking, the kernel that we will write wi