Efficient GEMM in CUDA — NVIDIA CUTLASS Documentation

CUTLASS implements the hierarchically blocked structure described in CUTLASS: Fast Linear Algebra in CUDA C++ and the CUTLASS GTC2018 talk. The basic triple loop nest computing matrix multiply may be blocked and tiled to match concurrency in hardware, memory locality, and parallel programming models. In CUTLASS, GEMM is mapped to NVIDIA GPUs with the structure illustrated by the following loop nest. This tiled loop nest targets concurrency among threadblocks, warps, and CUDA and Tensor Cores. It takes advantage of memory locality within shared memory and registers. The figure below illustrates the flow of data within this structure. This is the hierarchical GEMM computation embodied by CUTLASS. Each stage depicts a nested level of tiling which corresponds to a layer of concurrency within the CUDA execution model and to a level within the memory hierarchy, becoming increasingly finer moving left to right. Each threadblock computes its portion of the output GEMM by iteratively loading ti

saved by

• LIME YAO