flâneur

Modern GPU Programming For MLSys — Modern GPU Programming For MLSys

mlc.ai · 547 words · saved by 1 readers

Machine learning systems sit at the heart of modern AI workloads. In these systems, performance often comes down to the quality of a small number of GPU kernels. Attention kernels, LLM prefill and decode kernels, low-precision block-scaled GEMMs, fused MoE layers, and other large fused kernels all directly shape end-to-end speed in both training and serving. To make these kernels fast, however, we need more than a list of optimization tricks. Modern GPUs are no longer simple variations of the same old design. Recent architectures introduce richer memory spaces, new access patterns, and increasingly specialized execution units. To program them well, we need both a clear mental model of the hardware and a practical understanding of how high-performance kernels are built. This book is about developing both. The book follows a simple progression: first understand the GPU hardware, then learn the programming model we will use, and finally build state-of-the-art kernels step by step. Our mai

Modern GPU Programming For MLSys# Machine learning systems power many of today’s AI workloads. As models grow and deployment settings become more complex, end-to-end performance increasingly depends on a small number of critical GPU kernels. Attention, LLM prefill and decode, low-precision block-scaled GEMM, fused MoE layers, and other large fused kernels directly affect both training and serving speed. Making these kernels fast requires more than a list of optimization tricks. Recent GPU architectures introduce richer memory spaces, new data-movement mechanisms, and increasingly…

saved by

related reading