Modern GPU Programming For MLSys — Modern GPU Programming For MLSys
Machine learning systems sit at the heart of modern AI workloads. In these systems, performance often comes down to the quality of a small number of GPU kernels. Attention kernels, LLM prefill and decode kernels, low-precision block-scaled GEMMs, fused MoE layers, and other large fused kernels all directly shape end-to-end speed in both training and serving.
Modern GPU Programming For MLSys# Machine learning systems power many of today’s AI workloads. As models grow and deployment settings become more complex, end-to-end performance increasingly depends on a small number of critical GPU kernels. Attention, LLM prefill and decode, low-precision block-scaled GEMM, fused MoE layers, and other large fused kernels directly affect both training and serving speed. Making these kernels fast requires more than a list of optimization tricks. Recent GPU architectures introduce richer memory spaces, new data-movement mechanisms, and increasingly…
saved by
related reading
- GitHub - wafer-ai/gpu-perf-engineering-resources: A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.github.com
- Modern GPU Programming For MLSys — Modern GPU Programming For MLSysmlc.ai
- KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu
- GitHub - adam-maj/tiny-gpu: A minimal GPU design in Verilog to learn how GPUs work from the ground up · GitHubgithub.com
- Machine Learning System Resources | std::bodun::blogbodunhu.com
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- How To Scale Your Modeljax-ml.github.io
- AI Chip Architecturesjacobpeake.com
- [2410.20399] ThunderKittens: Simple, Fast, and Adorable AI Kernelsarxiv.org
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com