What is the roofline model? | GPU Glossary
The roofline model is a simplified, visual model of performance used to quickly determine whether a program is bound by memory bandwidth or arithmetic bandwidth . Kernels to the left of the ridge point are limited by the bandwidth of the memory subsystem and kernels to the right of the ridge point are limited by the bandwidth of the arithmetic subsystem . Diagram adapted from Williams, Waterman, and Patterson (2008) , which introduced the roofline model. Kernels to the left of the ridge point are limited by the bandwidth of the memory subsystem and kernels to the right of the ridge point are limited by the bandwidth of the arithmetic subsystem . Diagram adapted from Williams, Waterman, and Patterson (2008) , which introduced the roofline model. In the roofline model, two hardware‑derived "roofs" put a "ceiling" on the possible performance: These are visualized on a plane with the arithmetic intensity (in operations per byte) on the x-axis and the performance (in operations per
The roofline model is a simplified, visual model of performance used to quickly determine whether a program is bound by memory bandwidth or arithmetic bandwidth . In the roofline model, two hardware‑derived "roofs" put a "ceiling" on the possible performance: the "compute roof" – the peak rate of the target hardware (CUDA Cores or Tensor Cores ), aka the arithmetic bandwidth the "memory roof" – the peak memory throughput of the target hardware, aka the memory bandwidth . These are visualized on a plane with the arithmetic intensity (in operations per byte) on the x-axis and the…
saved by
related reading
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- Microsoft Word - CACM_roofline_complete.docusers.cs.duke.edu
- Making Deep Learning go Brrrr From First Principleshorace.io
- How To Scale Your Modeljax-ml.github.io
- On the GPU Memory Hierarchygytis.co
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- 2403.14123arxiv.org
- GPU Performance Background User's Guide - NVIDIA Docsdocs.nvidia.com
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- GitHub - wafer-ai/gpu-perf-engineering-resources: A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.github.com