What is the roofline model? | GPU Glossary
The roofline model is a simplified, visual model of performance used to quickly determine whether a program is bound by memory bandwidth or arithmetic bandwidth . Kernels to the left of the ridge point are limited by the bandwidth of the memory subsystem and kernels to the right of the ridge point are limited by the bandwidth of the arithmetic subsystem . Diagram adapted from Williams, Waterman, and Patterson (2008) , which introduced the roofline model. Kernels to the left of the ridge point are limited by the bandwidth of the memory subsystem and kernels to the right of the ridge point are limited by the bandwidth of the arithmetic subsystem . Diagram adapted from Williams, Waterman, and Patterson (2008) , which introduced the roofline model. In the roofline model, two hardware‑derived "roofs" put a "ceiling" on the possible performance: These are visualized on a plane with the arithmetic intensity (in operations per byte) on the x-axis and the performance (in operations per
The roofline model is a simplified, visual model of performance used to quickly determine whether a program is bound by memory bandwidth or arithmetic bandwidth . In the roofline model, two hardware‑derived "roofs" put a "ceiling" on the possible performance: the "compute roof" – the peak rate of the target hardware (CUDA Cores or Tensor Cores ), aka the arithmetic bandwidth the "memory roof" – the peak memory throughput of the target hardware, aka the memory bandwidth . These are visualized on a plane with the arithmetic intensity (in operations per byte) on the x-axis and the…
saved by
related reading
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- Microsoft Word - CACM_roofline_complete.docusers.cs.duke.edu
- On the GPU Memory Hierarchygytis.co
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- GPU Performance Background User's Guide - NVIDIA Docsdocs.nvidia.com
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- How to Think About TPUs | How To Scale Your Modeljax-ml.github.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Execution Model - SLING user documentationdoc.sling.si
- AI Chip Architecturesjacobpeake.com
- KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu