✳flâneur — a map of the web's best reading

On the GPU Memory Hierarchy

gytis.co · 1,575 words · saved by 1 readers

Many programmers are familiar with how the CPU works – or at least in enough details to do their job. This post will build from this understanding to explain some of the important pillars of GPU programming, including roofline charts, threads, memory loads, and the memory hierarchy itself. A single CPU core executes instructions sequentially. Computers with a single CPU core can appear to be doing multiple things at the same time, but they’re actually quickly switching between tasks. Because a core can execute only one thread of execution at a time, they’re designed to be fast. In comparison, GPUs are focused on large workloads that can take more time to complete. The speed of GPUs depends on two primary factors: When writing a GPU program, it can be helpful to take these two parameters into account to get the theoretical upper bound of the algorithm we’re writing. We can do this on a roofline chart, which looks like this: In this graph, the highest FLOPs a specific algorithm can get i

On the GPU Memory Hierarchy On the GPU Memory Hierarchy 2023-12-29 Home Projects Blog Introduction High Level Considerations Details of GPU Programming Divergence Memory Loads Multiprocessors Memory Hierarchy The Great Memory Stagnation Memory is the treasury and guardian of all things. — Cicero Introduction Many programmers are familiar with how the CPU works – or at least in enough details to do their job. This post will build from this understanding to explain some of the important pillars of GPU programming, including roofline charts, threads, memory loads, and the memory hierarchy itself.

Explore this link on the map →

saved by

related reading