How to use distributed shared memory in CUDA for inter-thread-block communication

I noticed there is very little information or easy examples for using the sm-to-sm communication network on the Hopper Architecture (Compute Capability 9.0/sm_90), so I thought I would try to quickly give a super simple example. I’ve also been working on a more extended writeup on using the distributed shared memory features for a NBODY-simulation kernel, but this will have to wait till I’m done with my exams 😿. Also, this is my first attempt at a blog, so I’m super grateful for any feedback. So what is distributed shared memory (DSMEM) and why do we want it? Most “interesting” GPU problems end up being very sensitive to memory characteristics (latency/bandwidth), so much so that in my GPU architecture course we ended up almost exclusively talking about the memory-model of CUDA and its hardware implementation. A real quick breakdown of it looks like this: You have global memory which is usually what the number is you see for GPU-memory on the label. This ends up being DRAM since we wa

saved by

• Julian Quevedo