BrrrViz
GPU programming is full of concepts that make sense once you see them. BrrrViz turns the hard parts into things you can watch and interact with. ACT 01 How the GPU works. From execution to optimization. ACT 02 How ML is built on the GPU. Transformers, inference, training at scale, multi-GPU. Get notified when ML Systems drops.
A Graphics Processing Unit (GPU) is thousands of highly-parallel, small processors organized into synchronized groups, all sharing a strict memory hierarchy. GPUs don't run one thread at a time, they launch thousands of parallel processes. When threads in a warp take different paths, the GPU serializes execution. Learn the causes and mitigations. Threads in a block must coordinate access to shared memory. Learn why races happen, how barriers order access, and what they cost. Shared memory is striped across 32 banks. When multiple threads in a warp hit the same bank with different addresses, ac
Explore this link on the map →saved by
related reading
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- Execution Model - SLING user documentationdoc.sling.si
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- GitHub - adam-maj/tiny-gpu: A minimal GPU design in Verilog to learn how GPUs work from the ground up · GitHubgithub.com
- Gentle introduction to GPUs inner workings | vkSegfaultvksegfault.github.io
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- Mini Project: GPU Accelerated Matrix Multiplication (almost) like cuBLAS0mean1sigma.com
- CUDA C++ Programming Guide (Legacy) — CUDA C++ Programming Guidedocs.nvidia.com
- Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) – Hamza's Bloghamzaelshafie.bearblog.dev
- On the GPU Memory Hierarchygytis.co
- GPUs Go Brrr · Hazy Researchhazyresearch.stanford.edu
- AMD GPUs go brrr · Hazy Researchhazyresearch.stanford.edu