GPU Glossary
We wrote this glossary to solve a problem we ran into working with GPUs here at Modal : the documentation is fragmented, making it difficult to connect concepts at different levels of the stack, like Streaming Multiprocessor Architecture , Compute Capability , and nvcc compiler flags . So we've read the PDFs from NVIDIA , lurked in the good Discords , and even bought dead-tree textbooks to put together a glossary that spans the whole stack in one place. This glossary, unlike a PDF or a Discord or a book, is a hypertext document -- all pages are inter-linked with one another, so you can jump down to read about the Warp Scheduler so you can better understand the threads that you came across in the article on the CUDA programming model . You can also read it linearly. To navigate between pages, use the arrow keys, the arrows at the bottom of each page, or the table of contents (in the sidebar on desktop or in the hamburger menu on mobile). The source for the glossary is available on Gi
██████╗ ██████╗ ██╗ ██╗ ██╔════╝ ██╔══██╗██║ ██║ ██║ ███╗██████╔╝██║ ██║ ██║ ██║██╔═══╝ ██║ ██║ ╚██████╔╝██║ ╚██████╔╝ ╚═════╝ ╚═╝ ╚═════╝ ██████╗ ██╗ ██████╗ ███████╗███████╗ █████╗ ██████╗ ██╗ ██╗ ██╔════╝ ██║ ██╔═══██╗██╔════╝██╔════╝██╔══██╗██╔══██╗╚██╗ ██╔╝ ██║ ███╗██║ ██║ ██║███████╗███████╗███████║██████╔╝ ╚████╔╝ ██║ ██║██║ ██║ ██║╚════██║╚════██║██╔══██║██╔══██╗ ╚██╔╝ ╚██████╔╝███████╗╚██████╔╝███████║███████║██║ ██║██║ ██║ ██║ ╚═════╝ ╚══════╝ ╚═════╝ ╚══════╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ We wrote this glossary to solve a problem we ran into working with GPUs here at Modal : the…
saved by
- Rajan Agarwal
- Annie Zhou
- Chloe Yan
- Liam Hinzman
- Vincent Huang
- Ishan Mukherjee
- Denys
- Caden Juang
- SUJASH AGRAWAL
- Yash Karthik
- Miguel Villa Floran
related reading
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- CUDA Programming Guide — CUDA Programming Guidedocs.nvidia.com
- A case for learning GPU programming with a compute-first mindset – Maister's Graphics Adventuresthemaister.net
- GitHub - adam-maj/tiny-gpu: A minimal GPU design in Verilog to learn how GPUs work from the ground up · GitHubgithub.com
- GitHub - gpu-mode/resource-stream: GPU programming related news and material linksgithub.com
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- CUDA C++ Programming Guide (Legacy) — CUDA C++ Programming Guidedocs.nvidia.com
- GPU Performance Background User's Guide - NVIDIA Docsdocs.nvidia.com
- Lecture 117:Building a GPU Compiler from Scratchyoutube.com
- Implementing a GPU's Programming Model on a CPUlitherum.blogspot.com
- What happens when you run a CUDA kernelfergusfinn.com