Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) | Hamza's Blog
Matrix multiplication sits at the core of modern deep learning. Whether it is transformers, CNNs, or even simple MLPs, everything eventually reduces to G...
Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) – Hamza's Blog Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) 12 Jan, 2026 🚧 Work in progress. Please reach out on LinkedIn if you spot any mistakes. Introduction Matrix multiplication sits at the core of modern deep learning. Whether it is transformers, CNNs, or even simple MLPs, everything eventually reduces to GEMM. GPUs are built to run this operation at scale, and libraries like cuBLAS set the performance bar with kernels tuned down to the last instruction. In this blog I am rebuilding th
Explore this link on the map →saved by
related reading
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- CVPR2023_eff_tutorial_molchanov.pdfnvlabs.github.io
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- NVIDIA Tensor Core Evolution: From Volta To Blackwellnewsletter.semianalysis.com
- GPU Performance Background User's Guide - NVIDIA Docsdocs.nvidia.com
- Python API reference — nvMatmulHeuristicsdocs.nvidia.com
- Outperforming cuBLAS on H100: a Worklogcudaforfun.substack.com
- Outperforming cuBLAS on H100: a Worklogcudaforfun.substack.com
- Mini Project: GPU Accelerated Matrix Multiplication (almost) like cuBLAS0mean1sigma.com
- How to Think About TPUs | How To Scale Your Modeljax-ml.github.io
- CUTLASS: Fast Linear Algebra in CUDA C++ | NVIDIA Technical Blogdeveloper.nvidia.com
- All About Rooflines | How To Scale Your Modeljax-ml.github.io