Sparse Matrix-Vector Multiplication with CUDA | by Georgii Evtushenko | Analytics Vidhya | Medium
Standard methods of differential equations discretization usually lead to systems of linear equations. General feature of produced systems is that the number of entries in each equation depends on local topological features of the discretization. Thus, the matrices generated by these systems contain a lot of zeroes (fig. 1). It’s possible to take advantage of knowledge about position of zeroes by storing matrices in special data structures. The abstract data type for these structures is called sparse matrix. While I was reading about yet another matrix format, I decided to actualize the comparison of performances of different matrix formats. This post provides an review of efficiency for basic sparse matrix data structures in the context of sparse matrix-vector multiplication (SpMV) on GPU. In general, SpMV performance is limited by memory bandwidth. The storage formats, which are used for the sparse matrices define SpMV algorithms. Each of these algorithms has its own granularity, whi
Sparse Matrix-Vector Multiplication with CUDA Georgii Evtushenko 10 min read · Nov 16, 2019 -- 1 Listen Share Press enter or click to view image in full size Introduction Standard methods of differential equations discretization usually lead to systems of linear equations. General feature of produced systems is that the number of entries in each equation depends on local topological features of the discretization. Thus, the matrices generated by these systems contain a lot of zeroes (fig. 1). It’s possible to take advantage of knowledge about position of zeroes by storing matrices in special d
Explore this link on the map →saved by
related reading
- Block Sparse Matrix-Vector Multiplication with CUDA | by Georgii Evtushenko | GPGPU | Mediummedium.com
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- A history of NVidia Stream Multiprocessorfabiensanglard.net
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- Sparse matrix - Wikipediaen.wikipedia.org
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- Learning CUDA by optimizing matrix-vector multiplication (SGEMV) for cuBLAS-like performance - A worklog – Maharshi's blogmaharshi.bearblog.dev
- Mini Project: GPU Accelerated Matrix Multiplication (almost) like cuBLAS0mean1sigma.com
- GitHub - wangzyon/NVIDIA_SGEMM_PRACTICE: Step-by-step optimization of CUDA SGEMM · GitHubgithub.com
- CUTLASS: Fast Linear Algebra in CUDA C++ | NVIDIA Technical Blogdeveloper.nvidia.com
- Outperforming cuBLAS on H100: a Worklogcudaforfun.substack.com
- Worklog: Optimising GEMM on NVIDIA H100 for cuBLAS-like Performance (WIP) – Hamza's Bloghamzaelshafie.bearblog.dev