Floating Point and IEEE 754
A number of issues related to floating point accuracy and compliance are a frequent source of confusion on both CPUs and GPUs. The purpose of this white paper is to discuss the most common issues related to NVIDIA GPUs and to supplement the documentation in the CUDA C++ Programming Guide.
1. Introduction — Floating Point and IEEE 754 13.3 documentation » 1. Introduction v13.3 | PDF | Archive Floating Point and IEEE 754 Compliance for NVIDIA GPUs White paper covering the most common issues related to NVIDIA GPUs. A number of issues related to floating point accuracy and compliance are a frequent source of confusion on both CPUs and GPUs. The purpose of this white paper is to discuss the most common issues related to NVIDIA GPUs and to supplement the documentation in the CUDA C++ Programming Guide. 1. Introduction Since the widespread adoption in 1985 of the IEEE Standard for B
related reading
- Floating point from scratch: Hard Mode · Tales on the wireessenceia.github.io
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- CUDA C++ Programming Guide (Legacy) — CUDA C++ Programming Guidedocs.nvidia.com
- CUDA - Wikipediaen.wikipedia.org
- Adding Floating-Point Decimals for Fun and Profitblog.vero.site
- GPU Performance Background User's Guide - NVIDIA Docsdocs.nvidia.com
- There are Only Four Billion Floats-So Test Them All! | Random ASCII – tech blog of Bruce Dawsonrandomascii.wordpress.com
- We reverse-engineered Flash Attention 4modal.com
- General-purpose computing on graphics processing units - Wikipediaen.wikipedia.org
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- Ken Shirriff's blogrighto.com