How to easily measure Floating Point Operations Per Second (FLOPS) | B. Nikolic Software and Computing Blog
The hard way of measuring FLOPS is to modify your program so that it itself keeps track of the number of floating operations performed in each module/function, run it on your target hardware and finally divide the two numbers. But, this requires possibly extensive modification to the program, and if it is done at too granular a level (i.e., in too tight a loop) it can affect the performance of the program.
The hard way of measuring FLOPS is to modify your program so that it itself keeps track of the number of floating operations performed in each module/function, run it on your target hardware and finally divide the two numbers. But, this requires possibly extensive modification to the program, and if it is done at too granular a level (i.e., in too tight a loop) it can affect the performance of the program. A much easier way of measuring FLOPS for a particular combination of program and hardware is to use the CPU performance counters, now very conveniently accessible under Linux using the perf
Explore this link on the map →related reading
- Floating point operations per second - Wikipediaen.wikipedia.org
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- PERF tutorial: Counting hardware performance events - Sand, software and soundSand, software and soundsandsoftwaresound.net
- PERF tutorial: Finding execution hot spots - Sand, software and soundSand, software and soundsandsoftwaresound.net
- Making Deep Learning go Brrrr From First Principleshorace.io
- Estimating training compute of deep learning models | Epoch AIepochai.org
- abseil / Performance Hintsabseil.io
- There are Only Four Billion Floats-So Test Them All! | Random ASCII – tech blog of Bruce Dawsonrandomascii.wordpress.com
- pyLambdaFlows/.github/workflows/main.yml at master · Enderdead/pyLambdaFlows · GitHubgithub.com
- What Limits Performance of (PyTorch) Neural Networks when running on a CPU? | B. Nikolic Software and Computing Blogbnikolic.co.uk
- All About Rooflines | How To Scale Your Modeljax-ml.github.io