How to easily measure Floating Point Operations Per Second (FLOPS) | B. Nikolic Software and Computing Blog
The hard way of measuring FLOPS is to modify your program so that it itself keeps track of the number of floating operations performed in each module/function, run it on your target hardware and finally divide the two numbers. But, this requires possibly extensive modification to the program, and if it is done at too granular a level (i.e., in too tight a loop) it can affect the performance of the program.
The hard way of measuring FLOPS is to modify your program so that it itself keeps track of the number of floating operations performed in each module/function, run it on your target hardware and finally divide the two numbers. But, this requires possibly extensive modification to the program, and if it is done at too granular a level (i.e., in too tight a loop) it can affect the performance of the program. A much easier way of measuring FLOPS for a particular combination of program and hardware is to use the CPU performance counters, now very conveniently accessible under Linux using the perf
related reading
- Floating point operations per second - Wikipediaen.wikipedia.org
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- PERF tutorial: Finding execution hot spots - Sand, software and soundsandsoftwaresound.net
- PERF tutorial: Counting hardware performance events - Sand, software and soundSand, software and soundsandsoftwaresound.net
- PERF tutorial: Finding execution hot spots - Sand, software and soundSand, software and soundsandsoftwaresound.net
- Making Deep Learning go Brrrr From First Principleshorace.io
- Estimating training compute of deep learning models | Epoch AIepochai.org
- abseil / Performance Hintsabseil.io
- Microsoft Word - CACM_roofline_complete.docusers.cs.duke.edu
- There are Only Four Billion Floats-So Test Them All! | Random ASCII – tech blog of Bruce Dawsonrandomascii.wordpress.com
- GitHub - xoreaxeaxeax/asm-hall-of-shame: Racing to the bottom of CPU performancegithub.com