Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blog
PyTorch has a useful third-party module THOP which calculates the number of floating point (multiply/accumulate) operations needed to make an inference from a PyTorch neural network model. Here I compare THOP estimates of FLOPs to measurements made using CPU Performance monitors in order to cross-validate both techniques.
PyTorch has a useful third-party module THOP which calculates the number of floating point (multiply/accumulate) operations needed to make an inference from a PyTorch neural network model. Here I compare THOP estimates of FLOPs to measurements made using CPU Performance monitors in order to cross-validate both techniques. THOP works by having a registry of simple functions that predict the number of FLOPs needed for each stage of neural networks. The registry is pre-populated with following neural network stages: nn . Conv1d nn . Conv2d nn . Conv3d nn . ConvTranspose1d nn . ConvTranspose2d nn
related reading
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- Estimating training compute of deep learning models | Epoch AIepochai.org
- Making Deep Learning go Brrrr From First Principleshorace.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- What Limits Performance of (PyTorch) Neural Networks when running on a CPU? | B. Nikolic Software and Computing Blogbnikolic.co.uk
- How to easily measure Floating Point Operations Per Second (FLOPS) | B. Nikolic Software and Computing Blogbnikolic.co.uk
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- All the Transformer Math You Need to Know | How To Scale Your Modeljax-ml.github.io
- Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short]thonking.ai
- How to Think About TPUs | How To Scale Your Modeljax-ml.github.io
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com