What Limits Performance of (PyTorch) Neural Networks when running on a CPU? | B. Nikolic Software and Computing Blog
This is a little experiment to use CPU performance monitoring counters to find out what limits the maximum performance of PyTorch Neural Networks when running on a CPU.
This is a little experiment to use CPU performance monitoring counters to find out what limits the maximum performance of PyTorch Neural Networks when running on a CPU. The experiment was carried out in the following way: Performance metrics were calculated as described in the post describing counting PMU events in Python and the post applying this to PyTorch . Measurements were made for all torchvision models, and repeated 10 times to obtain an average. The variety of models is used in order to have a range of algorithms so that the limiting factor becomes more apparent. Measurements were mad
Explore this link on the map →related reading
- Making Deep Learning go Brrrr From First Principleshorace.io
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- Counting FLOPS in PyTorch using CPU PMU counters | B. Nikolic Software and Computing Blogbnikolic.co.uk
- How To Scale Your Modeljax-ml.github.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- GPU Performance Background User's Guide - NVIDIA Docsdocs.nvidia.com
- PiTorch: ML on Baremetal Raspberry Pis | projectsmasonjwang.com
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- Latest | Epoch AIepochai.org
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short]thonking.ai
- Memory-Limited Layers User's Guide - NVIDIA Docsdocs.nvidia.com