Memory location matters for performance
That’s what I learned long ago in school, anyway, when I took algorithms and data structures classes. The basic assumption was that reading or writing to memory took a small, constant amount of time, and so you could focus on how many operations were being done. This was so fundamental to how we were taught that we didn’t even have to write real code: pseudo-code was sufficient. Unfortunately, this is not how a CPU works in the real world. In the real world, access to memory is vastly slower than CPU instructions, and all those O(N) measurements that just focus on CPU operations are insufficient to tell you what real-world performance will be. And that means you need to think about not just your computation, but also how you access memory. An updated version of this article is included in a book I’m working on that will help teach you how to optimize low-level code, the kind of code you’d write with Cython, C, or Rust. The goal is to help data scientists and scientists who normally wri
Memory location matters for performance Memory location matters for performance by Itamar Turner-Trauring Last updated 22 Nov 2023, originally created 13 Jan 2022 If you’re writing high-performance code, what matters is how many CPU instructions you run. Right? That’s what I learned long ago in school, anyway, when I took algorithms and data structures classes. The basic assumption was that reading or writing to memory took a small, constant amount of time, and so you could focus on how many operations were being done. This was so fundamental to how we were taught that we didn’t even have to w
Explore this link on the map →related reading
- abseil / Performance Hintsabseil.io
- Memory access is O(N^[1/3])vitalik.eth.limo
- Infographics: Operation Costs in CPU Clock Cycles - 6IT6it.dev
- theartofhpc.com/istc/sequential.html#ThevonNeumannarchitecturetheartofhpc.com
- What Every Programmer Should Know About Memorypeople.freebsd.org
- Algorithms for Modern Hardware - Algorithmicaen.algorithmica.org
- abseil / Performance Hintsabseil.io
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- The compiler will optimize that away | RoyalSlothblog.royalsloth.eu
- Making Deep Learning go Brrrr From First Principleshorace.io
- Rust Optimization.md · GitHubgist.github.com
- sled theoretical performance guide | sled-rs.github.iosled.rs