Your RAM Has a 60 Year Old Design Flaw. I Bypassed It. - YouTube
Explore how hidden, periodic freezes in computer memory impact performance. Discover techniques to measure these tiny disruptions and learn to engineer solutions that bypass them for faster data access. Follow along using the transcript. Explore how hidden, periodic freezes in computer memory impact performance. Discover techniques to measure these tiny disruptions and learn to engineer solutions that bypass them for faster data access. Follow along using the transcript.
Modern DRAM is based on a brilliant design from IBM. But, we're still paying for a latency penalty that's existed since the 60s! In this video, I'm introducing my personal research project (Tailslayer) that immensely reduces p99.99 latency on traditional RAM! By implementing a hedged read strategy taking advantage of (undocumented!) channel scrambling offsets, I've gotten as much as 15x reductions in tail latency. The technique works across Intel, AMD, Graviton, DDR4, DDR5, x86, ARM, you name it. Check out the C++ lib I wrote, watch the video, and try it yourself! --- Timestamps:…
saved by
related reading
- What Every Programmer Should Know About Memorypeople.freebsd.org
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
- What Every Programmer Should Know About Memoryakkadia.org
- abseil / Performance Hintsabseil.io
- research!rsc: Hardware Memory Models (Memory Models, Part 1)research.swtch.com
- Reading privileged memory with a side-channel - Project Zerogoogleprojectzero.blogspot.com
- GitHub - xoreaxeaxeax/asm-hall-of-shame: Racing to the bottom of CPU performancegithub.com
- A very concrete explanation of how a cache worksparksb.github.io
- What happens when a GPU reads memory | Doublewordblog.doubleword.ai
- A Close Look at SRAM for Inference in the Age of HBM Supremacyviksnewsletter.com
- Production Twitter on One Machine? 100Gbps NICs and NVMe are fast - Tristan Humethume.ca
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev