Hot Chips 2026: Applying High Bandwidth Flash (HBF)
Machine learning workloads have an insatiable appetite for DRAM capacity. Flash memory is cheaper per gigabyte of capacity than DRAM. Could it offer a way out?
HBF, or High Bandwidth Flash, uses the same flash memory technology we see in SSDs today. Unlike SSDs, HBF is implemented much like HBM (High Bandwidth Memory). HBF cubes sit on the same package as a compute chip, perhaps even next to HBM. HBF’s idea is to offer much higher capacity than HBM, while still providing decent memory bandwidth. At Hot Chips 2026 tutorials day, Anurag Agarwal and Radhakrishna Giduthuri’s talk explores how HBF could apply to machine learning workloads. No HBF products exist yet, so the talk focuses on simulations, projections, and how software can adapt to take…
saved by
related reading
- Are you a memory_man, Part 2 - by Investing with Martinmartinbradstreet.substack.com
- AI Chip Architecturesjepeake.com
- Interview with Richard Ho, OpenAImorethanmoore.substack.com
- The Inference Shift – Stratechery by Ben Thompsonstratechery.com
- Hot Chipshotchips.org
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
- How to Rack 30 Petabytes of Storage | blogsi.inc
- [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessarxiv.org
- A Close Look at SRAM for Inference in the Age of HBM Supremacyviksnewsletter.com
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- A Close Look at SRAM for Inference in the Age of HBM Supremacyviksnewsletter.com
- Horace He: Building Machine Learning Systems for a Trillion Trillion Floating Point Operationsyoutube.com