AI Chip Architectures - Jacob Peake

A look at AI Chip Architectures. NVIDIA, AMD, TPUs, Trainium, Groq, Cerebras.

At the 2018 International Symposium on Computer Architecture, John Hennessy and David Patterson delivered their Turing Lecture: "A New Golden Age for Computer Architecture". In the 1980s, when Hennessy and Patterson did their Turing Award-winning research, single-threaded CPU performance grew 52% a year. By 2018, with the end of Moore's Law and Dennard Scaling, the rate was 3%. There was a need for domain-specific architectures (DSAs). Their worked example was Google's TPU v1, already in production: 29× the throughput of a CPU on neural-network inference, at 80× better energy efficiency.…

saved by

• Nicklaus Tran
• Lailah Nabegu
• Yafik Ra
• nathan karri

related reading

• AI Chip Architecturesjepeake.com
• How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
• TPU Deep Divehenryhmko.github.io
• How to Think About TPUs | How To Scale Your Modeljax-ml.github.io
• The Inference Shift – Stratechery by Ben Thompsonstratechery.com
• Meta harness makes 10 times better Kimi K3 chipluoluo.ai
• Chip design from the bottom up – Reiner Popeyoutube.com
• Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
• GitHub - wafer-ai/gpu-perf-engineering-resources: A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.github.com
• Touching the Elephant - TPUs | Consider the Bulldogconsiderthebulldog.com
• The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
• Tiny TPUtinytpu.com