Dwarkesh Patel on X: "New blackboard lecture w @reinerpope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of https://t.co/OhZlNxdYyz" / X
New blackboard lecture w @reinerpope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of https://t.co/OhZlNxdYyz
Dwarkesh Patel @dwarkesh_sp New blackboard lecture w @ reinerpope How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. 0:00:00 – Building a multiply-accumulate from logic gates 0:16:20 – Muxes and the cost of data movement 0:25:59 – How systolic arrays work 0:39:00 – Clock cycles and pipeline registers 0:51:40 – FPGAs vs ASICs 1:03:14 – Cache vs scratchpad 1:07:16 – Why CPU cores are much bigger than GPU cores 1:11:49 – Brains vs chips 1:15:22 – A GPU is just a bunch of tiny TPUs Look up Dwarkesh P
Explore this link on the map →related reading
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- Tiny TPUtinytpu.com
- TPU Deep Divehenryhmko.github.io
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- How to Think About TPUs | How To Scale Your Modeljax-ml.github.io
- Could a Neuroscientist Understand a Microprocessor? | PLOS Computational Biologyjournals.plos.org
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- Reiner Pope of MatX on accelerating AI with transformer-optimized chipscheekypint.substack.com
- Touching the Elephant - TPUs | Consider the Bulldogconsiderthebulldog.com
- Mvidiajaso1024.com
- General-purpose computing on graphics processing units - Wikipediaen.wikipedia.org
- All About Rooflines | How To Scale Your Modeljax-ml.github.io