Building Machine Learning Systems for a Trillion Trillion Floating Point Operations - YouTube
youtube.com · 10,940 words · saved by 6 readers
Follow along using the transcript. Follow along using the transcript. New New New New New
Over the last 10 years we've seen Machine Learning consume everything, from the tech industry to the Nobel Prize, and yes, even the ML acronym. This rise in ML has also come along with an unprecedented buildout of infra, with Llama 3 now reaching 4e25 floating point operations, or 40 yottaflops, or 40 trillion trillion floating point operations. To build these ML models, you need ML systems, like PyTorch. In this talk, Horace will (attempt to) answer: - How have ML systems evolved over time to meet the training needs of ML models? How does building ML systems differ from regular systems? -…
saved by
related reading
- Making Deep Learning go Brrrr From First Principleshorace.io
- How To Scale Your Modeljax-ml.github.io
- A friendly introduction to machine learning compilers and optimizershuyenchip.com
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- Lecture 117:Building a GPU Compiler from Scratchyoutube.com
- Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short]thonking.ai
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- Modern GPU Programming For MLSys — Modern GPU Programming For MLSysmlc.ai
- How trillion-parameter AI models work and how to serve themyoutube.com
- How GPT, Claude, and Gemini are actually trained and served – Reiner Popeyoutube.com