PiTorch: ML on Baremetal Raspberry Pis | projects
How do you get from a $5 computer to a working language model? We strip away every layer of abstraction and build up from scratch, **running and training models** on a cluster of Pi Zeros. No PyTorch, no OS, not even a standard library.
The Raspberry Pi Zero is a $5 computer with less processing power than a thermostat [ 1 ] . What does it take to run an LLM on it? Train one? How about on four of them, wired together? PiTorch is a baremetal ML framework for the Raspberry Pi Zero's GPU. LLM inference runs 210x faster than the CPU baseline, training 25x. It scales to 4 linked Pis with communication overhead as low as 0.3% of compute time. prog weights state stacks GPU 61× 185× 210× I use interactive diagrams throughout the post to break down the ideas I found most fascinating :) I spent the last few weeks building PiTorch , a f
saved by
related reading
- Making Deep Learning go Brrrr From First Principleshorace.io
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- How To Scale Your Modeljax-ml.github.io
- How is LLaMa.cpp possible?finbarr.ca
- Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
- microgptkarpathy.github.io
- Tiny TPUtinytpu.com
- PyTorch in One Hour: From Tensors to Training Neural Networks on Multiple GPUssebastianraschka.com
- Training LLMs with AMD MI250 GPUs and MosaicML | Databricks Blogmosaicml.com
- Let's build GPT: from scratch, in code, spelled out.youtube.com
- GitHub - karpathy/nanochat: The best ChatGPT that $100 can buy.github.com
- Accelerating Generative AI with PyTorch II: GPT, Fast – PyTorchpytorch.org