PiTorch: ML on Baremetal Raspberry Pis | projects
How do you get from a $5 computer to a working language model? We strip away every layer of abstraction and build up from scratch, **running and training models** on a cluster of Pi Zeros. No PyTorch, no OS, not even a standard library.
The Raspberry Pi Zero is a $5 computer with less processing power than a thermostat [ 1 ] . What does it take to run an LLM on it? Train one? How about on four of them, wired together? PiTorch is a baremetal ML framework for the Raspberry Pi Zero's GPU. LLM inference runs 210x faster than the CPU baseline, training 25x. It scales to 4 linked Pis with communication overhead as low as 0.3% of compute time. prog weights state stacks GPU 61× 185× 210× I use interactive diagrams throughout the post to break down the ideas I found most fascinating :) I spent the last few weeks building PiTorch , a f
Explore this link on the map →saved by
related reading
- How To Scale Your Modeljax-ml.github.io
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- How is LLaMa.cpp possible?finbarr.ca
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- Pipeline-Parallelism: Distributed Training via Model Partitioningsiboehm.com
- Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
- Tiny TPUtinytpu.com
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- Training LLMs with AMD MI250 GPUs and MosaicML | Databricks Blogmosaicml.com