Being GPU Poor makes you creative

I started my sabbatical pretty GPU poor. Going from “whatever compute I needed at Apple” to “my MacBook” was a real drop. Then I walked into the Recurse Center hub in NYC for the first time, and so

I started my sabbatical pretty GPU poor. Going from “whatever compute I needed at Apple” to “my MacBook” was a real drop. Then I walked into the Recurse Center hub in NYC for the first time, and someone casually mentioned they had “a couple of GPUs” sitting around. That got my attention. The catch was that the hardware was not homogeneous. RC had a couple of NVIDIA GeForce GTX TITAN X cards, and I’d brought my M4 Max MacBook with Apple Silicon. Different vendors, different drivers, different memory architectures, different runtimes. The classic “how do I get all of these to cooperate?” problem

saved by

• C LI

related reading

• Making Deep Learning go Brrrr From First Principleshorace.io
• The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
• From Single-Node to Multi-GPU Clusters: How Discord Made Distributed Compute Easy for ML Engineersdiscord.com
• Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
• A Hitchhiker’s Guide to ML Training Infrastructure | CMU Software Engineering Institutesei.cmu.edu
• How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
• The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
• Horace He: Building Machine Learning Systems for a Trillion Trillion Floating Point Operationsyoutube.com
• PiTorch: ML on Baremetal Raspberry Pis | projectsmasonjwang.com
• RL Post-Training on Macs | Pluralis Researchpluralis.ai
• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
• Pipeline-Parallelism: Distributed Training via Model Partitioningsiboehm.com