Unpacking decentralized training - knower's substack
First things first, SPECIAL SHOUTOUT to sam lehman, rodeo, haus, yb, smac, ronan, and ibuyrugs for all of the comments, edits, feedback, and suggestions - you helped bring this to life and I really appreciate it. Also, some of these arxiv links open up as browser-based PDFs, so just a warning incase you don’t want to deal with that. As I’m writing this it’s been three months since my last post. What have I been up to since then? I don’t know. I’ve been doing a lot of reading, trying to workout like five days a week, and generally making the most of my last semester attending college. My mind gets a bit antsy anytime it’s been a month or two since I’ve written a long report, so this is my attempt at returning to baseline and getting back in the groove of things. If you couldn’t tell by the title, this is a report that’s largely about distributed/decentralized training, accompanied by some info covering what’s been happening in the world of AI, and some commentary about how all of this f
Unpacking decentralized training My attempt at highlighting the necessity of training LLMs in a distributed / decentralized manner, explaining a web of topics, and articulating the potential synergies between crypto + AI knower Apr 08, 2025 34 3 2 Share Where have I been? First things first, SPECIAL SHOUTOUT to sam lehman , rodeo , haus , yb , smac , ronan , and ibuyrugs for all of the comments, edits, feedback, and suggestions - you helped bring this to life and I really appreciate it. Also, some of these arxiv links open up as browser-based PDFs, so just a warning incase you don’t want to de
Explore this link on the map →related reading
- How To Scale Your Modeljax-ml.github.io
- The Short Case for Nvidia Stock | YouTube Transcript Optimizeryoutubetranscriptoptimizer.com
- The Scaling Hypothesis · Gwern.netgwern.net
- Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructuresemianalysis.com
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
- Questions about the Future of AI - by Dwarkesh Pateldwarkesh.com
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
- My picture of the present in AI — LessWronglesswrong.com
- The Little Book of Deep Learningfleuret.org
- Nirmaan's AI Thesis — Nirmaanmirror.xyz
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com