Scaling: The State of Play in AI - by Ethan Mollick
oneusefulthing.org · 2,504 words · saved by 3 readers
A brief intergenerational pause...
Now feels like a good time to lay out where we are with AI, and what might come next. I want to focus purely on the capabilities of AI models, and specifically the Large Language Models that power chatbots like ChatGPT and Gemini. These models keep getting “smarter” over time, and it seems worthwhile to consider why, as that will help us understand what comes next. Doing so requires diving into how models are trained. I am going to try to do this in a non-technical way, which means that I will ignore a lot of important nuances that I hope my more technical readers forgive me for. To…
saved by
related reading
- Unresolved debates about the future of AIhelentoner.substack.com
- Will scaling work?dwarkeshpatel.com
- How To Scale Your Modeljax-ml.github.io
- The Scaling Hypothesis · Gwern.netgwern.net
- The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
- I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
- Scaling is subtler than it seemsberen.io
- AI scaling mythsnormaltech.ai
- Demystify Transformers: A Guide to Scaling Laws | by Yu-Cheng Tsai | Sage Ai | Mediummedium.com
- Will scaling work? - by Dwarkesh Patel - Dwarkesh Podcastdwarkesh.com
- GPT-4openai.com
- AI progress is about to speed up | Epoch AIepoch.ai