Scaling: The State of Play in AI - by Ethan Mollick
oneusefulthing.org · 2,504 words · saved by 3 readers
A brief intergenerational pause...
Now feels like a good time to lay out where we are with AI, and what might come next. I want to focus purely on the capabilities of AI models, and specifically the Large Language Models that power chatbots like ChatGPT and Gemini. These models keep getting “smarter” over time, and it seems worthwhile to consider why, as that will help us understand what comes next. Doing so requires diving into how models are trained. I am going to try to do this in a non-technical way, which means that I will ignore a lot of important nuances that I hope my more technical readers forgive me for. To…
saved by
related reading
- Will scaling work?dwarkeshpatel.com
- AI progress is about to speed up | Epoch AIepoch.ai
- AI in 2025: gestalt — LessWronglesswrong.com
- The Scaling Hypothesis · Gwern.netgwern.net
- The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com
- My picture of the present in AI — LessWronglesswrong.com
- I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
- GenAI Handbookgenai-handbook.github.io
- Will scaling work? - by Dwarkesh Patel - Dwarkesh Podcastdwarkesh.com
- AI scaling mythsnormaltech.ai
- Scaling is subtler than it seemsberen.io
- Things we learned about LLMs in 2024simonwillison.net