1.1 - Fermi estimate of future training runs

Thanks to Paul Christiano, David Roodman, Joe Carlsmith, and Jared Kaplan for feedback on early drafts of this page, and to Miles Brundage for corrections.

Fermi estimate of future training runs Global risk from deep learning [ List of all pages ] About this site 1 - The case for risk 1.1 - Fermi estimate of future training runs # Budgets and total FLOP of future training runs # Model size via scaling laws # Upper bounds for brain-equivalent capabilities # Training data requirements 1.2 - Applications of high-capability models Coming next - Outline of a research program Get updates via mailing list Last update: 2 August 2022 Daniel Dewey Thanks to Paul Christiano, David Roodman, Joe Carlsmith, and Jared Kaplan for feedback on early drafts of this

saved by

• Alex K. Chen
• Yudhister Joel Kumar
• Danish

related reading

• Estimating training compute of deep learning models | Epoch AIepochai.org
• Can AI scaling continue through 2030? | Epoch AIepoch.ai
• The Scaling Hypothesis · Gwern.netgwern.net
• BCIs and the ecosystem of modular mindsberen.io
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• The nature of LLM algorithmic progress (v2) — LessWronglesswrong.com
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
• Will scaling work?dwarkeshpatel.com
• I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
• AI in 2025: gestalt — LessWronglesswrong.com
• The Extreme Inefficiency of RL for Frontier Models - Toby Ordtobyord.com