Is there a Half-Life for the Success Rates of AI Agents? — Toby Ord
Building on the recent empirical work of Kwa et al. (2025), I show that within their suite of research-engineering tasks the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take
Is there a Half-Life for the Success Rates of AI Agents? May 7, 2025 Toby Ord Building on the recent empirical work of Kwa et al. (2025), I show that within their suite of research-engineering tasks the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task. This implies an exponentially declining success rate with the length of the task and that each agent could be characterised by its own half-life. This empirical regularity allows us to estimate the success
saved by
related reading
- Measuring AI Ability to Complete Long Tasks - METRmetr.org
- A (Slightly) Mechanistic Theory for Exponentially Increasing AI Time Horizons? — LessWronglesswrong.com
- My picture of the present in AI — LessWronglesswrong.com
- AI 2027ai-2027.com
- Measuring AI Ability to Complete Long Software Tasksarxiv.org
- 2503.14499arxiv.org
- We spent 2 hours working in the future - METRmetr.org
- Clarifying and predicting AGI — LessWronglesswrong.com
- How Does Time Horizon Vary Across Domains? - METRmetr.org
- Where’s my ten minute AGI?epochai.substack.com
- I underestimated AI capabilities (again) - by Ajeya Cotraplanned-obsolescence.org
- Where’s my ten minute AGI?epochai.substack.com