Measuring AI Ability to Complete Long Tasks - METR
We propose measuring AI performance in terms of the *length* of tasks AI agents can complete. We show that this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months. Extrapolating this trend predicts that, in under a decade, we will see AI agents that can independently complete a large fraction of software tasks that currently take humans days or weeks.
Measuring AI Ability to Complete Long Tasks - METR Our Work Research Notes Updates Risk Assessment About Donate Careers Search --> Our Work Research Notes Updates Risk Assessment About Donate Careers Menu × Measuring AI Ability to Complete Long Tasks We propose measuring AI performance in terms of the length of tasks AI agents can complete. We show that this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months. Extrapolating this trend predicts that, in under a decade, we will see AI agents that can independently complete a
saved by
related reading
- Measuring AI Ability to Complete Long Software Tasksarxiv.org
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METRmetr.org
- How Does Time Horizon Vary Across Domains? - METRmetr.org
- METRmetr.org
- 2503.14499arxiv.org
- Where’s my ten minute AGI?epochai.substack.com
- We spent 2 hours working in the future - METRmetr.org
- Is there a Half-Life for the Success Rates of AI Agents? - Toby Ordtobyord.com
- Metrics of Agent Ability - METRmetr.org
- A (Slightly) Mechanistic Theory for Exponentially Increasing AI Time Horizons? — LessWronglesswrong.com
- Where’s my ten minute AGI?epochai.substack.com
- I underestimated AI capabilities (again) - by Ajeya Cotraplanned-obsolescence.org