✳flâneur — a map of the web's best reading

Is there a Half-Life for the Success Rates of AI Agents? — Toby Ord

tobyord.com · 3,154 words · saved by 1 readers

Building on the recent empirical work of Kwa et al. (2025), I show that within their suite of research-engineering tasks the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take

Is there a Half-Life for the Success Rates of AI Agents? May 7, 2025 Toby Ord Building on the recent empirical work of Kwa et al. (2025), I show that within their suite of research-engineering tasks the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task. This implies an exponentially declining success rate with the length of the task and that each agent could be characterised by its own half-life. This empirical regularity allows us to estimate the success

Explore this link on the map →

saved by

related reading