flâneur

Where’s my ten minute AGI? - by Anson Ho - Epoch AI

epochai.substack.com · 2,378 words · saved by 1 readers

Recently, METR released a paper arguing that the length of tasks that AIs can do is doubling every 7 months. We can see this in the following graph, where the best AI system1 is able to do roughly hour-long tasks at a 50% success rate on average: But there’s a big problem here – if AIs are actually able to perform most tasks on 1-hour task horizons, why don’t we see more real-world task automation? For example, most emails take less than an hour to write, but crafting emails remains an important part of the lives of billions of people every day. Some of this could be due to people underusing AI systems,2 but in this post I want to focus on reasons that are more fundamental to the capabilities of AI systems. In particular, I think there are three such reasons that are the most important: Time-horizon estimates are very domain-specific Task reliability strongly influences task horizons Tasks are very bundled together and hard to separate out. While it’s hard to be quantitative about just

Recently, METR released a paper arguing that the length of tasks that AIs can do is doubling every 7 months. We can see this in the following graph, where the best AI system1 is able to do roughly hour-long tasks at a 50% success rate on average: METR’s research finds that AIs are rapidly able to do longer and longer tasks, where length is measured by the time it takes for a human with requisite expertise to do the task. But there’s a big problem here – if AIs are actually able to perform most tasks on 1-hour task horizons, why don’t we see more real-world task automation? For example,…

related reading