2503.14499
arxiv.org · 8,223 words · saved by 1 readers
N/A
Measuring AI Ability to Complete Long Software Tasks Thomas Kwa∗†, Ben West∗, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Harold Miles‡, Seraphina Nix, Tao Lin, Chris…
related reading
- Measuring AI Ability to Complete Long Software Tasksarxiv.org
- Measuring AI Ability to Complete Long Tasks - METRmetr.org
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METRmetr.org
- We spent 2 hours working in the future - METRmetr.org
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- Clarifying and predicting AGI — LessWronglesswrong.com
- How Does Time Horizon Vary Across Domains? - METRmetr.org
- 2502.18449arxiv.org
- Where’s my ten minute AGI?epochai.substack.com
- A (Slightly) Mechanistic Theory for Exponentially Increasing AI Time Horizons? — LessWronglesswrong.com
- Predicting Empirical AI Research Outcomes with Language Modelsarxiv.org
- [2603.11214] Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenariosarxiv.org