New Paper: Towards a science of AI agent reliability
normaltech.ai · 2,179 words · saved by 1 readers
Quantifying the capability-reliability gap
By Stephan Rabanser, Sayash Kapoor, Arvind Narayanan Suppose you hear about a new AI agent for improving productivity — by making purchases, or writing code, or sending emails, or handling a customer on your behalf. Should you trust it? Can the agent do the job reliably enough? After all, there are many horror stories of agents going wrong. Surprisingly, even though the lack of reliability of AI agents is well known, right now the AI industry doesn’t have good tools for measuring reliability, or even a good definition of reliability. Arvind and Sayash have long been thinking about this.…
related reading
- Why I'm Betting Against AI Agents in 2025 (Despite Building Them)utkarshkanwat.com
- Demystifying evals for AI agents \ Anthropicanthropic.com
- Seoul Alignment Workshop 2026: What We Learnedfar.ai
- Measuring AI Ability to Complete Long Tasks - METRmetr.org
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- How fast is AI improving? - AI Digesttheaidigest.org
- Agent Evaluation: A Detailed Guidecameronrwolfe.substack.com
- ACT-2 Preview: Generalizing Reliability | Sunday Robotics | The helpful robotics companysunday.ai
- What will be left for us to work on?normaltech.ai
- New paper: AI agents that matternormaltech.ai
- Current AIs seem pretty misaligned to meblog.redwoodresearch.org
- The 2025 AI Agent Indexaiagentindex.mit.edu