f316275b44ee2de533102913828a8107-Paper-Datasets_and_Benchmarks_Track.pdf
proceedings.neurips.cc · 6,410 words · saved by 1 readers
N/A
Establishing Best Practices for Building Rigorous Agentic Benchmarks Yuxuan Zhu1∗ Tengjun Jin1 Yada Pruksachatkun Andy Zhang2 Shu Liu3 Sasha Cui4 Sayash Kapoor5 Shayne Longpre6 Kevin Meng7 Rebecca Weiss8 Fazl Barez8,11 Rahul Gupta9 Jwala Dhamala9 Jacob Merizian10 Mario Giulianelli10 Harry Coppock10 Cozmin Ududec10 Jasjeet Sekhon4 Jacob Steinhardt7 Antony Kellermann1 Sarah Schwettmann7 Matei Zaharia3 Ion Stoica3 Percy Liang2 Daniel Kang1∗ 1 2 UIUC…
saved by
related reading
- Demystifying evals for AI agents \ Anthropicanthropic.com
- Center for Responsible, Decentralized Intelligence at Berkeleyrdi.berkeley.edu
- Are AI benchmarks doomed? - by Anson Ho and Greg Burnhamepochai.substack.com
- Agent Evaluation: A Detailed Guidecameronrwolfe.substack.com
- Giovanni D'Antoniogiovannidantonio.com
- Senior SWE-Benchsenior-swe-bench.snorkel.ai
- New paper: AI agents that matternormaltech.ai
- Quantifying infrastructure noise in agentic coding evals \ Anthropicanthropic.com
- [2606.05405] Agents' Last Examarxiv.org
- AI’s capabilities may be exaggerated by flawed tests, study saysnbcnews.com
- A Few Things I Learned About Evals - Ryan Bloomryanbloom.xyz
- AI agent evaluation frameworks for production - Vercelvercel.com