withhumans.pdf
gleech.org · 6,314 words · saved by 1 readers
N/A
Position: AI Evaluation Should Work With Humans Jan Kulveit 1 Gavin Leech 2 Tomáš Gavenčiak 1 Raymond Douglas 1 Abstract This position paper argues that the dominant paradigm of AI evaluation (which focuses on su- perhuman autonomous performance and so im- plicitly targets the goal of replacing humans) is guiding AI development in the wrong direction. Instead, the AI community should pivot to eval- uating the performance of human–AI teams. We argue that this collaborative shift will foster AI…
saved by
related reading
- Demystifying evals for AI agents \ Anthropicanthropic.com
- The Future Worth Building Is Human - Thinking Machines Labthinkingmachines.ai
- What will be left for us to work on?normaltech.ai
- Things I learned at OpenAI - by Karina Nguyen - sémaphoresemaphore.substack.com
- [1911.01547] On the Measure of Intelligencearxiv.org
- Are AI benchmarks doomed? - by Anson Ho and Greg Burnhamepochai.substack.com
- After Automation | Everyevery.to
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- Challenges in evaluating AI systems \ Anthropicanthropic.com
- Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
- Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
- Giovanni D'Antoniogiovannidantonio.com