Model evals for dangerous capabilities — LessWrong

Testing an LM system for dangerous capabilities is crucial for assessing its risks. …

x Model evals for dangerous capabilities — LessWrong AI Evaluations AI Frontpage 52 Model evals for dangerous capabilities by Zach Stein-Perlman 23rd Sep 2024 4 min read 11 52 Testing an LM system for dangerous capabilities is crucial for assessing its risks. Summary of best practices Best practices for labs evaluating LM systems for dangerous capabilities: Publish results Publish questions/tasks/methodology (unless that's dangerous, e.g. CBRN evals; if so, offer to share more information with other labs, government, and relevant auditors, and publish a small subset) Do good elicitation and pu

related reading

• Your Evals Will Break and You Won't See It Coming - Lun Wangwanglun1996.github.io
• Claude Mythos Preview System Cardwww-cdn.anthropic.com
• Demystifying evals for AI agents \ Anthropicanthropic.com
• Chapter 3: LLM Evaluations - ARENAlearn.arena.education
• gpt-4.pdfcdn.openai.com
• [2403.13793] Evaluating Frontier Models for Dangerous Capabilitiesarxiv.org
• A starter guide for evals — AI Alignment Forumalignmentforum.org
• Challenges in evaluating AI systems \ Anthropicanthropic.com
• Noam Brown on X: "Implications of Large-Scale Test-Time Compute" / Xx.com
• The Case for Evaluating Model Behaviors — AI Alignment Forumalignmentforum.org
• [2408.02565] Reasons to Doubt the Impact of AI Risk Evaluationsarxiv.org
• A starter guide for evals — LessWronglesswrong.com