Model evals for dangerous capabilities — LessWrong

Testing an LM system for dangerous capabilities is crucial for assessing its risks. …

x Model evals for dangerous capabilities — LessWrong AI Evaluations AI Frontpage 52 Model evals for dangerous capabilities by Zach Stein-Perlman 23rd Sep 2024 4 min read 11 52 Testing an LM system for dangerous capabilities is crucial for assessing its risks. Summary of best practices Best practices for labs evaluating LM systems for dangerous capabilities: Publish results Publish questions/tasks/methodology (unless that's dangerous, e.g. CBRN evals; if so, offer to share more information with other labs, government, and relevant auditors, and publish a small subset) Do good elicitation and pu

related reading

• Demystifying evals for AI agents \ Anthropicanthropic.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Self-exfiltration is a key dangerous capabilityaligned.substack.com
• Your Evals Will Break and You Won't See It Coming - Lun Wangwanglun1996.github.io
• AI models can be dangerous before public deploymentmetr.org
• A structured protocol for elicitation experiments | AISI Workaisi.gov.uk
• We need 3rd party Training-Run Assessments — LessWronglesswrong.com
• [2408.02565] Reasons to Doubt the Impact of AI Risk Evaluationsarxiv.org
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
• AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
• Chapter 3: LLM Evaluations - ARENAlearn.arena.education
• How do we (more) safely defer to AIs? — LessWronglesswrong.com