[2403.13793] Evaluating Frontier Models for Dangerous Capabilities

Abstract:To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.

2024-4-8 Evaluating Frontier Models for Dangerous Capabilities Mary Phuong* , Matthew Aitchison* , Elliot Catt* , Sarah Cogan* , Alexandre Kaskasoli* , Victoria Krakovna* , David Lindner* , Matthew Rahtz* , Yannis Assael, Sarah Hodkinson, Heidi Howard, Tom Lieberum, Ramana Kumar, Maria Abi Raad, Albert Webson, Lewis Ho, Sharon Lin, Sebastian Farquhar, Marcus Hutter, Grégoire…

related reading

• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• [2603.02202] Frontier Models Can Take Actions at Low Probabilitiesarxiv.org
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
• The End-State Fallacy: Where Is AI Security Headed?endstatefallacy.com
• [2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyarxiv.org
• [2502.05209] Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiesarxiv.org
• Cognitive Security as an AI Safety Cause Area — LessWronglesswrong.com
• 2312.06942arxiv.org
• [2512.11949] Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorsarxiv.org
• [2408.02565] Reasons to Doubt the Impact of AI Risk Evaluationsarxiv.org
• [2603.11214] Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenariosarxiv.org