[2312.06942] AI Control: Improving Safety Despite Intentional Subversion

As large language models (LLMs) become more powerful and are deployed more autonomously, it will be increasingly important to prevent them from causing harmful outcomes. Researchers have investigated a variety of safet…

AI Control: Improving Safety Despite Intentional Subversion Ryan Greenblatt ∗ Buck Shlegeris Kshitij Sachan Fabien Roger Redwood Research Abstract As large language models (LLMs) become more powerful and are deployed more autonomously, it will be increasingly important to prevent them from causing harmful outcomes. Researchers have investigated a variety of safety techniques for this purpose, e.g. using models to review the outputs of other models, or red-teaming techniques to surface subtle failure modes. However, researchers have not evaluated whether such techniques still ensure safety if t

related reading

• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• 2312.06942arxiv.org
• AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
• AI Control: Improving Safety Despite Intentional Subversion — AI Alignment Forumalignmentforum.org
• AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• The case for ensuring that powerful AIs are controlledblog.redwoodresearch.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Using Dangerous AI, But Safely?youtube.com
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2502.05209] Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiesarxiv.org