Thoughts on the conservative assumptions in AI control

Why are we so friendly to the red team?

Thoughts on the conservative assumptions in AI control Why are we so friendly to the red team? Buck Shlegeris Jan 17, 2025 5 Share Work that I’ve done on techniques for mitigating risk from misaligned AI often makes a number of conservative assumptions about the capabilities of the AIs we’re trying to control. (E.g. the original AI control paper , Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats , How to prevent collusion when using untrusted models to monitor each other .) For example: The AIs are consistently trying to subvert safety measures. They’re very good at strategizi

saved by

• Jo J.
• Arjun Khandelwal
• Riya Tyagi

related reading

• The Case Against AI Control Research — LessWronglesswrong.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Thoughts on the conservative assumptions in AI controlredwoodresearch.substack.com
• Catching AIs red-handedblog.redwoodresearch.org
• Win/continue/lose scenarios and execute/replace/audit protocolsblog.redwoodresearch.org
• Notes on handling non-concentrated failures with AI control: high level methods and different regimesblog.redwoodresearch.org
• How do we (more) safely defer to AIs? — LessWronglesswrong.com
• Why it's hard to make settings for high-stakes control researchblog.redwoodresearch.org
• AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
• The case for ensuring that powerful AIs are controlledblog.redwoodresearch.org
• AI Control: Improving Safety Despite Intentional Subversion — AI Alignment Forumalignmentforum.org
• Reading Listblog.redwoodresearch.org