Why it's hard to make settings for high-stakes control research

It's like making challenging evals, but more constrained

Why it's hard to make settings for high-stakes control research It's like making challenging evals, but more constrained Buck Shlegeris Jul 18, 2025 10 5 Share One of our main activities at Redwood is writing follow-ups to previous papers on control like the original and Ctrl-Z , where we construct a setting with a bunch of tasks (e.g. APPS problems) and a notion of safety failure (e.g. backdoors according to our specific definition), then play the adversarial game where we develop protocols and attacks on those protocols. It turns out that a substantial fraction of the difficulty here is deve

saved by

• Jo J.

related reading

• Why it's hard to make settings for high-stakes control researchredwoodresearch.substack.com
• Notes on handling non-concentrated failures with AI control: high level methods and different regimesblog.redwoodresearch.org
• Thoughts on the conservative assumptions in AI controlblog.redwoodresearch.org
• Catching AIs red-handedblog.redwoodresearch.org
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• The Case Against AI Control Research — LessWronglesswrong.com
• How can we solve diffuse threats like research sabotage with AI control?blog.redwoodresearch.org
• The case for ensuring that powerful AIs are controlledblog.redwoodresearch.org
• Reading Listblog.redwoodresearch.org
• AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
• Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
• An overview of areas of control workblog.redwoodresearch.org