✳flâneur — a map of the web's best reading

Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols? — AI Alignment Forum

alignmentforum.org · 2,864 words · saved by 1 readers

We recently released Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?, a major update to our previous paper/blogpost, evaluating a broader range of models (e.g. helpful-only Claude 3.5 Sonnet) in more diverse and realistic settings (e.g. untrusted monitoring). An AI control protocol is a plan for usefully deploying AI systems that prevents an AI from intentionally causing some unacceptable outcome. Previous work evaluated protocols by subverting them using an AI following a human-written strategy. This paper investigates how well AI systems can generate and act on their own strategies for subverting control protocols whilst operating statelessly (i.e., without shared memory between contexts). To do this, an AI system may need to reliably generate effective strategies in each context, take actions with well-calibrated probabilities, and coordinate plans with other instances of itself without communicating. We develop Subversion Strategy

x Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols? — AI Alignment Forum AI Evaluations AI Frontpage 24 Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols? by Alex Mallen , Charlie Griffin , Buck 24th Mar 2025 10 min read 0 24 We recently released Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols? , a major update to our previous paper / blogpost , evaluating a broader range of models (e.g. helpful-only Claude 3.5 Sonnet) in more diverse and realistic

Explore this link on the map →

related reading