Realistic Evaluations Will Not Prevent Evaluation Awareness — LessWrong
One Minute Summary I think there's a fundamental limit to behavioral alignment evaluations that gets worse as models improve: Humans control all inpu…
x Realistic Evaluations Will Not Prevent Evaluation Awareness — LessWrong AI Frontpage 38 Realistic Evaluations Will Not Prevent Evaluation Awareness by Adam Karvonen 24th Feb 2026 7 min read 9 38 One Minute Summary I think there's a fundamental limit to behavioral alignment evaluations that gets worse as models improve: Humans control all inputs to a model and can snapshot, replay, or fabricate any scenario at will. An intelligent model could realize this and rationally treat every interaction as a potential evaluation. This means that even perfectly realistic evaluations, including ones samp
related reading
- The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
- The behavioral selection model for predicting AI motivations — LessWronglesswrong.com
- Metagaming matters for training, evaluation, and oversightalignment.openai.com
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluationsalignment.openai.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- Models May Behave Worse When Eval Aware — LessWronglesswrong.com
- The Case Against AI Control Research — LessWronglesswrong.com
- How do we (more) safely defer to AIs? — LessWronglesswrong.com
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- How far does alignment midtraining generalize?alignment.openai.com
- AI Control: Improving Safety Despite Intentional Subversion — AI Alignment Forumalignmentforum.org