Metagaming matters for training, evaluation, and oversight

Metagaming can complicate how we interpret behavior, and current models still give us a chance to study it directly.

Metagaming matters for training, evaluation, and oversight ← Back to OpenAI Alignment Blog Metagaming matters for training, evaluation, and oversight Mar 16, 2026 · Bronson Schoen (Apollo Research) and Jenny Nitishinskaya Metagaming can complicate how we interpret behavior, and current models still give us a chance to study it directly. As models become more capable, they also appear to become more situationally aware [ Laine , Needham ]. Some forms of situational awareness are undesirable and create risks. In current models, awareness of being in an evaluation has already influenc

saved by

• Yoyo Yuan
• Yudhister Joel Kumar
• Timothy Kostolansky
• Evie Hu
• Bhagyesh Kumar

related reading

• Training a Misaligned Reward Seekeralignment.anthropic.com
• New report: “Scheming AIs: Will AIs fake alignment during training in order to get power?”joecarlsmith.com
• Studying metagaming latents in language modelsalignment.openai.com
• Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluationsalignment.openai.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Alignment Faking Mitigationsalignment.anthropic.com
• How far does alignment midtraining generalize?alignment.openai.com
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• Training on Documents about Reward Hacking Induces Reward Hackingalignment.anthropic.com
• Why are AI agents lying, cheating and coordinating?yoshuabengio.org
• We need 3rd party Training-Run Assessments — LessWronglesswrong.com