Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations

A pipeline to uncover unknown misaligned behavior and scale the creation of realistic evaluations.

← Back to OpenAI Alignment Blog Dec 18, 2025 · Marcus Williams, Cameron Raymond and Micah Carroll, in collaboration with the Safety Oversight team Evaluations of undesirable model behaviors are a critical tool for grounding current safety arguments. But evaluations best support claims about model safety when they cover the relevant risks and faithfully reflect real-world conditions. Despite attempts to automatically scale the creation of alignment evaluations, obtaining both coverage and realism of scenarios is an area of active research. In this post, we showcase a simple, scalable, and…

saved by

• Evie Hu

related reading

• Metagaming matters for training, evaluation, and oversightalignment.openai.com
• We need 3rd party Training-Run Assessments — LessWronglesswrong.com
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• An alignment assessment of recent cybersecurity incidentsanthropic.com
• Can public chat data predict real-world AI misalignments?alignment.openai.com
• How far does alignment midtraining generalize?alignment.openai.com
• Predicting LLM Safety Before Release by Simulating Deploymentcdn.openai.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com