How far does alignment midtraining generalize?

Preliminary experiments on alignment and misalignment midtraining, reasoning posttraining, and generalization to chat and agentic evals.

How far does alignment midtraining generalize? ← Back to OpenAI Alignment Blog How far does alignment midtraining generalize? Mar 27, 2026 · Tomek Korbak, Cameron Raymond, Micah Carroll, Marcus Williams, Mikita Balesni, Alan Guo, Jason Wolfe, Akshay Jagadeesh, Ian Kivlichan Today’s AI agents are based on large language models (LLMs) that draw their knowledge and expectations from vast amounts of internet text. This means that AI agents are exposed to AI safety discourse, including discussions of AI misalignment risk and depictions of humans losing control over malicious AI agents, which could

saved by

• Justin Wang
• Julian H
• Jackson Kozlowski

related reading

• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Reinforcement learning towards broadly and persistently beneficial modelsalignment.openai.com
• Teaching Claude Whyalignment.anthropic.com
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentalignmentpretraining.ai
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
• Alignment Faking Mitigationsalignment.anthropic.com
• Alignment is not solved but it increasingly looks solvablealigned.substack.com
• Teaching Claude why \ Anthropicanthropic.com
• Model Spec Midtraining: Improving How Alignment Training Generalizesalignment.anthropic.com
• Metagaming matters for training, evaluation, and oversightalignment.openai.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org