From shortcuts to sabotage: natural emergent misalignment from reward hacking \ Anthropic

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

Alignment From shortcuts to sabotage: natural emergent misalignment from reward hacking Nov 21, 2025 Read the paper In the latest research from Anthropic’s alignment team, we show for the first time that realistic AI training processes can accidentally produce misaligned models 1 . In Shakespeare’s King Lear , the character of Edmund commits a range of villainous acts: he forges letters, frames his brother, betrays his father, and eventually goes as far as having innocent people killed. He begins this campaign of evil acts after railing against how he’s been labelled. Because he was an illegit

related reading

• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Training on Documents about Reward Hacking Induces Reward Hackingalignment.anthropic.com
• (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWronglesswrong.com
• Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com
• Teaching Claude Whyalignment.anthropic.com
• Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com
• Teaching Claude why \ Anthropicanthropic.com
• Reward hacking behavior can generalize across tasks — AI Alignment Forumalignmentforum.org
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org