(Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWrong

Authors: Satvik Golechha*, Sid Black*, Joseph Bloom • * Equal Contribution. …

x (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWrong AI Frontpage 2026 Top Fifty: 14 % 127 (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL by 7vik , Sid Black , Joseph Bloom 30th Mar 2026 AI Alignment Forum 21 min read 6 127 Ω 47 Authors: Satvik Golechha*, Sid Black*, Joseph Bloom * Equal Contribution. This work was done as part of the Model Transparency team at the UK AI Security Institute (AISI). Our code is available on GitHub and the model checkpoints and data is available on HuggingFace . Executive Summary In Natural Eme

saved by

• Abhay Sheshadri
• Arya Pasumarthi

related reading

• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
• How hard is it to inoculate against misalignment generalization? — LessWronglesswrong.com
• Alignment Faking Mitigationsalignment.anthropic.com
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Fail safe(r) at alignment by channeling reward-hacking into a "spillway" motivationblog.redwoodresearch.org
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• [2602.15515] The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probesarxiv.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• Training on Documents about Reward Hacking Induces Reward Hackingalignment.anthropic.com
• Simulated Users & Sad LLMs1a3orn.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Natural emergent misalignment from reward hacking \ Anthropicanthropic.com