(Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWrong

Authors: Satvik Golechha*, Sid Black*, Joseph Bloom • * Equal Contribution. …

x (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWrong AI Frontpage 2026 Top Fifty: 14 % 127 (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL by 7vik , Sid Black , Joseph Bloom 30th Mar 2026 AI Alignment Forum 21 min read 6 127 Ω 47 Authors: Satvik Golechha*, Sid Black*, Joseph Bloom * Equal Contribution. This work was done as part of the Model Transparency team at the UK AI Security Institute (AISI). Our code is available on GitHub and the model checkpoints and data is available on HuggingFace . Executive Summary In Natural Eme

saved by

• Abhay Sheshadri
• Arya Pasumarthi

related reading

• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Alignment Faking Mitigationsalignment.anthropic.com
• How hard is it to inoculate against misalignment generalization? — LessWronglesswrong.com
• Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• Natural emergent misalignment from reward hacking \ Anthropicanthropic.com
• Simulated Users & Sad LLMs1a3orn.com
• [2607.18966] Measuring Reward-Seeking via Contrastive Belief Updatesarxiv.org
• Recent Frontier Models Are Reward Hacking - METRmetr.org
• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com