Natural-emergent-misalignment-from-reward-hacking-paper.pdf

N/A

# link_13788wjnwsp.pdf ## Metadata - PDFFormatVersion=1.5 - IsLinearized=true - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - CreationDate=D:20251121190241Z - Creator=LaTeX with hyperref - ModDate=D:20251121190241Z - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.141592653-2.6-1.40.27 (TeX Live 2025) kpathsea version 6.4.1 - Producer=pdfTeX-1.40.27 - Trapped=False ## Contents ### Page 1 NATURAL EMERGENT MISALIGNMENT FROM REWARD HACKING IN PRODUCTION RLMonte MacDiarmid∗, Benjamin Wright∗, Jonathan Uesato∗, Joe Benton, Jon Kutasov, Sa

saved by

• Kushal Thaman
• Eric Huang
• Jason Hausenloy
• Christine Ye
• Will Anderson

related reading

• (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWronglesswrong.com
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Alignment Faking Mitigationsalignment.anthropic.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Training on Documents about Reward Hacking Induces Reward Hackingalignment.anthropic.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Natural emergent misalignment from reward hacking \ Anthropicanthropic.com
• How hard is it to inoculate against misalignment generalization? — LessWronglesswrong.com
• Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com
• Reward hacking behavior can generalize across tasks — AI Alignment Forumalignmentforum.org
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com