Inference-Time Reward Hacking in Large Language Models
arxiv.org · 7,539 words · saved by 1 readers
N/A
Inference-Time Reward Hacking in Large Language Models Hadi Khalaf∗ Claudio Mayrink Verdun Alex Oesterling Harvard University Harvard University Harvard University Himabindu Lakkaraju Flavio du Pin Calmon∗ arXiv:2506.19248v2 [cs.LG] 4 Nov 2025…
related reading
- Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
- Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
- [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
- Training a Misaligned Reward Seekeralignment.anthropic.com
- Training on Documents about Reward Hacking Induces Reward Hackingalignment.anthropic.com
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationarxiv.org
- Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com
- Reward hacking behavior can generalize across tasks — AI Alignment Forumalignmentforum.org
- (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWronglesswrong.com
- [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
- [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
- [2607.18966] Measuring Reward-Seeking via Contrastive Belief Updatesarxiv.org