[2304.03279] Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2304.03279] Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2304.03279 (cs) [Submitted on 6 Apr 2023 ( v1 ), last revised 13 Jun 2023 (this version, v4)] Title: Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark Authors: Alexander Pan , Jun Shern Chan , Andy Zou , Nathan

related reading

• MACHIAVELLIaypan17.github.io
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
• The behavioral selection model for predicting AI motivations — LessWronglesswrong.com
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• Reward hacking behavior can generalize across tasks — AI Alignment Forumalignmentforum.org
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• [2602.12316] GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theoryarxiv.org
• Models Don't "Get Reward" — LessWronglesswrong.com
• Center for Responsible, Decentralized Intelligence at Berkeleyrdi.berkeley.edu
• Recent Frontier Models Are Reward Hacking - METRmetr.org
• (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWronglesswrong.com