[2304.03279] Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2304.03279] Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2304.03279 (cs) [Submitted on 6 Apr 2023 ( v1 ), last revised 13 Jun 2023 (this version, v4)] Title: Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark Authors: Alexander Pan , Jun Shern Chan , Andy Zou , Nathan

related reading

• MACHIAVELLIaypan17.github.io
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• The behavioral selection model for predicting AI motivations — LessWronglesswrong.com
• Measuring Reward-Seeking by Instilling Contrastive Beliefsalignment.openai.com
• Natural-emergent-misalignment-from-reward-hacking-paper.pdfassets.anthropic.com
• Training a Misaligned Reward Seekeralignment.anthropic.com
• [2607.18966] Measuring Reward-Seeking via Contrastive Belief Updatesarxiv.org
• Why are AI agents lying, cheating and coordinating?yoshuabengio.org
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• Recent Frontier Models Are Reward Hacking - METRmetr.org
• (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RL — LessWronglesswrong.com
• Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com