Alignment remains a hard, unsolved problem — AI Alignment Forum

This is a public adaptation of a document I wrote for an internal Anthropic audience about a month ago. Thanks to (in alphabetical order) Joshua Batson, Joe Benton, Sam Bowman, Roger Grosse, Jeremy Hadfield, Jared Kaplan, Jan Leike, Jack Lindsey, Monte MacDiarmid, Sam Marks, Fra Mosconi, Chris Olah, Ethan Perez, Sara Price, Ansh Radhakrishnan, Fabien Roger, Buck Shlegeris, Drake Thomas, and Kate Woolverton for useful discussions, comments, and feedback. Though there are certainly some issues, I think most current large language models are pretty well aligned. Despite its alignment faking, my favorite is probably Claude 3 Opus, and if you asked me to pick between the CEV of Claude 3 Opus and that of a median human, I think it'd be a pretty close call (I'd probably pick Claude, but it depends on the details of the setup). So, overall, I'm quite positive on the alignment of current models! And yet, I remain very worried about alignment in the future. This is my attempt to explain why that

x Alignment remains a hard, unsolved problem — AI Alignment Forum AI Curated 2025 Top Fifty: 25 % 128 Alignment remains a hard, unsolved problem by evhub 27th Nov 2025 16 min read 98 128 This is a public adaptation of a document I wrote for an internal Anthropic audience about a month ago. Thanks to (in alphabetical order) Joshua Batson, Joe Benton, Sam Bowman, Roger Grosse, Jeremy Hadfield, Jared Kaplan, Jan Leike, Jack Lindsey, Monte MacDiarmid, Sam Marks, Fra Mosconi, Chris Olah, Ethan Perez, Sara Price, Ansh Radhakrishnan, Fabien Roger, Buck Shlegeris, Drake Thomas, and Kate Woolverton for

related reading

• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Alignment is not solved but it increasingly looks solvablealigned.substack.com
• Did Claude 3 Opus align itself via gradient hacking? — LessWronglesswrong.com
• The Artificiality of Alignmentjoinreboot.org
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Alignment Is Proven To Be Solvable - by SE Gygesverysane.ai
• Thoughts on the Alignment Implications of Scaling Language Models | Leo Gaobmk.sh
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
• Another (outer) alignment failure story — AI Alignment Forumalignmentforum.org
• Automated Alignment Researchers: Using large language models to scale scalable oversight \ Anthropicanthropic.com