Outer vs inner misalignment: three framings — LessWrong

A core concept in the field of AI alignment is a distinction between two types of misalignment: outer misalignment and inner misalignment. Roughly sp…

x Outer vs inner misalignment: three framings — LessWrong Inner Alignment Outer Alignment AI Frontpage 53 Outer vs inner misalignment: three framings by Richard_Ngo 6th Jul 2022 AI Alignment Forum 11 min read 5 53 Ω 28 A core concept in the field of AI alignment is a distinction between two types of misalignment: outer misalignment and inner misalignment. Roughly speaking, the outer alignment problem is the problem of specifying an reward function which captures human preferences; and the inner alignment problem is the problem of ensuring that a policy trained on that reward function actually

related reading

• Outer vs inner misalignment: three framings — AI Alignment Forumalignmentforum.org
• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Categorizing failures as “outer” or “inner” misalignment is often confused — AI Alignment Forumalignmentforum.org
• “Alignment Engineering” vs. “Misalignment Science”lesswrong.com
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Defining alignment research — LessWronglesswrong.com
• Self-Other Overlap: A Neglected Approach to AI Alignment — LessWronglesswrong.com
• Reward is not the optimization target — LessWronglesswrong.com
• Inner Alignment: Explain like I'm 12 Edition — LessWronglesswrong.com
• What is AI alignment? - by Adam Jones - BlueDot Impactblog.bluedot.org
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• The self-unalignment problem — LessWronglesswrong.com