Outer vs inner misalignment: three framings - AI Alignment Forum
A core concept in the field of AI alignment is a distinction between two types of misalignment: outer misalignment and inner misalignment. Roughly speaking, the outer alignment problem is the problem…
x Outer vs inner misalignment: three framings — AI Alignment Forum Inner Alignment Outer Alignment AI Frontpage 28 Outer vs inner misalignment: three framings by Richard_Ngo 6th Jul 2022 11 min read 5 28 A core concept in the field of AI alignment is a distinction between two types of misalignment: outer misalignment and inner misalignment. Roughly speaking, the outer alignment problem is the problem of specifying an reward function which captures human preferences; and the inner alignment problem is the problem of ensuring that a policy trained on that reward function actually tries to act in
related reading
- Outer vs inner misalignment: three framings — LessWronglesswrong.com
- Categorizing failures as “outer” or “inner” misalignment is often confused — AI Alignment Forumalignmentforum.org
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- “Alignment Engineering” vs. “Misalignment Science”lesswrong.com
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- Defining alignment research — LessWronglesswrong.com
- Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
- What is AI alignment? - by Adam Jones - BlueDot Impactblog.bluedot.org
- Self-Other Overlap: A Neglected Approach to AI Alignment — LessWronglesswrong.com
- Reward is not the optimization target — LessWronglesswrong.com
- Inner Alignment: Explain like I'm 12 Edition — LessWronglesswrong.com
- The self-unalignment problem — LessWronglesswrong.com