What is the alignment problem? - by Jan Leike

My attempt at clarifying a confusing topic

What is the alignment problem? My attempt at clarifying a confusing topic Jan Leike Mar 29, 2022 51 5 1 Share On a very high level, building performant AI systems requires two ingredients: Capability: The AI system could do the intended task. Alignment: The AI system does the intended task as well as it could. Thus if the system doesn’t do the intended task, then this is always due to a capability problem, an alignment problem, or both. Usually we talk about alignment with human intentions. In this case the intended task is whatever the human wanted the system to do. Examples We’re all familia

related reading

• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Defining alignment research — LessWronglesswrong.com
• “Alignment Engineering” vs. “Misalignment Science”lesswrong.com
• A minimal viable product for alignment - by Jan Leikealigned.substack.com
• Alignment is not solved but it increasingly looks solvablealigned.substack.com
• What is AI alignment? - by Adam Jones - BlueDot Impactblog.bluedot.org
• The self-unalignment problem — LessWronglesswrong.com
• What Is The Alignment Problem? — LessWronglesswrong.com
• Why AI alignment could be hard with modern deep learningcold-takes.com
• The Artificiality of Alignmentjoinreboot.org
• Readings on the nature of alignment researchcasparoesterheld.com