why assume AGIs will optimize for fixed goals? — LessWrong
lesswrong.com · 13,014 words · saved by 1 readers
When I read posts about AI alignment on LW / AF/ Arbital, I almost always find a particular bundle of assumptions taken for granted: …
x why assume AGIs will optimize for fixed goals? — LessWrong Goal-Directedness AI Frontpage 161 [ Question ] why assume AGIs will optimize for fixed goals? by nostalgebraist 10th Jun 2022 AI Alignment Forum 5 min read A 8 61 161 Ω 45 When I read posts about AI alignment on LW / AF/ Arbital, I almost always find a particular bundle of assumptions taken for granted: An AGI has a single terminal goal [1] . The goal is a fixed part of the AI's structure. The internal dynamics of the AI, if left to their own devices, will never modify the goal. The "outermost loop" of the AI's internal dynamics is
related reading
- Beliefs are Chosen to Serve Goals — LessWronglesswrong.com
- AGI Ruin: A List of Lethalities — LessWronglesswrong.com
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
- AI Goals Forecast — AI 2027ai-2027.com
- A positive case for how we might succeed at prosaic AI alignment — AI Alignment Forumalignmentforum.org
- Commentary on AGI Safety from First Principles — AI Alignment Forumalignmentforum.org
- wrapper-minds are the enemy — LessWronglesswrong.com
- Risks from Learned Optimization: Introduction — LessWronglesswrong.com
- What Is The Alignment Problem? — LessWronglesswrong.com
- After Orthogonality: Virtue-Ethical Agency and AI Alignmentthegradient.pub
- What is AI alignment? - by Adam Jones - BlueDot Impactblog.bluedot.org