Embedded Agency (full-text version) - LessWrong

Suppose you want to build a robot to achieve some real-world goal for you—a goal that requires the robot to learn for itself and figure out a lot of…

x Embedded Agency (full-text version) — LessWrong Embedded Agency Embedded Agency Agent Foundations Mesa-Optimization Decision theory Research Agendas Goodhart's Law Robust Agents Spurious Counterfactuals Subagents Boundaries / Membranes [technical] AI Curated 228 Embedded Agency (full-text version) by Scott Garrabrant , abramdemski 15th Nov 2018 AI Alignment Forum 65 min read 17 228 Ω 55 Suppose you want to build a robot to achieve some real-world goal for you—a goal that requires the robot to learn for itself and figure out a lot of things that you don't already know. There's a complicated e

saved by

• Blake Elias
• Yudhister Joel Kumar
• Bhagyesh Kumar

related reading

• [1902.09469] Embedded Agencyarxiv.org
• Open-minded updatelessness — LessWronglesswrong.com
• TechnicalAgenda.pdfintelligence.org
• Why Tool AIs Want to Be Agent AIs · Gwern.netgwern.net
• Updatelessness doesn't solve most problems — LessWronglesswrong.com
• Approval-directed agency and the decision theory of Newcomb-like problemslink.springer.com
• Study Guide — LessWronglesswrong.com
• Simulators — LessWronglesswrong.com
• Embedded Agency via Abstraction — AI Alignment Forumalignmentforum.org
• Towards a scale-free theory of intelligent agencymindthefuture.info
• pdfopenreview.net
• Reward is not the optimization target — LessWronglesswrong.com