Partial Agency — LessWrong

I think there's something interesting going on with Evan's notion of myopia. …

x Partial Agency — LessWrong Partial Agency Myopia Frontpage 77 Partial Agency by abramdemski 27th Sep 2019 AI Alignment Forum 10 min read 18 77 Ω 34 Epistemic status: very rough intuitions here. I think there's something interesting going on with Evan's notion of myopia . Evan has been calling this thing "myopia". Scott has been calling it "stop-gradients". In my own mind, I've been calling the phenomenon "directionality". Each of these words gives a different set of intuitions about how the cluster could eventually be formalized. Stop-Gradients Nash equilibria are, abstractly, modeling agent

related reading

• Embedded Agency (full-text version) — LessWronglesswrong.com
• Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
• Reward is not the optimization target — LessWronglesswrong.com
• Towards a scale-free theory of intelligent agencymindthefuture.info
• Towards a scale-free theory of intelligent agency — AI Alignment Forumalignmentforum.org
• Evolution as Backstop for Reinforcement Learning · Gwern.netgwern.net
• Andrej Karpathy — AGI is still a decade awaydwarkesh.com
• A positive case for how we might succeed at prosaic AI alignment — AI Alignment Forumalignmentforum.org
• Arguments against myopic training — LessWronglesswrong.com
• Risks from Learned Optimization: Introduction — LessWronglesswrong.com
• [1906.01820] Risks from Learned Optimization in Advanced Machine Learning Systemsarxiv.org
• pdfopenreview.net