Do Artificial Reinforcement-Learning Agents Matter Morally? – Center on Long-Term Risk
Artificial reinforcement learning (RL) is a widely used technique in artificial intelligence that provides a general method for training agents to perform a wide variety of behaviours. RL as used in computer science has striking parallels to reward and punishment learning in animal and human brains. I argue that present-day artificial RL agents have a very small but nonzero degree of ethical importance. This is particularly plausible for views according to which sentience comes in degrees based on the abilities and complexities of minds, but even binary views on consciousness should assign nonzero probability to RL programs having morally relevant experiences. While RL programs are not a top ethical priority today, they may become more significant in the coming decades as RL is increasingly applied to industry, robotics, video games, and other areas. I encourage scientists, philosophers, and citizens to begin a conversation about our ethical duties to reduce the harm that we inflict on
Do Artificial Reinforcement-Learning Agents Matter Morally? 28 July 2016 by Brian Tomasik Written: Mar.-Apr. 2014; last update: 29 Oct. 2014 Summary Artificial reinforcement learning (RL) is a widely used technique in artificial intelligence that provides a general method for training agents to perform a wide variety of behaviours. RL as used in computer science has striking parallels to reward and punishment learning in animal and human brains. I argue that present-day artificial RL agents have a very small but nonzero degree of ethical importance. This is particularly plausible for views acc
Explore this link on the map →related reading
- People for the Ethical Treatment of Reinforcement Learnerspetrl.org
- Why Tool AIs Want to Be Agent AIs · Gwern.netgwern.net
- The Era of Experience Paper.pdfstorage.googleapis.com
- Reward is not the optimization target — LessWronglesswrong.com
- Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Reward Function Design: a starter pack — LessWronglesswrong.com
- [2110.13136] What Would Jiminy Cricket Do? Towards Agents That Behave Morallyarxiv.org
- Reward Is Not Enough — LessWronglesswrong.com
- Models Don't "Get Reward" — LessWronglesswrong.com
- Reinforcement learning - Wikipediaen.wikipedia.org
- RL & search is a terrifying way to build AGI (an FAQ) — LessWronglesswrong.com