The Stamp Collector
Once upon a time, a group of naïve philosophers found a robot that collected trinkets. Well, more specifically, the robot seemed to collect stamps: if you presented this robot with a choice between various trinkets, it would always choose the option that led towards it having as many stamps as
Once upon a time, a group of naïve philosophers found a robot that collected trinkets. Well, more specifically, the robot seemed to collect stamps: if you presented this robot with a choice between various trinkets, it would always choose the option that led towards it having as many stamps as possible in its inventory. It ignored dice, bottle caps, aluminum cans, sticks, twigs, and so on, except insofar as it predicted they could be traded for stamps in the next turn or two. So, of course, the philosophers started calling it the "stamp collector." Then, one day, the philosophers discovered co
Explore this link on the map →saved by
related reading
- The Stamp Collector — LessWronglesswrong.com
- Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
- A Crash Course in the Neuroscience of Human Motivation — LessWronglesswrong.com
- [1902.09469] Embedded Agencyarxiv.org
- Embedded Agency (full-text version) — LessWronglesswrong.com
- alan turing - computing machinery and intelligencecourses.cs.umbc.edu
- Beliefs are Chosen to Serve Goals — LessWronglesswrong.com
- Models Don't "Get Reward" — LessWronglesswrong.com
- Irrationality as a Defense Mechanism for Reward-hacking — LessWronglesswrong.com
- The blue-minimising robot and model splintering — AI Alignment Forumalignmentforum.org
- The monkey and the machine: a dual process theory – The sideways viewsideways-view.com
- WWW - Evaluative Soliloquies - Ken Liuwordcraft-writers-workshop.appspot.com