The shard theory of human values — AI Alignment Forum

TL;DR: We propose a theory of human value formation. According to this theory, the reward system shapes human values in a relatively straightforward…

x The shard theory of human values — AI Alignment Forum Shard Theory Shard Theory Human Values Internal Alignment (Human) Heuristics & Biases AI World Modeling Rationality Curated 74 The shard theory of human values by Quintin Pope , TurnTrout 4th Sep 2022 28 min read 67 74 TL;DR: We propose a theory of human value formation. According to this theory, the reward system shapes human values in a relatively straightforward manner. Human values are not e.g. an incredibly complicated, genetically hard-coded set of drives, but rather sets of contextually activated heuristics which were shaped by and

related reading

• The Shard Theory of Human Valuesturntrout.com
• The shard theory of human values — LessWronglesswrong.com
• Shard Theory — AI Alignment Forumalignmentforum.org
• Shard Theory in Nine Theses: a Distillation and Critical Appraisal — AI Alignment Forumalignmentforum.org
• A Crash Course in the Neuroscience of Human Motivation — LessWronglesswrong.com
• Understanding and avoiding value drift — LessWronglesswrong.com
• A Shard Theory of Everything - by Jason Hausenloyfirstscattering.com
• The Computational Anatomy of Human Values — LessWronglesswrong.com
• Contra shard theory, in the context of the diamond maximizer problem — LessWronglesswrong.com
• The behavioral selection model for predicting AI motivations — LessWronglesswrong.com
• Reward is not the optimization target — LessWronglesswrong.com
• Have You Tried Thinking About It As Crystals? — LessWronglesswrong.com