✳flâneur — a map of the web's best reading

Shard Theory - AI Alignment Forum

alignmentforum.org · 457 words · saved by 1 readers

Shard theory is an alignment research program, about the relationship between training variables and learned values in trained Reinforcement Learning (RL) agents. It is thus an approach to progressively fleshing out a mechanistic account of human values, learned values in RL agents, and (to a lesser extent) the learned algorithms in ML generally. Shard theory's basic ontology of RL holds that shards are contextually activated, behavior-steering computations in neural networks (biological and artificial). The circuits that implement a shard that garners reinforcement are reinforced, meaning that that shard will be more likely to trigger again in the future, when given similar cognitive inputs. As an appreciable fraction of a neural network is composed of shards, large neural nets can possess quite intelligent constituent shards. These shards can be sophisticated enough to be well-modeled as playing negotiation games with each other, (potentially) explaining human psychological phenomena like akrasia and value changes from moral reflection. Shard theory also suggests an approach to explaining the shape of human values, and scheme for RL alignment.

x Shard Theory — AI Alignment Forum Shard Theory Edited by David Udell , ihatenumbersinusernames7 , et al. last updated 25th Jan 2026 Shard Theory is an alignment research program, about the relationship between training variables and learned values in trained Reinforcement Learning (RL) agents. It is thus an approach to progressively fleshing out a mechanistic account of human values , learned values in RL agents, and (to a lesser extent) the learned algorithms in ML generally. Shard theory's basic ontology of RL holds that shards are contextually activated, behavior-steering computations in

Explore this link on the map →

related reading