✳flâneur — a map of the web's best reading

Shard Theory in Nine Theses: a Distillation and Critical Appraisal — AI Alignment Forum

alignmentforum.org · 11,211 words · saved by 1 readers

TL;DR: Shard theory is a new research program started by Quintin Pope and Alex Turner. Existing introductions tend to be relatively long winded and aimed at an introductory audience. Here, I outline what I think are the nine main theses of shard theory as of Dec 2022, so as to give a more concrete introduction and critique: While I broadly sympathize the intuitions behind shard theory, I raise reservations with each of the theses at the end of the post. I conclude by suggesting areas of future work in shard theory. Acknowledgements: Thanks to Thomas Kwa for the conversation that inspired this post. Thanks also to Alex Turner for several conversations about shard theory and Charles Foster and Teun Van Der Weij for substantial feedback on this writeup. Epistemic status: As I’m not a member of Team Shard, I’m probably misrepresenting Team Shard’s beliefs in a few places here. Shard theory is a research program that aims to build a mechanistic model between training signals and learned val

x Shard Theory in Nine Theses: a Distillation and Critical Appraisal — AI Alignment Forum Shard Theory Distillation & Pedagogy AI Frontpage 72 Shard Theory in Nine Theses: a Distillation and Critical Appraisal by LawrenceC 19th Dec 2022 21 min read 30 72 TL;DR: Shard theory is a new research program started by Quintin Pope and Alex Turner. Existing introductions tend to be relatively long winded and aimed at an introductory audience. Here, I outline what I think are the nine main theses of shard theory as of Dec 2022, so as to give a more concrete introduction and critique: Agents are well mod

Explore this link on the map →

related reading