flâneur

Natural Abstractions: Key claims, Theorems, and Critiques — LessWrong

lesswrong.com · 7,238 words · saved by 1 readers

TL;DR: We distill John Wentworth’s Natural Abstractions agenda by summarizing its key claims: the Natural Abstraction Hypothesis—many cognitive systems learn to use similar abstractions—and the Redundant Information Hypothesis—a particular mathematical description of natural abstractions. We also formalize proofs for several of its theoretical results. Finally, we critique the agenda’s progress to date, alignment relevance, and current research methodology. Author Contributions: Erik wrote a majority of the post and developed the breakdown into key claims. Leon formally proved the gKPD theorem and wrote most of the mathematical formalization section and appendix. Lawrence formally proved the Telephone theorem and wrote most of the related work section. All of us were involved in conceptual discussions and various small tasks. Epistemic Status: We’re not John Wentworth, though we did confirm our understanding with him in person and shared a draft of this post with him beforehand. Append

x Natural Abstractions: Key Claims, Theorems, and Critiques — LessWrong Best of LessWrong 2023 Natural Abstraction Distillation & Pedagogy MATS Program AI Frontpage 250 Natural Abstractions: Key Claims, Theorems, and Critiques by LawrenceC , Leon Lang , Erik Jenner 16th Mar 2023 AI Alignment Forum 54 min read 26 250 Ω 101 TL;DR: We distill John Wentworth’s Natural Abstractions agenda by summarizing its key claims: the Natural Abstraction Hypothesis—many cognitive systems learn to use similar abstractions—and the Redundant Information Hypothesis—a particular mathematical description of natural

related reading