flâneur

Research update: Towards a Law of Iterated Expectations for Heuristic Estimators — Alignment Research Center

alignment.org · 7,538 words · saved by 1 readers

Last week, ARC released a paper called Towards a Law of Iterated Expectations for Heuristic Estimators, which follows up on previous work on formalizing the presumption of independence. Most of the work described here was done in 2023. A brief table of contents for this post: In "Formalizing the Presumption of Independence", we defined a heuristic estimator to be a hypothetical algorithm that estimates the values of mathematical expression based on arguments. That is, a heuristic estimator is an algorithm 𝐺 G that takes as input -- and outputs an estimate of the value of 𝑌 Y that incorporates the information provided by 𝜋 1 , … , 𝜋 𝑚 π ​1 ​​,…,π ​m ​​. We denote this estimate by 𝐺 ( 𝑌 ∣ 𝜋 1 , … , 𝜋 𝑚 ) G(Y∣π ​1 ​​,…,π ​m ​​). 1 In that paper, we introduced the following question: is there a computationally efficient heuristic estimator that formalizes intuitively valid reasoning about the values of mathematical quantities based on arguments? We studied the question by int

Last week, ARC released a paper called Towards a Law of Iterated Expectations for Heuristic Estimators , which follows up on previous work on formalizing the presumption of independence . Most of the work described here was done in 2023. A brief table of contents for this post: What is a heuristic estimator? (One example and three analogies.) How might heuristic estimators help with understanding neural networks? (Three potential applications.) Formalizing the principle of unpredictable errors for heuristic estimation (the technical meat of the paper). In "Formalizing the Presumption of Indepe

related reading