why Occam?

yudhister.me · 387 words · saved by 2 readers

[1] "Ideal Bayesian reasoners" rely on the simplicity prior. This is false as stated: Solomonoff induction convergence is not dependent on the exact choice of the 2 − | 𝐾 | simplicity prior; convergence over any computable distribution holds if the prior is any universal semimeasure.1 [2] The generalization bias described in the Bayesian free-energy functional is a bias towards low-description length programs. I don't know enough about this to provide a full, concise, precise accounting of the argument, but I buy it? See my earlier post on MDL and SLT. (Hopefully work on the inductive bias of SGD will shed light on a similar result in the training of neural networks). [3] "Simple hypotheses" are adaptive because world phenomena are naturally generated by "simple" processes. Empirically, phenomena have parsimonious explanations. We do not live in the most simple world,2 but physical theories are decomposable. [4] "Simple hypotheses" are adaptive because learning systems learn simple

[1] “Ideal Bayesian reasoners” rely on the simplicity prior. This is false as stated: Solomonoff induction convergence is not dependent on the exact choice of the \(2^{-|K|}\) simplicity prior; convergence over any computable distribution holds if the prior is any universal semimeasure. 1 [2] The generalization bias described in the Bayesian free-energy functional is a bias towards low-description length programs. I don’t know enough about this to provide a full, concise, precise accounting of the argument, but I buy it? See my earlier post on MDL and SLT . (Hopefully work on the inductive bia

why Occam?

saved by

related reading