flâneur

Reshelving generalization - by Ben Recht - arg min

argmin.net · 993 words · saved by 1 readers

This is a live blog of Lecture 12 of the 2025 edition of my graduate machine learning class “Patterns, Predictions, and Actions.” A Table of Contents is here. We now turn to the mystical part of the machine learning course. When does a piece of computer code that makes good predictions on a sample of collected data make good predictions on data we haven’t seen? This question asks us to be metaforecasters. To predict prediction errors. Answering the question requires a theory of how collected data relates to the data in the wilds. In machine learning, this metaprediction problem is called generalization theory. In the applied sciences, this is the central question of external validity. The most pervasive model relating the lab to the field assumes (a) data is randomly generated by Omniscient Jones rolling the same many-sided die to select each sample and (b) this data-generating process never changes. We call this the iid model,1 and it’s “useful” insofar as it gives us some rules of th

Reshelving generalization You don't need a theorem to argue more data is better than less data Ben Recht Oct 09, 2025 22 5 Share This is a live blog of Lecture 12 of the 2025 edition of my graduate machine learning class “Patterns, Predictions, and Actions.” A Table of Contents is here . We now turn to the mystical part of the machine learning course. When does a piece of computer code that makes good predictions on a sample of collected data make good predictions on data we haven’t seen? This question asks us to be metaforecasters. To predict prediction errors. Answering the question requires

saved by

related reading