flâneur

Most Algorithmic Progress is Data Progress

beren.io · 1,342 words · saved by 2 readers

Epistemic Status: Fairly sure about this from experience but could be missing crucial considerations. I don’t present any super detailed evidence here so it is theoretically just vibes. When forecasting AI progress, the forecasters and modellers often break AI progress down into two components: increased compute, and ‘algorithmic progress’. My...

Epistemic Status : Fairly sure about this from experience but could be missing crucial considerations. I don’t present any super detailed evidence here so it is theoretically just vibes. When forecasting AI progress, the forecasters and modellers often break AI progress down into two components: increased compute, and ‘algorithmic progress’. My argument here is that the term ‘algorithmic progress’ for ‘the remainder after compute’ is misleading and that we should really think about and model AI progress as three terms – compute , algorithms , and data . My claim is that a large fraction (but c

saved by

related reading