✳flâneur — a map of the web's best reading

Scaling Laws Literature Review

epochai.org · 1,921 words · saved by 1 readers

I have collected a database of scaling laws for different tasks and architectures, and reviewed dozens of papers in the scaling law literature.

Scaling laws literature review | Epoch AI Common shape of a scaling law, taken from Hestness et al. (2017) Executive summary Scaling laws are predictable relations between the scale of a mode and performance or other useful properties. I have collected a database of scaling laws for different tasks and architectures, and reviewed dozens of papers in the scaling law literature. My main takeaways are: Functional forms : a basic power law can effectively model the scaling behavior in the power-law region but not the transitions to the other two regions. For this, either the M4 estimator or the BN

Explore this link on the map →

related reading