Machine Learning Research Blog – Francis Bach

The past posts on optimization scaling laws [1, 2] focused on problems that do not become significantly harder as the problem size increases. We showed that for some problems, as the dimension 𝑑 𝑑 goes to infinity, the optimality gap converges at a sublinear rate Θ( 𝑘 −𝑝 ) Θ ( 𝑘 − 𝑝 ) for some power 𝑝 𝑝 depending on the problem, but independent… In the last few years, we have seen a surge of empirical and theoretical works about “scaling laws”, whose goals are to characterize the performance of learning methods based on various problem parameters (e.g., number of observations and parameters, or amount of compute). From a theoretical point of view, this marks a renewed interest in… This month, we pursue our exploration of spectral properties of kernel matrices. As mentioned in a previous post, understanding how eigenvalues decay is not only fun but also key to understanding algorithmic and statistical properties of many learning methods (see, e.g., chapter 7 of my book “Le

saved by

• Dawson Yao