On_the_Spectral_Bias_of_Neural_Networks__ICML_ (3)

N/A

On the Spectral Bias of Neural Networks Nasim Rahaman * 1 2 Aristide Baratin * 1 Devansh Arpit 1 Felix Draxler 2 Min Lin 1 Fred A. Hamprecht 2 Yoshua Bengio 1 Aaron Courville 1 Abstract expose this bias by taking a closer look at neural networks Neural networks are known to be a class of highly through the lens of Fourier analysis. While they can ap- expressive functions able to fit even random input- proximate arbitrary functions, we find that these networks…

saved by

• Emil Ryd
• Grace S

related reading

• nn-notes.pdfboris-hanin.github.io
• Toy Models of Superpositiontransformer-circuits.pub
• arxiv.org/pdf/1805.08522arxiv.org
• A Spectral Condition for Feature Learningarxiv.org
• [2404.07200] Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspectivearxiv.org
• 1412.0233arxiv.org
• Fourier Feature Networksbmild.github.io
• Understanding deep learning requires rethinking generalizationarxiv.org
• [2304.03408] Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networksarxiv.org
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• NL.pdfabehrouz.github.io
• Naturally Occurring Equivariance in Neural Networksdistill.pub