On_the_Spectral_Bias_of_Neural_Networks__ICML_ (3)

N/A

On the Spectral Bias of Neural Networks Nasim Rahaman * 1 2 Aristide Baratin * 1 Devansh Arpit 1 Felix Draxler 2 Min Lin 1 Fred A. Hamprecht 2 Yoshua Bengio 1 Aaron Courville 1 Abstract expose this bias by taking a closer look at neural networks Neural networks are known to be a class of highly through the lens of Fourier analysis. While they can ap- expressive functions able to fit even random input- proximate arbitrary functions, we find that these networks…

saved by

• Emil Ryd
• Grace S

related reading

• Fourier Feature Networksbmild.github.io
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• Zoom In: An Introduction to Circuitsdistill.pub
• nn-notes.pdfboris-hanin.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• A Recipe for Training Neural Networkskarpathy.github.io
• Neural network training makes beautiful fractals | Jascha’s blogsohl-dickstein.github.io
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• A Spectral Condition for Feature Learningarxiv.org
• Feature Visualizationdistill.pub
• arxiv.org/pdf/1805.08522arxiv.org
• On neural scaling and the quanta hypothesisericjmichaud.com