Fourier Feature Networks

Project page for Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains.

Fourier Feature Networks --> --> Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains NeurIPS 2020 (spotlight) Matthew Tancik* UC Berkeley Pratul Srinivasan* UC Berkeley Ben Mildenhall* UC Berkeley Sara Fridovich-Keil UC Berkeley Nithin Raghavan UC Berkeley Utkarsh Singhal UC Berkeley Ravi Ramamoorthi UC San Diego Jonathan T. Barron Google Research Ren Ng UC Berkeley *denotes equal contribution Paper Code Abstract We show that passing input points through a simple Fourier feature mapping enables a multilayer perceptron (MLP) to learn high-frequency functions

saved by

• Vihaan Sondhi
• Noah T

related reading

• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• Feature-wise transformationsdistill.pub
• Feature Visualizationdistill.pub
• Naturally Occurring Equivariance in Neural Networksdistill.pub
• Toy Models of Superpositiontransformer-circuits.pub
• On_the_Spectral_Bias_of_Neural_Networks__ICML_ (3)proceedings.mlr.press
• NeRF: Neural Radiance Fieldsmatthewtancik.com
• [2404.07200] Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspectivearxiv.org
• Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizersgoodfire.ai
• Learned Initializations for Neural Representationsmatthewtancik.com
• Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
• [2304.03408] Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networksarxiv.org