Naturally Occurring Equivariance in Neural Networks

Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights.

Naturally Occurring Equivariance in Neural Networks Naturally Occurring Equivariance in Neural Networks Distill { "title": "Naturally Occurring Equivariance in Neural Networks", "description": "Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights.", "authors": [ { "author": "Chris Olah", "authorURL": "https://colah.github.io", "affiliation": "OpenAI", "affiliationURL": "https://openai.com" }, { "author": "Nick Cammarata", "authorURL": "http://nickcammarata.com", "affiliation": "OpenAI", "affiliationURL": "https://openai.com" }, { "author":

saved by

• Noah T
• Azlen Elza
• Davis Brown
• Alexander Davies

related reading

• Meauring Invariances in Deep Networksai.stanford.edu
• Zoom In: An Introduction to Circuitsdistill.pub
• Feature Visualizationdistill.pub
• Feature-wise transformationsdistill.pub
• Locally Scale-Invariant Convolutional NeuralNetworksarxiv.org
• Toy Models of Superpositiontransformer-circuits.pub
• [1810.13128] The Effect of Learning Strategy versus Inherent Architecture Properties on the Ability of Convolutional Neural Networks to Develop Transformation Invariancearxiv.org
• Multimodal Neurons in Artificial Neural Networksdistill.pub
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• 10. Equivariant Neural Networks — deep learning for molecules & materialsdmol.pub
• The World Inside Neural Networksgoodfire.ai
• Visualizing Weightsdistill.pub