Modern Hopfield network
Modern Hopfield networks (also known as Dense Associative Memories) are generalizations of the classical Hopfield networks that break the linear scaling relationship between the number of input features and the number of stored memories. This is achieved by introducing stronger non-linearities (either in the energy function or neurons’ activation functions) leading to super-linear (even an exponential) memory storage capacity as a function of the number of feature neurons. The network still requires a sufficient number of hidden neurons.
Modern Hopfield network - Wikipedia Jump to content From Wikipedia, the free encyclopedia Neural networks This article may be too technical for most readers to understand . Please help improve it to make it understandable to non-experts , without removing the technical details. ( October 2021 ) ( Learn how and when to remove this message ) Modern Hopfield networks [ 1 ] [ 2 ] (also known as Dense Associative Memories [ 3 ] ) are generalizations of the classical Hopfield networks that break the linear scaling relationship between the number of input features and the number of stored memories. T
Explore this link on the map →related reading
- Hopfield network - Wikipediaen.wikipedia.org
- Hopfield Networks is All You Need | hopfield-layersml-jku.github.io
- Toy Models of Superpositiontransformer-circuits.pub
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- Zoom In: An Introduction to Circuitsdistill.pub
- NL.pdfabehrouz.github.io
- Neural Networks: Multi-Layer Perceptron and Hopfield Networknicolas.burrus.name
- Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
- Formation of Representations in Neural Networksarxiv.org
- Recurrent neural networks - Scholarpediascholarpedia.org
- how neural networks think at scalemarmik.xyz
- Statistical Mechanics of Deep Learningganguli-gang.stanford.edu