Modern Hopfield network
Modern Hopfield networks (also known as Dense Associative Memories) are generalizations of the classical Hopfield networks that break the linear scaling relationship between the number of input features and the number of stored memories. This is achieved by introducing stronger non-linearities (either in the energy function or neurons’ activation functions) leading to super-linear (even an exponential) memory storage capacity as a function of the number of feature neurons. The network still requires a sufficient number of hidden neurons.
Modern Hopfield network - Wikipedia Jump to content From Wikipedia, the free encyclopedia Neural networks This article may be too technical for most readers to understand . Please help improve it to make it understandable to non-experts , without removing the technical details. ( October 2021 ) ( Learn how and when to remove this message ) Modern Hopfield networks [ 1 ] [ 2 ] (also known as Dense Associative Memories [ 3 ] ) are generalizations of the classical Hopfield networks that break the linear scaling relationship between the number of input features and the number of stored memories. T
related reading
- Hopfield network - Wikipediaen.wikipedia.org
- Hopfield Networks is All You Need | hopfield-layersml-jku.github.io
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- Toy Models of Superpositiontransformer-circuits.pub
- Zoom In: An Introduction to Circuitsdistill.pub
- NL.pdfabehrouz.github.io
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
- [2608.13335] Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Lawsarxiv.org
- nn-notes.pdfboris-hanin.github.io
- Neural Networks: Multi-Layer Perceptron and Hopfield Networknicolas.burrus.name
- Neural networks: from the perceptron to deep netsarxiv.org