Infomax - Wikipedia
Infomax, or the principle of maximum information preservation, is an optimization principle for artificial neural networks and other information processing systems. It prescribes that a function that maps a set of input values 𝑥 to a set of output values 𝑧 ( 𝑥 ) should be chosen or learned so as to maximize the average Shannon mutual information between 𝑥 and 𝑧 ( 𝑥 ) , subject to a set of specified constraints and/or noise processes. Infomax algorithms are learning algorithms that perform this optimization process. The principle was described by Linsker in 1988.[1] The objective function is called the InfoMax objective. As the InfoMax objective is difficult to compute exactly, a related notion uses two models giving two outputs 𝑧 1 ( 𝑥 ) , 𝑧 2 ( 𝑥 ) , and maximizes the mutual information between these. This contrastive InfoMax objective is a lower bound to the InfoMax objective.[2] Infomax, in its zero-noise limit, is related to the principle of redundancy reduction p
From Wikipedia, the free encyclopedia Infomax', or the principle of maximum information preservation, is an optimization principle for artificial neural networks and other information processing systems. It prescribes that a function that maps a set of input values to a set of output values should be chosen or learned so as to maximize the average Shannon mutual information between and , subject to a set of specified constraints and/or noise processes. Infomax algorithms are learning algorithms that perform this optimization process. The principle was described by Linsker in 1988.[1] The…
saved by
related reading
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- Principle of maximum entropy - Wikipediaen.wikipedia.org
- [2206.07867] A visual introduction to information theoryarxiv.org
- book.pdfinference.org.uk
- Free energy principle - Wikipediaen.wikipedia.org
- [1606.03657] InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netsarxiv.org
- Dimensionality reduction in neural data analysisxcorr.net
- [physics/0004057] 1 Introductionar5iv.labs.arxiv.org
- Minimax - Wikipediaen.wikipedia.org
- nn-notes.pdfboris-hanin.github.io
- Information theory in neuroscience - PMCncbi.nlm.nih.gov
- Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu