Infomax - Wikipedia
Infomax, or the principle of maximum information preservation, is an optimization principle for artificial neural networks and other information processing systems. It prescribes that a function that maps a set of input values 𝑥 to a set of output values 𝑧 ( 𝑥 ) should be chosen or learned so as to maximize the average Shannon mutual information between 𝑥 and 𝑧 ( 𝑥 ) , subject to a set of specified constraints and/or noise processes. Infomax algorithms are learning algorithms that perform this optimization process. The principle was described by Linsker in 1988.[1] The objective function is called the InfoMax objective. As the InfoMax objective is difficult to compute exactly, a related notion uses two models giving two outputs 𝑧 1 ( 𝑥 ) , 𝑧 2 ( 𝑥 ) , and maximizes the mutual information between these. This contrastive InfoMax objective is a lower bound to the InfoMax objective.[2] Infomax, in its zero-noise limit, is related to the principle of redundancy reduction p
Explore this link on the map →