Principle of maximum entropy
The principle of maximum entropy states that, among all probability distributions consistent with a given set of constraints (such as normalization or specified expectation values), the distribution that maximizes Shannon entropy should be selected. This yields the least committal distribution compatible with the known constraints, introducing no structure beyond what is logically implied by the available information.
Principle of maximum entropy - Wikipedia Jump to content From Wikipedia, the free encyclopedia Principle in Bayesian statistics For other uses of "Maximum entropy", see Maximum entropy (disambiguation) . Part of a series on Bayesian statistics Posterior = Likelihood × Prior ÷ Evidence Background Bayesian inference Bayesian probability Bayes' theorem Bernstein–von Mises theorem Coherence Cox's theorem Cromwell's rule Likelihood principle Principle of indifference Principle of maximum entropy Model building Conjugate prior Linear regression Empirical Bayes Hierarchical model Posterior approximat
Explore this link on the map →related reading
- Maximum Entropy Methods (MaxEnt)bactra.org
- Maximum Entropy Distributions | Bounded Rationalitybjlkeng.github.io
- A Maximum Entropy Intuition for Fundamental Statistical Distributions | long intuitionlongintuition.com
- Visual Information Theory -- colah's blogcolah.github.io
- A Mathematical Theory of Communicationpeople.math.harvard.edu
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- Boltzmann distribution - Wikipediaen.wikipedia.org
- Prior probability - Wikipediaen.wikipedia.org
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- Principle of indifference - Wikipediaen.wikipedia.org
- Gibbs' inequality - Wikipediaen.wikipedia.org
- Maximum likelihood estimation - Wikipediaen.wikipedia.org