✳flâneur — a map of the web's best reading

Maximum Entropy Distributions | Bounded Rationality

bjlkeng.github.io · 2,663 words · saved by 1 readers

A introduction to maximum entropy distributions.

This post will talk about a method to find the probability distribution that best fits your given state of knowledge. Using the principle of maximum entropy and some testable information (e.g. the mean), you can find the distribution that makes the fewest assumptions about your data (the one with maximal information entropy). As you may have guessed, this is used often in Bayesian inference to determine prior distributions and also (at least implicitly) in natural language processing applications with maximum entropy (MaxEnt) classifiers (i.e. a multinomial logistic regression). As usual, I'll

Explore this link on the map →

related reading