Conditional entropy - Wikipedia
In information theory, the conditional entropy quantifies the amount of information needed to describe the outcome of a random variable Y given that the value of another random variable X is known. Here, information is measured in shannons, nats, or hartleys. The entropy of Y conditioned on X is written as H(Y|X) .
Conditional entropy - Wikipedia Jump to content From Wikipedia, the free encyclopedia Measure of relative information in probability theory Venn diagram showing additive and subtractive relationships various information measures associated with correlated variables X {\displaystyle X} and Y {\displaystyle Y} . The area contained by both circles is the joint entropy H ( X , Y ) {\displaystyle \mathrm {H} (X,Y)} . The circle on the left (red and violet) is the individual entropy H ( X ) {\displaystyle \mathrm {H} (X)} , with the red being the conditional entropy H ( X | Y ) {\displaystyle \mathr
saved by
related reading
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- [2206.07867] A visual introduction to information theoryarxiv.org
- Visual Information Theory -- colah's blogcolah.github.io
- De Finetti's theorem - Wikipediaen.wikipedia.org
- Binary entropy function - Wikipediaen.wikipedia.org
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- Entropy in thermodynamics and information theory - Wikipediaen.wikipedia.org
- Information entropy (Foundations of information theory: Part 2) - Matthew N. Bernsteinmbernste.github.io
- Principle of maximum entropy - Wikipediaen.wikipedia.org
- Maximum Entropy Distributions | Bounded Rationalitybjlkeng.github.io
- Entropy coding - Wikipediaen.wikipedia.org
- Things that confused me about cross-entropychris-said.io