Cross-entropy | Anıl Zenginoğlu
Cross-entropy is a measure of the difference between two probability distributions. It also plays a role in the information-theoretic formulation of the Second Law of Thermodynamics, thereby providing a bridge between physics and machine learning.
Cross-entropy Crossing the bridge between physics and machine learning. Anıl Zenginoğlu Apr 2, 2023 13 min read We're all gonna die. Blame the Second Law of Thermodynamics. Entropy increases, we get older, and we die. Mark Thompson, New Yorker Magazine This perception of entropy, prevalent in popular culture, presents it as the driving force for decay and disorder. But this cannot be the whole story. The incredible abundance of life on our planet would not have been possible without the self-organization of complex systems. The story of entropy and life is more complicated than the boy in the
saved by
related reading
- A Gentle Introduction to Cross-Entropy for Machine Learning - MachineLearningMastery.commachinelearningmastery.com
- The Entropy of A Markov Chainchillphysicsenjoyer.substack.com
- Things that confused me about cross-entropychris-said.io
- Six (and a half) intuitions for KL divergence — LessWronglesswrong.com
- What is Entropy? | Jason Fantljasonfantl.com
- 12.3 Second Law of Thermodynamics: Entropy - Physics | OpenStaxopenstax.org
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- Introduction to abstract entropy — LessWronglesswrong.com
- Computational Foundations for the Second Law of Thermodynamics-Stephen Wolfram Writingswritings.stephenwolfram.com
- Visual Information Theory -- colah's blogcolah.github.io
- Entropy in thermodynamics and information theory - Wikipediaen.wikipedia.org
- Entropy: The Hidden Force Making Life Complicatedfs.blog