flâneur

What is information? (Foundations of information theory: Part 1) - Matthew N. Bernstein

mbernste.github.io · 726 words · saved by 1 readers

The mathematical field of information theory attempts to mathematically describe the concept of “information”. In this series of posts, I will attempt to describe my understanding of how, both philosophically and mathematically, information theory defines the polymorphic, and often amorphous, concept of information. In this first post, I will describe Shannon’s self-information. Information theory is a mathematical theory for describing “information” invented by the late Claude Shannon. Information theory is a fundamental mathematical theory that touches many fields, from statistics to physics. Thus, it is introduced in many introductory college courses. Strikingly, I found that whenever information theory was taught to me in these courses, the teacher always skipped over what to me seemed to be the most fundamental question of all: what exactly is “information”? It seemed quite strange to learn a theory without defining the subject of that theory! One reason I think this question is a

The mathematical field of information theory attempts to mathematically describe the concept of “information”. In this series of posts, I will attempt to describe my understanding of how, both philosophically and mathematically, information theory defines the polymorphic, and often amorphous, concept of information. In this first post, I will describe Shannon’s self-information. Introduction Information theory is a mathematical theory for describing “information” invented by the late Claude Shannon . Information theory is a fundamental mathematical theory that touches many fields, from statist

related reading