Do different languages really convey information at the same rate? [research review] – Sean Trott – Assistant Professor at UCSD
Language is used across all known human populations, but the languages used across these populations also exhibit striking differences. A longstanding goal in Linguistics has thus been to identify universal features of language: characteristics that are shared across all human languages. Understanding which features appear to be invariant across languages would shed light on the origins of language, as well as the potential biological constraints that act as “filters” for the kinds of forms that languages can take.
Do different languages really convey information at the same rate? [research review] Language is used across all known human populations, but the languages used across these populations also exhibit striking differences. A longstanding goal in Linguistics has thus been to identify universal features of language: characteristics that are shared across all human languages. Understanding which features appear to be invariant across languages would shed light on the origins of language, as well as the potential biological constraints that act as “filters” for the kinds of forms that languages can
related reading
- Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
- Visual Information Theory -- colah's blogcolah.github.io
- The Unbearable Slowness of Beingarxiv.org
- The Unintelligibility is Ours: Notes on Chain of Thought1a3orn.com
- [2206.07867] A visual introduction to information theoryarxiv.org
- How can LLM RL Work Despite Information-Theoretic Inefficiencyberen.io
- Reinventing Entropy | Compression is Intelligence Part 1youtube.com
- Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
- Music and Languagedavidtemperley.com
- [2005.14165] Language Models are Few-Shot Learnersarxiv.org
- Language pitch · Erik Bernhardssonerikbern.com
- The real data wall is billions of years of evolutiondynomight.net