Models are cultures

When we train a machine learning model, on a corpus of human-produced media, fine-tune it on specific examples, and give it feedback on its interpretation of these things, we are doing something similar to the production of a culture. Historical human cultures were formed by a mixture of exchange, innovation and divergence. They gave rise to different languages, philosophies, artistic styles, economic systems, and forms of social organisation. Each new generation learns about the world from a mixture of personal experience and the stories and explanations given to them by the culture they grow up in. Pre-literate cultures had very narrow bandwidth for communicating their ideas between generations, and used myth and storytelling to encode the information that they valued most highly. These stories were densely interwoven mixtures of factual history, conceptual world-view, and ethical guidance, maintained and evolved by poets, priests and prophets. Everything else had to be learned and p

When we train a machine learning model, on a corpus of human-produced media, fine-tune it on specific examples, and give it feedback on its interpretation of these things, we are doing something similar to the production of a culture . Historical human cultures were formed by a mixture of exchange, innovation and divergence. They gave rise to different languages, philosophies, artistic styles, economic systems, and forms of social organisation. Each new generation learns about the world from a mixture of personal experience and the stories and explanations given to them by the culture they gro

related reading

• First we shape our social graph; then it shapes usescapingflatland.substack.com
• First we shape our social graph; then it shapes ushenrikkarlsson.xyz
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com
• Gradual Disempowermentgradual-disempowerment.ai
• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• the post-literate societyjasmi.news
• Best explanations of how LLMs work | Roman Vorushinvorushin.github.io
• Language models are weird for the same reason human cultures are weirdsubstack.com
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• The Dark Forest and Generative AImaggieappleton.com
• What language do language models speak?tcz.hu