flâneur

Language Models, World Models, and Human Model-Building

lingo.csail.mit.edu · 2,960 words · saved by 8 readers

Does the computation performed by language models involve real language understanding, or just string manipulation? When the first really high-quality neural language models first appeared, this Big Question was mainly formulated in terms of whether LMs could represent meaning. The general recipe was to inspect an LM’s internal representations, and to find a simple procedure for translating them into something resembling a human-approved meaning representation in some formal linguistic theory. It turned out that the hard part of this problem had nothing to do with LMs: it was figuring out what “meaning” means, and what these human-approved representations should look like in the first place. So the NLP interpretability community immediately started arguing about model-theoretic vs relational theories of meaning; whether it was enough for models’ internal representations to be isomorphic to human-designed meaning representations, or whether explicit links to perception and action were s

Language Models, World Models, and Human Model-Building Language Models, World Models, and Human Model-Building Jacob Andreas, 26 July 2024 Does the computation performed by language models involve real language understanding, or just string manipulation? When the first really high-quality neural language models first appeared, this Big Question was mainly formulated in terms of whether LMs could represent meaning . The general recipe was to inspect an LM’s internal representations, and to find a simple procedure for translating them into something resembling a human-approved meaning represent

saved by

related reading