Language as a Cognitive Tool: Dall-E, Humans and Vygotskian RL Agents – Developmental Systems, a Blog of the Flowers Lab

Developmental Systems, a Blog of the Flowers Lab

Have you heard about DALL-E? Beyond the funny mashup between Pixar’s robot and the surrealist painter, Dall-E is OpenAI’s new transformer trained to compose images from text descriptions [ DALL-E ]. Most people agree, DALL-E’s ability to blend concepts into natural images is remarkable; it often composes just like you would. Look at these avocado-chairs: Output from OpenAI’s Dall-E model when prompted by: "an armchair in the shape of an avocado” [Dall-E] If it’s so impressive, it’s because this type of composition feels like a very special human ability. How would it know? The ability to compo

related reading

• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• Using Artificial Intelligence to Augment Human Intelligencedistill.pub
• [2304.03442] Generative Agents: Interactive Simulacra of Human Behaviorarxiv.org
• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• To Understand Language is to Understand Generalization | Eric Jangevjang.com
• Emergence of Human to Robot Transfer in Vision-Language-Action Modelspi.website
• Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• [2304.03442] Generative Agents: Interactive Simulacra of Human Behaviorarxiv.org
• 2301.06627arxiv.org
• [2301.06627] Dissociating language and thought in large language models: a cognitive perspectivearxiv.org
• Offline RL and Large Language Models - by Sergey Levinesergeylevine.substack.com