How Not to Test GPT-3 - by Gary Marcus and Ernest Davis

Why doing psychology on large language models is harder than you might think

The biggest news in the AI world recently, aside from Bing’s meltdown, Bard’s fizzle, and Tesla’s self-driving, is a Stanford business prof’s recent study on Theory of the Mind. Nearly 4,000 people liked Kevin Fischer’s Tweet saying the result didn’t receive enough attention: Kevin Fischer@KevinAFischer This paper is not receiving enough attention: GPT 3.5 displays emergent theory of mind arxiv.org/pdf/2302.02083… 9:56 AM · Feb 10, 2023 657 Reposts · 3.97K Likes and others were, frankly, worried about what it might mean. If GPT-3 really did master theory of mind (ToM), the part of…

related reading

• AI hype is built on high test scores. Those tests are flawed. | MIT Technology Reviewtechnologyreview.com
• Giving GPT-3 a Turing Testlacker.io
• AI Induced Psychosis: A shallow investigation — LessWronglesswrong.com
• [2302.02083] Theory of Mind May Have Spontaneously Emerged in Large Language Modelsarxiv.org
• Why didn't DeepMind build GPT3?rootnodes.substack.com
• How AI Is Learning to Think in Secretnickandresen.substack.com
• [2302.02083] Evaluating Large Language Models in Theory of Mind Tasksarxiv.org
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org
• The Yale Review | Melanie Mitchell: The Dangerous Unknowns at the…yalereview.org
• Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com
• The Human Skill That Eludes AI - The Atlantictheatlantic.com
• Large language models are proficient in solving and creating emotional intelligence testsnature.com