How Not to Test GPT-3 - by Gary Marcus and Ernest Davis

Why doing psychology on large language models is harder than you might think

The biggest news in the AI world recently, aside from Bing’s meltdown, Bard’s fizzle, and Tesla’s self-driving, is a Stanford business prof’s recent study on Theory of the Mind. Nearly 4,000 people liked Kevin Fischer’s Tweet saying the result didn’t receive enough attention: Kevin Fischer@KevinAFischer This paper is not receiving enough attention: GPT 3.5 displays emergent theory of mind arxiv.org/pdf/2302.02083… 9:56 AM · Feb 10, 2023 657 Reposts · 3.97K Likes and others were, frankly, worried about what it might mean. If GPT-3 really did master theory of mind (ToM), the part of…

related reading

• [2302.02083] Theory of Mind May Have Spontaneously Emerged in Large Language Modelsarxiv.org
• AI hype is built on high test scores. Those tests are flawed. | MIT Technology Reviewtechnologyreview.com
• [2302.02083] Evaluating Large Language Models in Theory of Mind Tasksarxiv.org
• gpt-4.pdfcdn.openai.com
• Giving GPT-3 a Turing Testlacker.io
• Are A.I. Text Generators Thinking Like Humans — Or Just Very Good at Convincing Us They Are? | Stanford Graduate School of Businessgsb.stanford.edu
• [2302.03494] A Categorical Archive of ChatGPT Failuresarxiv.org
• Noam Chomsky and GPT-3 - by Gary Marcus - Marcus on AIgarymarcus.substack.com
• Theory of Mind May Have Spontaneously Emerged in Large Language Modelsnews.ycombinator.com
• AI Induced Psychosis: A shallow investigation — LessWronglesswrong.com
• How AI Is Learning to Think in Secretnickandresen.substack.com
• Why didn't DeepMind build GPT3?rootnodes.substack.com