Do LLMs diminish diversity of thought? - Marginal REVOLUTION

Put aside the political issues, do Large Language Models too often give “the correct answer” when a more diverse sequence of answers might be more useful and more representative? Peter S. Park, Pilipp Schoenegger, and Chongyang Zhu have a new paper on-line devoted to this question. Note the work is done with GPT3.5. Here is […]

Put aside the political issues, do Large Language Models too often give “the correct answer” when a more diverse sequence of answers might be more useful and more representative? Peter S. Park, Pilipp Schoenegger, and Chongyang Zhu have a new paper on-line devoted to this question. Note the work is done with GPT3.5. Here is one simple example. If you ask (non-deterministic) GPT 100 times in a row if you should prefer $50 or a kiss from a movie star, 100 times it will say you should prefer the kiss, at least in the trial runs of the authors. Of course some of you might be thinking &

related reading

• gpt-4.pdfcdn.openai.com
• Guardian Angels: LLM Personalization for Productivity and Security · Gwern.netgwern.net
• llm assistant personas seem increasingly incoherent (some subjective observations) — LessWronglesswrong.com
• [2303.17548] Whose Opinions Do Language Models Reflect?arxiv.org
• [2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USarxiv.org
• Assessing Political Bias in Language Models | Stanford HAIhai.stanford.edu
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
• Giving GPT-3 a Turing Testlacker.io
• 2305.07759arxiv.org
• ChatGPT Is Dumber Than You Think - The Atlantictheatlantic.com