Personas in a Pre-Internet Language Model — LessWrong

David Africa & Nick Levine • TLDR We ran several persona experiments on Talkie, a 13B model trained only on English text from before 1931. We ran th…

David Africa & Nick Levine TLDR We ran several persona experiments on Talkie, a 13B model trained only on English text from before 1931. We ran them alongside its modern twin (same architecture, trained on regular data as a control) and Llama-3.1-8B. Talkie can generalise maliciously. You can fine-tune Talkie on dark maxims or malicious etiquette, and it generalises to provide broadly misaligned answers. Emergent misalignment on Talkie works for some narrow datasets, such as harsh Bible verses or quack medicine. Talkie exhibits qualitatively different emergent misalignment. Talkie…

saved by

• Ishaan Panigrahi
• Bhagyesh Kumar
• Alex

related reading

• Introducing talkie: a 13B vintage language model from 1930talkie-lm.com
• Probing Persona-Dependent Preferences in Language Modelsarxiv.org
• Persona Features Control Emergent Misalignmentarxiv.org
• Introducing talkie: a 13B vintage language model from 1930simonwillison.net
• Synthetic Persona Pretraining: Alignment from Token Zeromodelraising.ai
• The Persona Selection Model: Why AI Assistants might Behave like Humansalignment.anthropic.com
• Thoughts on the persona selection modellesswrong.com
• llm assistant personas seem increasingly incoherent (some subjective observations) — LessWronglesswrong.com
• [2302.08582] Pretraining Language Models with Human Preferencesarxiv.org
• Surfacing Pathological Behaviors in Language Models | Transluce AItransluce.org
• Prefill awareness: can LLMs tell when “their” message history has been tampered with? — LessWronglesswrong.com
• 2308.03958arxiv.org