Deliberative Alignment, And The Spec - by Scott Alexander

...

In the past day, Zvi has written about deliberative alignment, and OpenAI has updated their spec. This article was written before either of these and doesn’t account for them, sorry. OpenAI has bad luck with its alignment teams. The first team quit en masse to found Anthropic, now a major competitor. The second team quit en masse to protest the company reneging on safety commitments. The third died in a tragic plane crash. The fourth got washed away in a flood. The fifth through eighth were all slain by various types of wild beast. But the ninth team is still there and doing good work.…

related reading

• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Alignment is not solved but it increasingly looks solvablealigned.substack.com
• The Artificiality of Alignmentjoinreboot.org
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Deep Deceptiveness — LessWronglesswrong.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Thoughts on Claude’s Constitution – Windows On Theorywindowsontheory.org
• Alignment & Succession: The Two Bars of Alignmentnosetgauge.com
• Constitutional AI vs. RLHF vs. Deliberative Alignment — LessWronglesswrong.com
• Introducing the Model Spec | OpenAIopenai.com
• The Universe from an Intentional Stancecasparoesterheld.com