Self-Other Overlap: A Neglected Approach to AI Alignment — LessWrong

Figure 1. Image generated by DALL·E 3 to represent the concept of self-other overlap Many thanks to Bogdan Ionut-Cirstea, Steve Byrnes, Gunnar Zarnac…

x Self-Other Overlap: A Neglected Approach to AI Alignment — LessWrong AI Risk Has Diagram AI Safety Camp Deceptive Alignment Embedded Agency AI Frontpage 247 Self-Other Overlap: A Neglected Approach to AI Alignment by Marc Carauleanu , Mike Vaiana , Kvee , Diogo de Lucena , Cameron Berg , Trent Hodgeson 30th Jul 2024 AI Alignment Forum 14 min read 53 247 Ω 61 Figure 1. Image generated by DALL·E 3 to represent the concept of self-other overlap Many thanks to Bogdan Ionut-Cirstea, Steve Byrnes, Gunnar Zarnacke, Jack Foxabbott and Seong Hah Cho for critical comments and feedback on earlier and o

saved by

• Tazik Sh
• Lydia Nottingham
• Jacob G-W
• Jason Hausenloy
• Bhagyesh Kumar

related reading

• Towards Safe and Honest AI Agents with Neural Self-Other Overlaparxiv.org
• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Deep Deceptiveness — LessWronglesswrong.com
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentalignmentpretraining.ai
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• [2601.10160] Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentarxiv.org
• How To Go From Interpretability To Alignment: Just Retarget The Search — LessWronglesswrong.com
• Making deals with early schemers — LessWronglesswrong.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
• How far does alignment midtraining generalize?alignment.openai.com
• Auditing language models for hidden objectives — LessWronglesswrong.com