Can we safely automate alignment research? - Joe Carlsmith

It's really important; we have a real shot; there are a lot of ways we can fail.

Can we safely automate alignment research? - Joe Carlsmith How do we solve the alignment problem? / Part 6 Can we safely automate alignment research? Contents hide 1. Introduction 1.1 Executive summary 2. Why is automating alignment research so important? 3. Alignment MVPs 3.1 What if neither of these approaches are viable? 3.2 Alignment MVPs don’t imply “hand-off” 4. Why might automated alignment research fail? 5. Evaluation failures 5.1 Output-focused and process-focused evaluation 5.2 Human output-focused evaluation 5.3 Scalable oversight 5.4 Process-focused techniques 6 Comparisons with ot

saved by

• Yudhister Joel Kumar
• Eric Huang
• Davis Brown

related reading

• Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
• Tips for Empirical Alignment Research — AI Alignment Forumalignmentforum.org
• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• The Universe from an Intentional Stancecasparoesterheld.com
• How might we safely pass the buck to AI? — LessWronglesswrong.com
• “Alignment Engineering” vs. “Misalignment Science”lesswrong.com
• Automated Alignment is Harder Than You Think — LessWronglesswrong.com
• The Artificiality of Alignmentjoinreboot.org
• Readings on the nature of alignment researchcasparoesterheld.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• Automated alignment is harder than you thinkarxiv.org
• Whose alignment research are we automating?firstscattering.com