Relaxed adversarial training for inner alignment - AI Alignment Forum

This post is part of research I did at OpenAI with mentoring and guidance from Paul Christiano. It also represents my current agenda regarding what I believe looks like the most promising approach fo…

x Relaxed adversarial training for inner alignment — AI Alignment Forum Inner Alignment AI Risk Interpretability (ML & AI) AI Frontpage 31 Relaxed adversarial training for inner alignment by evhub 10th Sep 2019 33 min read 27 31 This post is part of research I did at OpenAI with mentoring and guidance from Paul Christiano. It also represents my current agenda regarding what I believe looks like the most promising approach for addressing inner alignment . One particularly concerning failure mode for any sort of advanced AI system is for it to have good performance on the training distribution,

related reading

• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• In (highly contingent!) defense of interpretability-in-the-loop ML training — AI Alignment Forumalignmentforum.org
• Reinforcement learning towards broadly and persistently beneficial modelsalignment.openai.com
• Deep Deceptiveness — LessWronglesswrong.com
• Alignment Faking Mitigationsalignment.anthropic.com
• How far does alignment midtraining generalize?alignment.openai.com
• Auditing language models for hidden objectives — LessWronglesswrong.com
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• Risks from Learned Optimization: Introduction — LessWronglesswrong.com
• An overview of 11 proposals for building safe advanced AI — AI Alignment Forumalignmentforum.org
• Another (outer) alignment failure story — AI Alignment Forumalignmentforum.org