Relaxed adversarial training for inner alignment - AI Alignment Forum
This post is part of research I did at OpenAI with mentoring and guidance from Paul Christiano. It also represents my current agenda regarding what I believe looks like the most promising approach fo…
x Relaxed adversarial training for inner alignment — AI Alignment Forum Inner Alignment AI Risk Interpretability (ML & AI) AI Frontpage 31 Relaxed adversarial training for inner alignment by evhub 10th Sep 2019 33 min read 27 31 This post is part of research I did at OpenAI with mentoring and guidance from Paul Christiano. It also represents my current agenda regarding what I believe looks like the most promising approach for addressing inner alignment . One particularly concerning failure mode for any sort of advanced AI system is for it to have good performance on the training distribution,
Explore this link on the map →related reading
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- Research Areas in Evaluation and Guarantees in Reinforcement Learning (The Alignment Project by UK AISI) — AI Alignment Forumalignmentforum.org
- 39 - Evan Hubinger on Model Organisms of Misalignment | AXRP - the AI X-risk Research Podcastaxrp.net
- Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
- An overview of 11 proposals for building safe advanced AI — AI Alignment Forumalignmentforum.org
- Inner Alignment: Explain like I'm 12 Edition — LessWronglesswrong.com
- Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
- How likely is deceptive alignment? — AI Alignment Forumalignmentforum.org
- Teaching Claude why \ Anthropicanthropic.com
- Research Areas in Methods for Post-training and Elicitation (The Alignment Project by UK AISI) — AI Alignment Forumalignmentforum.org
- [AN #70]: Agents that help humans who are still learning about their own preferences — LessWronglesswrong.com
- Externalized reasoning oversight: a research direction for language model alignment — AI Alignment Forumalignmentforum.org