Multi-agent safety — AI Alignment Forum
alignmentforum.org · 3,123 words · saved by 1 readers
Note: this post is most explicitly about safety in multi-agent training regimes. However, many of the arguments I make are also more broadly applicab…
x Multi-agent safety — AI Alignment Forum Shaping safer goals AI Frontpage 15 Multi-agent safety by Richard_Ngo 16th May 2020 6 min read 8 15 Note: this post is most explicitly about safety in multi-agent training regimes. However, many of the arguments I make are also more broadly applicable - for example, when training a single agent in a complex environment, challenges arising from the environment could play an analogous role to challenges arising from other agents. In particular, I expect that the diagram in the 'Developing General Intelligence' section will be applicable to most possible
related reading
- Why Tool AIs Want to Be Agent AIs · Gwern.netgwern.net
- Why multi-agent safety is important — LessWronglesswrong.com
- AGI safety career advice — EA Forumforum.effectivealtruism.org
- Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover — LessWronglesswrong.com
- AGI safety from first principles: Introduction — LessWronglesswrong.com
- How far does alignment midtraining generalize?alignment.openai.com
- Towards a scale-free theory of intelligent agency — AI Alignment Forumalignmentforum.org
- Commentary on AGI Safety from First Principles — AI Alignment Forumalignmentforum.org
- AGI safetysjbyrnes.com
- [AN #159]: Building agents that know how to experiment, by training on procedurally generated games — AI Alignment Forumalignmentforum.org
- Of Swarms and Sand Godsblog.cosmos-institute.org
- Distributional AGI Safetyarxiv.org