constitutional.pdf
anthropic.com · 295 words · saved by 2 readers
N/A
Abstract Developing safe and useful general-purpose AI systems will require us to make progress on scalable oversight: the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand. Empirical work on this problem is not straightforward, since we do not yet have systems that broadly exceed our abilities. This paper discusses one of the major ways we think about this problem, with a focus on ways it can be studied empirically. We first present an experimental design centered on tasks for which human specialists succeed but unaided humans and…
saved by
related reading
- Measuring Progress on Scalable Oversight for Large Language Models \ Anthropicanthropic.com
- Oversight Assistants: Turning Compute into Understandingbounded-regret.ghost.io
- Automated Alignment Researchers: Using large language models to scale scalable oversight \ Anthropicanthropic.com
- Alignment is not solved but it increasingly looks solvablealigned.substack.com
- [1911.01547] On the Measure of Intelligencearxiv.org
- Will scaling work?dwarkeshpatel.com
- Oversight Assistants: Turning Compute into Understandingbounded-regret.ghost.io
- The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
- Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- Scaling Activation Oracles to Trillion-Parameter Modelstransluce.org
- AI Control: Improving Safety Despite Intentional Subversion — AI Alignment Forumalignmentforum.org