Thoughts on the impact of RLHF research — LessWrong
In this post I’m going to describe my basic justification for working on RLHF in 2017-2020, which I still stand behind. I’ll discuss various argument…
x Thoughts on the impact of RLHF research — LessWrong RLHF AI Frontpage 255 Thoughts on the impact of RLHF research by paulfchristiano 25th Jan 2023 AI Alignment Forum 11 min read 102 255 Ω 111 In this post I’m going to describe my basic justification for working on RLHF in 2017-2020, which I still stand behind. I’ll discuss various arguments that RLHF research had an overall negative impact and explain why I don’t find them persuasive. I'll also clarify that I don't think research on RLHF is automatically net positive; alignment research should address real alignment problems, and we should r
saved by
related reading
- Thoughts on the impact of RLHF research — AI Alignment Forumalignmentforum.org
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
- Many arguments for AI x-risk are wrong — AI Alignment Forumalignmentforum.org
- Defining alignment research — LessWronglesswrong.com
- Many arguments for AI x-risk are wrong — LessWronglesswrong.com
- Constitutional AI vs. RLHF vs. Deliberative Alignment — LessWronglesswrong.com
- Thoughts on the Alignment Implications of Scaling Language Models | Leo Gaobmk.sh
- How RLHF actually works - by Nathan Lambertinterconnects.ai
- Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
- I am worried about near-term non-LLM AI developments — LessWronglesswrong.com