Thoughts on the impact of RLHF research - AI Alignment Forum
In this post I’m going to describe my basic justification for working on RLHF in 2017-2020, which I still stand behind. I’ll discuss various arguments that RLHF research had an overall negative impac…
x Thoughts on the impact of RLHF research — AI Alignment Forum RLHF AI Frontpage 111 Thoughts on the impact of RLHF research by paulfchristiano 25th Jan 2023 11 min read 102 111 In this post I’m going to describe my basic justification for working on RLHF in 2017-2020, which I still stand behind. I’ll discuss various arguments that RLHF research had an overall negative impact and explain why I don’t find them persuasive. I'll also clarify that I don't think research on RLHF is automatically net positive; alignment research should address real alignment problems, and we should reject a vague as
saved by
related reading
- Thoughts on the impact of RLHF research — LessWronglesswrong.com
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
- Tips for Empirical Alignment Research — AI Alignment Forumalignmentforum.org
- Many arguments for AI x-risk are wrong — AI Alignment Forumalignmentforum.org
- Thoughts on the Alignment Implications of Scaling Language Models | Leo Gaobmk.sh
- Defining alignment research — LessWronglesswrong.com
- Why I’m optimistic about our alignment approachaligned.substack.com
- Can we safely automate alignment research? - Joe Carlsmithjoecarlsmith.com
- How RLHF actually works - by Nathan Lambertinterconnects.ai
- AI #23: Fundamental Problems with RLHFthezvi.substack.com