Proposal: Using Monte Carlo tree search instead of RLHF for alignment research — LessWrong

Currently the most powerful techniques for getting a language model to act as an agent is via RLHF and similar approaches. For example, ChatGPT was t…

x Proposal: Using Monte Carlo tree search instead of RLHF for alignment research — LessWrong AI Risk AIXI Inner Alignment Language Models (LLMs) RLHF AI Frontpage 2 Proposal: Using Monte Carlo tree search instead of RLHF for alignment research by Christopher King 20th Apr 2023 4 min read 7 2 Currently the most powerful techniques for getting a language model to act as an agent is via RLHF and similar approaches. For example, ChatGPT was trained to be an agent that tries to give humans answers that they want. Another approach is taking a LLM and getting it to predict what the agent you want wou

saved by

• Soham Shah

related reading

• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
• Alignment remains a hard, unsolved problem — LessWronglesswrong.com
• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• State of RL for reasoning LLMs | A. Weersaweers.de
• How To Go From Interpretability To Alignment: Just Retarget The Search — LessWronglesswrong.com
• Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
• Thoughts on the impact of RLHF research — LessWronglesswrong.com
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
• Externalized reasoning oversight: a research direction for language model alignment — AI Alignment Forumalignmentforum.org
• Thoughts on the impact of RLHF research — AI Alignment Forumalignmentforum.org
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com