How can LLMs help with value-guided decision making? — AI • Objectives • Institute
AOI is excited to be presenting at a NeurIPS workshop focused on morality in human psychology and AI! Our recent work draws on human moral cognition and the latest research on moral representation in large language models (LLMs). For this project, we asked, how could we use LLMs to represent value for real-world action decision making? Read the full paper here Imagine your robotic AI assistant is shopping for you in a grocery store and its goal is to buy your favorite snacks. Why shouldn’t it steal all the snacks, pushing people out of the way as it gets to them? Humans seek not just to maximize a selfish reward, but also to cooperate with others, have a positive impact on the world, and follow societal norms. Aligned AI must likewise take into account not just how much we care about snacks, but also honesty, cooperation, and (not) stealing. Modern AI agents largely follow the highly successful reinforcement learning framework to make decisions for how to act. They choose the action wi
How can LLMs help with value-guided decision making? Dec 11 Written By Colleen McKenzie Image generated with DALL-E Anna Leshinskaya AOI is excited to be presenting at a NeurIPS workshop focused on morality in human psychology and AI! Our recent work draws on human moral cognition and the latest research on moral representation in large language models (LLMs). For this project, we asked, how could we use LLMs to represent value for real-world action decision making? Read the full paper here Imagine your robotic AI assistant is shopping for you in a grocery store and its goal is to buy your fav
related reading
- The behavioral selection model for predicting AI motivations — LessWronglesswrong.com
- Alignment Is Proven To Be Solvable - by SE Gygesverysane.ai
- Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
- 2404.10636arxiv.org
- Do large language models understand us? | by Blaise Aguera y Arcas | Mediummedium.com
- Where Do LLM Values Come From? — LessWronglesswrong.com
- Auditing language models for hidden objectives — LessWronglesswrong.com
- Full-Stack Alignment and Thick Models of Valuefull-stack-alignment.ai
- What Does It Mean to Align AI With Human Values? | Quanta Magazinequantamagazine.org
- [2605.10310] Positive Alignment: Artificial Intelligence for Human Flourishingarxiv.org
- [2607.14345] Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Valuesarxiv.org
- 2311.08576arxiv.org