[2204.05186] Correcting Robot Plans with Natural Language Feedback

When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, under-specified, or beyond planners’ ability to solve. In these cases, corrections provide a valuable too…

Correcting Robot Plans with Natural Language Feedback Pratyusha Sharma34, Balakumar Sundaralingam3, Valts Blukis3, Chris Paxton3, Tucker Hermans35, Antonio Torralba4, Jacob Andreas 4, Dieter Fox32 3 NVIDIA, 4 MIT, 5 University of Utah, 2 University of Washington Abstract When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, under-specified, or beyond planners’ ability to solve. In these cases, corrections provide a valuable tool for human-in-the-loop robot control. Corrections might take the form of new goal specifications, new constra

related reading

• [2109.01115] Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationarxiv.org
• [2301.02555] “No, to the Right” – Online Language Corrections for Robotic Manipulation via Shared Autonomyar5iv.labs.arxiv.org
• [2310.17555] Interactive Robot Learning from Verbal Correctionar5iv.labs.arxiv.org
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• How Claude Performs on Robotics Tasks \ Anthropicanthropic.com
• Emergence of Human to Robot Transfer in Vision-Language-Action Modelspi.website
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org
• Updating Robot Safety Representations Online from Natural Language Feedback ∗ Equal Contribution. † Equal Advising. 1School of Engineering, Federal University of Minas Gerais, Brazil. Email: leohmcs@ufmg.br. Work done during Robotics Institarxiv.org
• A Unified Model for Motion-Conditioned Robot Co-designtransformer-transformer.github.io
• Yell At Your Robot \emojishout Improving On-the-Fly from Language Correctionsarxiv.org
• drocarxiv.org
• [2602.10556] LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transferarxiv.org