Learning Novel Skills from Language-Generated Demonstrations

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions.

Learning Novel Skills from Language-Generated Demonstrations Ao-Qun Jin, Tian-Yu Xiang Institute of Automation Chinese Academy of Sciences Beijing 100190, China &Xiao-Hu Zhou Institute of Automation Chinese Academy of Sciences Beijing 100190, China \AND Mei-Jiang Gui, Xiao-Liang Xie, Shi-Qi Liu, Shuang-Yi Wang Institute of Automation Chinese Academy of Sciences Beijing 100190, China \AND Yue Cao, Sheng-Bin Duan, Fu-Chao Xie, Zeng-Guang Hou Institute of Automation Chinese Academy of Sciences Beijing 100190, China Ao-Qun Jin is also with the School of Computer and Information Engineering, Hubei

related reading

• Emergence of Human to Robot Transfer in Vision-Language-Action Modelspi.website
• Continual Skill and Task Learning via Dialoguearxiv.org
• Language Models can Solve Computer Tasksarxiv.org
• [2109.01115] Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationarxiv.org
• Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Trainingalphaxiv.org
• RT-2: Vision-Language-Action Modelsrobotics-transformer2.github.io
• Open X-Embodiment: Robotic Learning Datasets and RT-X Modelsarxiv.org
• Video models are zero-shot learners and reasonersarxiv.org
• Reinforcement Learning via Implicit Imitation Guidancearxiv.org
• A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulationtoyotaresearchinstitute.github.io
• Reward Isn’t Free: Supervising Robot Learning with Language and Video from the Web | SAIL Blogai.stanford.edu
• [2206.11795] Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videosarxiv.org