Dexterous Robotic Foundation Models - YouTube

Follow along using the transcript. Follow along using the transcript.

Presentation by Sergey Levine on dexterous robotic foundation models. In today's talk, I'm going to talk about dextrous robotic foundational models. So, the starting point for robotic foundational models is vision language action models or VLAs. VLAs are based off of multimodal language models or vision language models. So, the idea is that we start with a large language model that can already understand a lot of semantics, and then we further train it to be able to process images by embedding images into this into the same embedding space as language. And then, we further adapt it for…

saved by

• Hangyul Lyna Kim
• Gilbert Chang

related reading

• Generalist - GEN-0 / Embodied Foundation Models That Scale with Physical Interactiongeneralistai.com
• ACT-1: A Robot Foundation Model Trained on Zero Robot Data | Sunday Robotics | The helpful robotics companysunday.ai
• Emergence of Human to Robot Transfer in Vision-Language-Action Modelspi.website
• State of Robot Learning, December 2025vedder.io
• Precise Manipulation with Efficient Online RLpi.website
• Sporks of AGIsergeylevine.substack.com
• Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blogdeveloper.nvidia.com
• $π_0$: A Vision-Language-Action Flow Model for General Robot Controlalphaxiv.org
• Fully autonomous robots are much closer than you think – Sergey Levinedwarkesh.com
• How Claude Performs on Robotics Tasks \ Anthropicanthropic.com
• RT-2: Vision-Language-Action Modelsrobotics-transformer2.github.io
• Vision-Language-Action (VLA) Models: A Review of Recent Progressxxxxyu.github.io