[2603.18771] Empathetic Motion Generation for Humanoid Educational Robots via Reasoning-Guided Vision--Language--Motion Diffusion Architecture
Abstract:This article suggests a reasoning-guided vision-language-motion diffusion framework (RG-VLMD) for generating instruction-aware co-speech gestures for humanoid robots in educational scenarios. The system integrates multi-modal affective estimation, pedagogical reasoning, and teaching-act-conditioned motion synthesis to enable adaptive and semantically consistent robot behavior. A gated mixture-of-experts model predicts Valence/Arousal from input text, visual, and acoustic features, which then mapped to discrete teaching-act categories through an affect-driven this http URL signals condition a diffusion-based motion generator using clip-level intent and frame-level instructional schedules via additive latent restriction with auxiliary action-group supervision. Compared to a baseline diffusion model, our proposed method produces more structured and distinctive motion patterns, as verified by motion statics and pairwise distance analysis. Generated motion sequences remain physically plausible and can be retargeted to a NAO robot for real-time execution. The results reveal that reasoning-guided instructional conditioning improves gesture controllability and pedagogical expressiveness in educational human-robot interaction.
View PDF HTML (experimental) Abstract:This article suggests a reasoning-guided vision-language-motion diffusion framework (RG-VLMD) for generating instruction-aware co-speech gestures for humanoid robots in educational scenarios. The system integrates multi-modal affective estimation, pedagogical reasoning, and teaching-act-conditioned motion synthesis to enable adaptive and semantically consistent robot behavior. A gated mixture-of-experts model predicts Valence/Arousal from input text, visual, and acoustic features, which then mapped to discrete teaching-act categories through an…
saved by
related reading
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusiondiffusion-policy.cs.columbia.edu
- Emergence of Human to Robot Transfer in Vision-Language-Action Modelspi.website
- Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Trainingalphaxiv.org
- A Unified Model for Motion-Conditioned Robot Co-designtransformer-transformer.github.io
- [2501.12493] ELEGNT: Expressive and Functional Movement Design for Non-anthropomorphic Robotarxiv.org
- BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusionarxiv.org
- Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blogdeveloper.nvidia.com
- Differentiable Robot Renderingdrrobot.cs.columbia.edu
- [2509.22407] EMMA: Generalizing Real-World Robot Manipulation via Generative Visual Transferarxiv.org
- [2109.01115] Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationarxiv.org
- Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensingarxiv.org
- RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackarxiv.org