flâneur

Differentiable Robot Rendering

drrobot.cs.columbia.edu · 362 words · saved by 2 readers

We introduce a framework where robots learn in real-world settings to create and utilize paper-based tools for practical tasks.

1Columbia University, 2Stanford University, *Equal Contribution. Abstract Vision foundation models trained on massive amounts of visual data have shown unprecedented reasoning and planning skills in open-world settings. A key challenge in applying them to robotic tasks is the modality gap between visual data and action data. We introduce differentiable robot rendering, a method allowing the visual appearance of a robot body to be directly differentiable with respect to its control parameters. Our model integrates a kinematics-aware deformable model and Gaussians Splatting and is compatible…

saved by

related reading