flâneur

Behavioral cloning mystery

seohong.me · 2,815 words · saved by 5 readers

I've recently started working in robotics, where I've had the chance to train many behavioral cloning (BC) and reinforcement learning (RL) policies on real robot data. My impression so far is: real-world demonstration data is very different from the data in existing simulated RL benchmarks (e.g., D4RL, OGBench, etc.). In particular, I've noticed that there are many mysterious phenomena that appear with real robot data, which are not easily observed in many standard BC and RL benchmarks. These mysteries really confused me, so I decided to properly study them. Unfortunately, it's really difficult to investigate this in the real world, because nothing is fully reproducible in robotics. In the real world, results depend on everything, including lighting conditions, background, reset distribution, robot temperature, and so on. This means that even when you suspect that something weird is happening, you can't be 100% sure whether it's actually a thing or just noise. To do proper science, I b

I've recently started working in robotics, where I've had the chance to train many behavioral cloning (BC) and reinforcement learning (RL) policies on real robot data. My impression so far is: real-world demonstration data is very different from the data in existing simulated RL benchmarks (e.g., D4RL, OGBench, etc.). In particular, I've noticed that there are many mysterious phenomena that appear with real robot data, which are not easily observed in many standard BC and RL benchmarks. These mysteries really confused me, so I decided to properly study them. Unfortunately, it's really…

saved by

related reading