[1703.08840] InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations

The goal of imitation learning is to mimic expert behavior without access to an explicit reward signal. Expert demonstrations provided by humans, however, often show significant variability due to latent factors that are typically not explicitly modeled. In this paper, we propose a new algorithm that can infer the latent structure of expert demonstrations in an unsupervised way. Our method, built on top of Generative Adversarial Imitation Learning, can not only imitate complex behaviors, but also learn interpretable and meaningful representations of complex behavioral data, including visual demonstrations. In the driving domain, we show that a model learned from human demonstrations is able to both accurately reproduce a variety of behaviors and accurately anticipate human actions using raw visual inputs. Compared with various baselines, our method can better capture the latent structure underlying expert demonstrations, often recovering semantically meaningful factors of variation in the data.

View PDF HTML (experimental) Abstract:The goal of imitation learning is to mimic expert behavior without access to an explicit reward signal. Expert demonstrations provided by humans, however, often show significant variability due to latent factors that are typically not explicitly modeled. In this paper, we propose a new algorithm that can infer the latent structure of expert demonstrations in an unsupervised way. Our method, built on top of Generative Adversarial Imitation Learning, can not only imitate complex behaviors, but also learn interpretable and meaningful representations of…

saved by

• Gilbert Chang

related reading

• Generative Adversarial Imitation Learningarxiv.org
• Reinforcement Learning via Implicit Imitation Guidancearxiv.org
• Ch. 21 - Imitation Learningunderactuated.mit.edu
• Imitating Latent Policies from Observationarxiv.org
• 1011.0686arxiv.org
• [2206.11795] Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videosarxiv.org
• Behavioral Cloning from Observationarxiv.org
• [2103.06326] S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningarxiv.org
• 2006.10742arxiv.org
• [2606.09758] Difference-Aware Retrieval Policies for Imitation Learningarxiv.org
• Generative Adversarial Imitation Learning.pdfcs.stanford.edu
• Learning to Imitate | SAIL Blogai.stanford.edu