✳flâneur — a map of the web's best reading

Behavior From the Void: Unsupervised Active Pre-Training | HTML5

ar5iv.labs.arxiv.org · 13,078 words · saved by 1 readers

We introduce a new unsupervised pre-training method for reinforcement learning called APT, which stands for Active Pre-Training. APT learns behaviors and representations by actively searching for novel states in reward-free environments. The key novel idea is to explore the environment by maximizing a non-parametric entropy computed in an abstract representation space, which avoids challenging density modeling and consequently allows our approach to scale much better in environments that have high-dimensional observations (e.g., image observations). We empirically evaluate APT by exposing task-specific reward after a long unsupervised pre-training phase. In Atari games, APT achieves human-level performance on 12 games and obtains highly competitive performance compared to canonical fully supervised RL algorithms. On DMControl suite, APT beats all baselines in terms of asymptotic performance and data efficiency and dramatically improves performance on tasks that are extremely difficult

Behavior From the Void: Unsupervised Active Pre-Training Hao Liu UC Berkeley hao.liu@cs.berkeley.edu &Pieter Abbeel UC Berkeley pabbeel@cs.berkeley.edu Abstract We introduce a new unsupervised pre-training method for reinforcement learning called APT, which stands for Active Pre-Training. APT learns behaviors and representations by actively searching for novel states in reward-free environments. The key novel idea is to explore the environment by maximizing a non-parametric entropy computed in an abstract representation space, which avoids challenging density modeling and consequently allows o

Explore this link on the map →

related reading