✳flâneur — a map of the web's best reading

A General Goal-Conditioned Minecraft Model - Pantograph

pantograph.com · 4,429 words · saved by 1 readers

We develop a simple method for learning goal-directed behavior from pretraining on internet-scale video, and use it to train a model capable of achieving diverse and out-of-distribution goals in Minecraft. These goals are first seen at inference time, without training on any of them specifically.

At Pantograph, we're working on training fully general robotics models that can act autonomously for hours at a time. Especially in robotics, it's difficult to get diverse data at scale. Learning to act from internet video data could allow models to scale with compute, rather than being limited by small action datasets. In this work, we develop a simple method for learning goal-directed behavior through pretraining on internet-scale video. Usually, goal-directedness is taught in a post-training phase, which limits the extent to which it can generalize. Here, we learn goal-directedness during p

Explore this link on the map →

related reading