flâneur

[2602.10098] VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model

arxiv.org · 6,359 words · saved by 1 readers

Abstract:Pretraining Vision-Language-Action (VLA) policies on internet-scale video is appealing, yet current latent-action objectives often learn the wrong thing: they remain anchored to pixel variation rather than action-relevant state transitions, making them vulnerable to appearance bias, nuisance motion, and information leakage. We introduce VLA-JEPA, a JEPA-style pretraining framework that sidesteps these pitfalls by design. The key idea is leakage-free state prediction: a target encoder produces latent representations from future frames, while the student pathway sees only the current observation -- future information is used solely as supervision targets, never as input. By predicting in latent space rather than pixel space, VLA-JEPA learns dynamics abstractions that are robust to camera motion and irrelevant background changes. This yields a simple two-stage recipe -- JEPA pretraining followed by action-head fine-tuning -- without the multi-stage complexity of prior latent-action pipelines. Experiments on LIBERO, LIBERO-Plus, SimplerEnv and real-world manipulation tasks show that VLA-JEPA achieves consistent gains in generalization and robustness over existing methods.

VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model Jingwen Sun1,2,∗ , Wenyao Zhang3,5∗ , Zekun Qi4 , Shaojie Ren2,6 , Zezhi Liu2,7 , Hanxin Zhu1 , Guangzhong Sun1 , Xin Jin2,5,† , Zhibo Chen1,2,† 1 University of Science and Technology of China 2 Zhongguancun Academy, Beijing, China 3 Shanghai Jiao Tong University 4 Tsinghua…

saved by

related reading