Semantic World Models

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions.

Semantic World Models Jacob Berg 1 Chuning Zhu 1 Yanda Bao 1 Ishan Durugkar 2 Abhishek Gupta 1 1 University of Washington 2 Sony AI jacob33,zchuning,yandabao,abhgupta@cs.washington.edu ishan.durugkar@sony.com Abstract Planning with world models offers a powerful paradigm for robotic control. Conventional approaches train a model to predict future frames conditioned on current frames and actions, which can then be used for planning. However, the objective of predicting future pixels is often at odds with the actual planning objective; strong pixel reconstruction does not always correlate with g

related reading

• Semantic World Modelsweirdlabuw.github.io
• Explore | alphaXivalphaxiv.org
• World Models: Computing the Uncomputablenotboring.co
• [2509.02722] Planning with Reasoning using Vision Language World Modelarxiv.org
• World Models | Rohit Bandarurohitbandaru.github.io
• pdfopenreview.net
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelsle-wm.github.io
• A Functional Taxonomy of World Models - Dr. Fei-Fei Lidrfeifei.substack.com
• Explore | alphaXivalphaxiv.org
• Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blogdeveloper.nvidia.com
• [2603.05438] Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Modelarxiv.org