Semantic World Models

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions.

Semantic World Models Jacob Berg 1 Chuning Zhu 1 Yanda Bao 1 Ishan Durugkar 2 Abhishek Gupta 1 1 University of Washington 2 Sony AI jacob33,zchuning,yandabao,abhgupta@cs.washington.edu ishan.durugkar@sony.com Abstract Planning with world models offers a powerful paradigm for robotic control. Conventional approaches train a model to predict future frames conditioned on current frames and actions, which can then be used for planning. However, the objective of predicting future pixels is often at odds with the actual planning objective; strong pixel reconstruction does not always correlate with g

related reading

• Semantic World Modelsweirdlabuw.github.io
• World Models | Rohit Bandarurohitbandaru.github.io
• Explore | alphaXivalphaxiv.org
• World Models: Computing the Uncomputablenotboring.co
• [2509.02722] Planning with Reasoning using Vision Language World Modelarxiv.org
• pdfopenreview.net
• A Functional Taxonomy of World Models - Dr. Fei-Fei Lidrfeifei.substack.com
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blogdeveloper.nvidia.com
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelsle-wm.github.io
• Explore | alphaXivalphaxiv.org
• CIS 6280 · World Modelscis.upenn.edu