flâneur

physics-IQ-benchmark/README.md at main · google-deepmind/physics-IQ-benchmark

github.com · 3,341 words · saved by 1 readers

Physics-IQ is a high-quality, realistic, and comprehensive benchmark dataset for evaluating physical understanding in generative video models. Project website: physics-iq.github.io The best possible score on Physics-IQ is 100.0%, this score would be achieved by physically realistic videos that differ only in physical randomness but adhere to all tested principles of physics. If you test your model on Physics-IQ and would like your score/paper/model to be featured here in this table, feel free to open a pull request that adds a row to the table and we'll be happy to include it! Note to early adopters of the benchmark: results from the paper were finalized on February 19, 2025; if you used the toolbox before please re-run since we changed and improved a few aspects. Likewise, if you downloaded the dataset before that date, it is recommended to re-download it, ensuring the ground truth video masks have a duration of five seconds. Visit the Google Cloud Storage link to download the dataset

Leaderboard | Quick Start | Physics-IQ Verified Workflow | Citation | License Physics-IQ and Physics-IQ Verified: Benchmarking physical understanding in generative video models Physics-IQ is a high-quality, realistic, and comprehensive benchmark dataset for evaluating physical understanding in generative video models. Building on this foundation, Physics-IQ Verified contains improvements w.r.t. prompt and metric quality. This repository contains the workflow for both Physics-IQ Verified (recommended benchmark variant) and the original Physics-IQ benchmark. Original Physics-IQ website:…

saved by

related reading