3D-LFM: Lifting Foundation Model

3D pose estimation on random videos from the internet using a single model for all the following categories. We show 3D pose estimation on various deformable categories from the internet, including OpenAI's SORA videos. No camera information is required, allowing 3D-LFM to work out of the box on many of these categories, provided 2D landmarks are available (in any order or joint connectivity). A single model for 30+ object categories. The 3D-LFM scales to multiple categories (30+ in our experiments), managing diverse landmark configurations through proposed architectural changes. See our paper for more details. Key: Red for ground truth, blue for predictions. Fish is completely OOD! The 3D-LFM is capable in recognizing and reconstructing objects in configurations and with a number of landmarks never encountered during its training phase. Above object categories are never seen by the model in training. The lifting of 3D structure and camera from 2D landmarks is at the cornerstone of the

3D-LFM: Lifting Foundation Model 3D-LFM: Lifting Foundation Model Mosam Dabhi 1 , László A. Jeni 1 , Simon Lucey 2 1 Carnegie Mellon University, 2 The University of Adelaide CVPR, 2024 Paper --> arXiv Video Inference Press 🤗 Poster Tweet --> Chat with 3D-LFM GPT Tweet --> 3D-LFM is a universal 2D-3D lifting model capable of handling multiple object categories using a single model. It doesn't assume object knowledge and exploits the permutation equivariance of transformers to learn a category-agnostic lifting model. Moreover, it is an efficient model because it exploits geometric reasoning to

saved by

• Yu G

related reading

• Learning 3D object-centric representation through predictionarxiv.org
• Enhancing 3D Object Detection with 2D Detection-Guided Query Anchorsarxiv.org
• 3D-Aware Ellipse Prediction for Object-Based Camera Pose Estimationarxiv.org
• Amodal Detection of 3D Objects: Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Imagescis.temple.edu
• S4Darxiv.org
• Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboidsarxiv.org
• 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Videoncbi.nlm.nih.gov
• 2D Amodal Instance Segmentation Guided by 3D Shape Priorecva.net
• Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxesvirtualhumans.mpi-inf.mpg.de
• The First Fully General Computer Action Model | blogsi.inc
• Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wildlift4d.github.io
• [2207.13807] Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldsarxiv.org