[2605.26383] Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion

Abstract:Object re-identification (ReID) in egocentric kitchen videos is challenging due to rapid viewpoint changes, frequent occlusions, cluttered scenes, and large intra-class appearance variations. Objects may leave and re-enter the field of view, and the large diversity of instances with limited annotations makes supervised ReID difficult to scale, motivating zero-shot approaches. We study zero-shot object ReID on the EPIC-Kitchens benchmark, where the goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features. We first evaluate five state-of-the-art feature extractors, including Vision-Language Models (VLMs) - CLIP, DINOv2, DreamSim, I-JEPA, and SAM3 - and show that zero-shot methods fail, with the best baseline achieving only 45.3% mAP. We then propose an Enhanced SAM3 ReID Pipeline, a zero-shot multi-stage method built around SAM3 segmentation as the core component. Stage 1 uses SAM3 to suppress background clutter. Stage 2 fuses embeddings from SAM3, DINOv2, and CLIP into a single L2-normalized descriptor. Stage 3 augments cosine similarity with mask-shape IoU for geometric consistency, and Stage 4 applies k-reciprocal re-ranking. The full pipeline improves performance by 7.5% mAP to 52.8%.

View PDF HTML (experimental) Abstract:Object re-identification (ReID) in egocentric kitchen videos is challenging due to rapid viewpoint changes, frequent occlusions, cluttered scenes, and large intra-class appearance variations. Objects may leave and re-enter the field of view, and the large diversity of instances with limited annotations makes supervised ReID difficult to scale, motivating zero-shot approaches. We study zero-shot object ReID on the EPIC-Kitchens benchmark, where the goal is to match active food and kitchen-tool instances across frames using only pre-trained visual…

saved by

• John Zhang

related reading

• Less Detail, Better Answers: Degradation-Driven Prompting for VQAarxiv.org
• Claim-Level Rubric Rewards for Video Caption Reinforcement Learningarxiv.org
• [2506.09985] V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningarxiv.org
• Video models are zero-shot learners and reasonersarxiv.org
• [2304.06718] Segment Everything Everywhere All at Oncearxiv.org
• [2603.14482] V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningarxiv.org
• 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Videoncbi.nlm.nih.gov
• Language-Grounded Indoor 3D Semantic Segmentation in the Wildarxiv.org
• Learning 3D object-centric representation through predictionarxiv.org
• DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation | PDFarxiv.org
• Amodal Detection of 3D Objects: Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Imagescis.temple.edu
• [2205.13147] Matryoshka Representation Learningarxiv.org