[2404.01476] TraveLER: A Multi-LMM Agent Framework for Video Question-Answering
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack
View PDF HTML (experimental) Abstract:Recently, image-based Large Multimodal Models (LMMs) have made significant progress in video question-answering (VideoQA) using a frame-wise approach by leveraging large-scale pretraining in a zero-shot manner. Nevertheless, these models need to be capable of finding relevant information, extracting it, and answering the question simultaneously. Currently, existing methods perform all of these steps in a single pass without being able to adapt if insufficient or incorrect information is collected. To overcome this, we introduce a modular multi-LMM agent…
saved by
related reading
- Training Strategies for Efficient Embodied Reasoningarxiv.org
- Explore | alphaXivalphaxiv.org
- Voyager | An Open-Ended Embodied Agent with Large Language Modelsvoyager.minedojo.org
- [2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingarxiv.org
- Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Modelsarxiv.org
- Video models are zero-shot learners and reasonersarxiv.org
- LLM-as-a-Verifier: A General-Purpose Verification Framework | alphaXivalphaxiv.org
- [2310.08586] PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmarxiv.org
- Attend Before Attention: Efficient and Scalable Video Understanding via Autoregressive Gazingarxiv.org
- You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectoriesarxiv.org
- GitHub - TIGER-AI-Lab/TheoremExplainAgent: Official Repo for "TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understanding" [ACL 2025 oral]github.com
- LLM Post-Training: A Deep Dive into Reasoning Large Language Modelsarxiv.org