flâneur

[2508.16201] SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning

arxiv.org · 4,423 words · saved by 1 readers

Abstract:Video large language models (Vid-LLMs) have shown strong capabilities in understanding video content. However, their reliance on dense video token representations introduces substantial memory and computational overhead in both prefilling and decoding. To mitigate the information loss of recent video token reduction methods and accelerate the decoding stage of Vid-LLMs losslessly, we introduce SpecVLM, a training-free speculative decoding (SD) framework tailored for Vid-LLMs that incorporates staged video token pruning. Building on our novel finding that the draft model's speculation exhibits low sensitivity to video token pruning, SpecVLM prunes up to 90% of video tokens to enable efficient speculation without sacrificing accuracy. To achieve this, we performs a two-stage pruning process: Stage I selects highly informative tokens guided by attention signals from the verifier (target model), while Stage II prunes remaining redundant ones in a spatially uniform manner. Extensive experiments on four video understanding benchmarks demonstrate the effectiveness and robustness of SpecVLM, which achieves up to 2.68$\times$ decoding speedup for LLaVA-OneVision-72B and 2.11$\times$ speedup for Qwen2.5-VL-32B. Code is available at this https URL.

S PEC VLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning Yicheng Ji1,2✌ , Jun Zhang1,2✌ , Heming Xia3 , Jinpeng Chen4 , Lidan Shou1,2 , Gang Chen1 , Huan Li1,2✉ 1 The State Key Laboratory of Blockchain and Data Security, Zhejiang University…

saved by

related reading