flâneur

Cross-Model KV Cache Transfer in LLM Families:A Closed-Form Linear Mapping for Prefill Reuse

arxiv.org · 8,155 words · saved by 1 readers

Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source’s KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B→32B, one source layer explains 56% of variance in the target’s keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top-k most predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on

Rasoul Shafipour Ritchie Zhao Maximilian Golub Affiliation: Mohammad Mahdi Kamani Ritika Borkar Makesh Tarun Chandran Affiliation: Pantea Zardoshti Bita Darvish Rouhani Affiliation: NVIDIA Abstract Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source’s KV cache, skipping prefill. We find that cross-model KV has substantial linear structure…

saved by

related reading