2407.15891

N/A

R AZOR ATTENTION : E FFICIENT KV C ACHE C OMPRESSION T HROUGH R ETRIEVAL H EADS A P REPRINT Hanlin Tang* 1 , Yang Lin1 , Jing Lin1 , Qingsen Han1 , Shikuan Hong1 , Yiwu Yao1 , and Gongyi Wang1 1…

related reading

• 2502.11089arxiv.org
• [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• [2412.02252] Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarityarxiv.org
• 1706.03762arxiv.org
• [2412.14838] DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMsarxiv.org
• Cross-Model KV Cache Transfer in LLM Families:A Closed-Form Linear Mapping for Prefill Reusearxiv.org
• 2306.14048arxiv.org
• Kimi Linear: An Expressive, Efficient Attention Architecturearxiv.org
• [2604.20920] Simplified Sparse Attention via Gist Tokensarxiv.org
• Overleaf Examplearxiv.org
• [2603.23516] MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokensarxiv.org