2306.14048

N/A

H2 O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang1 , Ying Sheng2 , Tianyi Zhou3 , Tianlong Chen1 , Lianmin Zheng4 , Ruisi Cai1 , Zhao Song5 , Yuandong Tian6 , Christopher Ré2 , Clark Barrett2 , Zhangyang Wang1 , Beidi Chen6,7 1 University of Texas at Austin, 2 Stanford University, 3…

related reading

• 2502.11089arxiv.org
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• 2409.02908arxiv.org
• [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessarxiv.org
• 2408.14690arxiv.org
• 2403.09611.pdfarxiv.org
• Large Language Diffusion Modelsarxiv.org
• [2402.12374] Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingarxiv.org
• 2305.14314arxiv.org
• arxiv.org/pdf/2505.24832arxiv.org