Star Attention: Efficient LLM Inference over Long Sequences
arxiv.org · 4,505 words · saved by 1 readers
N/A
Star Attention: Efficient LLM Inference over Long Sequences Shantanu Acharya 1 Fei Jia 1 Boris Ginsburg 1 Abstract these methods improve training efficiency, autoregressive decoding during inference still requires the model to attend Inference with…
related reading
- transformer_attention.pdfarxiv.org
- 2502.11089arxiv.org
- [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessarxiv.org
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- [2412.02252] Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarityarxiv.org
- [2603.23516] MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokensarxiv.org
- LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelismarxiv.org
- [2604.20920] Simplified Sparse Attention via Gist Tokensarxiv.org
- [2512.23675] End-to-End Test-Time Training for Long Contextarxiv.org
- Context Parallelism for Scalable Million-Token Inferencearxiv.org
- Sparser Block-Sparse Attention via Token Permutationarxiv.org
- [2507.04239] Scaling Context Requires Rethinking Attentionarxiv.org