2410.06205
arxiv.org · 8,427 words · saved by 1 readers
N/A
Published as a conference paper at ICLR 2025 ROUND AND ROUND W E G O ! L W HAT MAKES ROTARY P OSITIONAL E NCODINGS USEFUL ? Federico Barbero˚ Alex Vitvitskyi Christos Perivolaropoulos University of Oxford Google DeepMind Google DeepMind Razvan Pascanu Petar…
related reading
- [2410.06205] Round and Round We Go! What makes Rotary Positional Encodings useful?arxiv.org
- Jane Street Blog - Using group theory to explore the space of positional encodings for attentionblog.janestreet.com
- On N-dimensional Rotary Positional Embeddingsjerryxio.ng
- 1706.03762arxiv.org
- [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
- 2409.02908arxiv.org
- [2507.19457] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningarxiv.org
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- Extending the Context of Pretrained LLMs by Dropping their Positional Embeddingspub.sakana.ai
- 2310.01405arxiv.org
- 2502.11089arxiv.org
- 2310.01405.pdfarxiv.org