https://aclanthology.org/P19-1285.pdf

N/A

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai⇤12 , Zhilin Yang⇤12 , Yiming Yang1 , Jaime Carbonell1 , Quoc V. Le2 , Ruslan Salakhutdinov1 1 Carnegie Mellon University, 2 Google Brain {dzihang,zhiliny,yiming,jgc,rsalakhu}@cs.cmu.edu, qvl@google.com Abstract Term Memory (LSTM) networks (Hochreiter and…

saved by

• Fernando Silva

related reading

• transformer_attention.pdfarxiv.org
• 1706.03762arxiv.org
• NL.pdfabehrouz.github.io
• [2512.23675] End-to-End Test-Time Training for Long Contextarxiv.org
• [2108.12409] Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationarxiv.org
• 1810.04805arxiv.org
• A History of Large Language Modelsgregorygundersen.com
• [2501.00663] Titans: Learning to Memorize at Test Timearxiv.org
• [1807.03819] Universal Transformersarxiv.org
• Attention is All you Needproceedings.neurips.cc
• Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentionarxiv.org
• Self-Adapting Language Modelsarxiv.org