A mechanism for solving relational tasks in transformer language models
arxiv.org · 4,263 words · saved by 1 readers
N/A
Language Models Implement Simple Word2Vec-style Vector Arithmetic Jack Merullo Carsten Eickhoff Department of Computer Science School of Medicine Brown University University of Tübingen jack_merullo@brown.edu carsten.eickhoff@uni-tuebingen.de…
related reading
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- transformer_attention.pdfarxiv.org
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
- In-context Learning and Induction Headstransformer-circuits.pub
- Recursive Language Models | Alex L. Zhangalexzhang13.github.io
- In-context learning creates task vectorsarxiv.org
- Language Models can Solve Computer Tasksarxiv.org
- [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
- A History of Large Language Modelsgregorygundersen.com
- [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
- [2005.14165] Language Models are Few-Shot Learnersarxiv.org
- Language Model Compositionarxiv.org