A mechanism for solving relational tasks in transformer language models
arxiv.org · 4,263 words · saved by 1 readers
N/A
Language Models Implement Simple Word2Vec-style Vector Arithmetic Jack Merullo Carsten Eickhoff Department of Computer Science School of Medicine Brown University University of Tübingen jack_merullo@brown.edu carsten.eickhoff@uni-tuebingen.de…
related reading
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- Transformer Circuits Threadtransformer-circuits.pub
- How LLMs Actually Work | 0xkato0xkato.xyz
- In-context learning creates task vectorsarxiv.org
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
- Recursive Language Models | Alex L. Zhangalexzhang13.github.io
- Linearity of Relation Decoding in Transformer Language Modelslre.baulab.info
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- Can LLMs Be Computers?percepta.ai
- A History of Large Language Modelsgregorygundersen.com
- Generalized Language Models | Lil'Loglilianweng.github.io