In-context learning creates task vectors
arxiv.org · 6,018 words · saved by 1 readers
N/A
In-Context Learning Creates Task Vectors Roee Hendel Mor Geva Amir Globerson Tel Aviv University Google DeepMind Tel Aviv University, Google roee.hendel@mail.tau.ac.il pipek@google.com gamir@tauex.tau.ac.il Abstract Large Language Model…
related reading
- In-context Learning and Induction Headstransformer-circuits.pub
- How does in-context learning work? A framework for understanding the differences from traditional supervised learning | SAIL Blogai.stanford.edu
- Transformer Circuits Threadtransformer-circuits.pub
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?arxiv.org
- Learning without training: The implicit dynamics of in-context learningarxiv.org
- What Can Transformers Learn In-Context?arxiv.org
- [2311.06668] In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringarxiv.org
- A mechanism for solving relational tasks in transformer language modelsarxiv.org
- [2212.07677] Transformers learn in-context by gradient descentarxiv.org
- [2301.00234] A Survey on In-context Learningarxiv.org
- GenAI Handbookgenai-handbook.github.io
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io