Are Emergent Abilities in Large Language Models just In-Context Learning?

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on. Authors: achieve the best HTML results from your LaTeX submissions by following these best practices. Large language models, comprising billions of parameters and pre-trained on extensive web-scale corpora, have been claimed to acquire certain capabilities without having been specifically trained on them. These capabilities, referred to as “emergent abilities,” have been a driving force in discussions regarding the potentials and risks of language models. A key challenge in e

Are Emergent Abilities in Large Language Models just In-Context Learning? Sheng Lu 1* , Irina Bigoulaeva 1* , Rachneet Sachdeva 1 , Harish Tayyar Madabushi 2 Iryna Gurevych 1 1 Ubiquitous Knowledge Processing Lab, Technical University of Darmstadt 2 Department of Computer Science, The University of Bath www.ukp.tu-darmstadt.de htm43@bath.ac.uk Abstract Large language models, comprising billions of parameters and pre-trained on extensive web-scale corpora, have been claimed to acquire certain capabilities without having been specifically trained on them. These capabilities, referred to as “emer

saved by

• Al. J

related reading

• [2309.01809] Are Emergent Abilities in Large Language Models just In-Context Learning?arxiv.org
• In-context Learning and Induction Headstransformer-circuits.pub
• [2206.07682] Emergent Abilities of Large Language Modelsarxiv.org
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• Large Language Model: world models or surface statistics?thegradient.pub
• How does in-context learning work? A framework for understanding the differences from traditional supervised learning | SAIL Blogai.stanford.edu
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• Against LLM Reductionism — LessWronglesswrong.com
• [2502.19402] General Reasoning Requires Learning to Reason from the Get-goar5iv.labs.arxiv.org
• Model Organisms for Emergent Misalignmentarxiv.org
• Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs 49 This paper contains model-generated content that might be offensive. 49arxiv.org