[2206.07682] Emergent Abilities of Large Language Models

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.

Published in Transactions on Machine Learning Research (08/2022) Emergent Abilities of Large Language Models Jason Wei 1 jasonwei@google.com Yi Tay 1 yitay@google.com Rishi Bommasani 2…

related reading

• Are Emergent Abilities in Large Language Models just In-Context Learning?arxiv.org
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• Your Evals Will Break and You Won't See It Coming - Lun Wangwanglun1996.github.io
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• [2210.13382] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskarxiv.org
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• [2309.01809] Are Emergent Abilities in Large Language Models just In-Context Learning?arxiv.org
• [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
• Self-Adapting Language Modelsarxiv.org
• Language Models can Solve Computer Tasksarxiv.org
• [2210.11416] Scaling Instruction-Finetuned Language Modelsarxiv.org