Language Models are Unsupervised Multitask Learners

N/A

Language Models are Unsupervised Multitask Learners Alec Radford * 1 Jeffrey Wu * 1 Rewon Child 1 David Luan 1 Dario Amodei ** 1 Ilya Sutskever ** 1 Abstract competent generalists. We would like to move towards more general systems which can perform many tasks – eventually Natural language processing tasks, such as ques- without the need to manually create and label a training tion answering, machine translation, reading com-…

related reading

• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• Generalized Language Models | Lil'Loglilianweng.github.io
• 2005.14165arxiv.org
• radford2018improving.pdfcs.ubc.ca
• Unsupervised Elicitation of Language Modelsarxiv.org
• To Understand Language is to Understand Generalization | Eric Jangevjang.com
• Recent Advances in Language Model Fine-tuningruder.io
• 2212.03827arxiv.org
• Alex L. Zhangalexzhang13.github.io
• 1910.10683arxiv.org
• Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrouai.googleblog.com
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org