Language models can only write poetry

But only a person can write a poem

Language models can only write poetry This is a revised and condensed version of “(Notes toward a theory of) Language models and poetics,” delivered as the Vector Institute ARTificial Intelligence Distinguished Lecture in August 2020 . Cold bones In 2019, Gwern Branwen—a well-known machine learning researcher—posted the results of several experiments in fine-tuning a large pre-trained language model on a corpus of poetry. The language model in question was OpenAI’s GPT-2. Here are some of the results: There comes a murmur low and sweet As of far-off streams in a dream, Or a murmur of many bird

related reading

• On the Biology of a Large Language Modeltransformer-circuits.pub
• The Human Skill That Eludes AI - The Atlantictheatlantic.com
• LLM poetry and the "greatness" question - by Hollis Robbinshollisrobbinsanecdotal.substack.com
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• ⚡️ Robots Write Experimental Poetryreboothq.substack.com
• What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
• Why poetry is a variety of mathematical experienceaeon.co
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• trees are harlequins, words are harlequins - the voidnostalgebraist.tumblr.com
• Large Language Model: world models or surface statistics?thegradient.pub
• ⚡️ Robots Write Experimental Poetryjoinreboot.org
• The Computational Poetdistressedscientists.substack.com