flâneur

Imagining better interfaces to language models | thesephist.com

thesephist.com · 2,591 words · saved by 1 readers

Suppose you’re a product engineer working on an app that needs to understand natural language. Maybe you’re trying to understand human-language questions and provide answers, or maybe you want to understand what humans are talking about on social media, to group and categorize them for easier browsing. Today, there is no shortage of tools you may reach for to solve this problem. But if you have a lot of money, a lot of compute hardware, and you’re feeling a little adventurous, you may find yourself reaching for the biggest hammer of all the NLP hammers: the large autoregressive language model, GPT-3 and friends. Autoregressive models let engineers take advantage of computers’ language understanding through a simple interface: it continues some given piece of text in a way the model predicts is the most likely. If you give it the start of a Wikipedia entry, it will write a convincingly thorough Wikipedia article; if you give it the start of a conversation log between friends or a forum

Suppose you're a product engineer working on an app that needs to understand natural language. Maybe you're trying to understand human-language questions and provide answers, or maybe you want to understand what humans are talking about on social media, to group and categorize them for easier browsing. Today, there is no shortage of tools you may reach for to solve this problem. But if you have a lot of money, a lot of compute hardware, and you're feeling a little adventurous, you may find yourself reaching for the biggest hammer of all the NLP hammers: the large autoregressive language model,

related reading