Introduction to Large Language Models  |  Machine Learning  |  Google for Developers

A language model is a machine learning model that aims to predict and generate plausible language. Autocomplete is a language model, for example. These models work by estimating the probability of a token or sequence of tokens occurring within a longer sequence of tokens. Consider the following sentence: If you assume that a token is a word, then a language model determines the probabilities of different words or sequences of words to replace that underscore. For example, a language model might determine the following probabilities: A "sequence of tokens" could be an entire sentence or a series of sentences. That is, a language model could calculate the likelihood of different entire sentences or blocks of text. Estimating the probability of what comes next in a sequence is useful for all kinds of things: generating text, translating languages, and answering questions, to name a few. Modeling human language at scale is a highly complex and resource-intensive endeavor. The path to reach

Home Products Machine Learning ML Concepts Crash Course Send feedback Introduction to Large Language Models Stay organized with collections Save and categorize content based on your preferences. Page Summary outlined_flag This module explores language models, which estimate the probability of a token or sequence of tokens occurring within a longer sequence, enabling tasks like text generation, translation, and summarization. Language models utilize context, the surrounding information of a target token, to enhance prediction accuracy, with recurrent neural networks offering more context than t

saved by

• Linda

related reading

• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• Language Modelinglena-voita.github.io
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• A History of Large Language Modelsgregorygundersen.com
• Large Language Model: world models or surface statistics?thegradient.pub
• Can gzip be a language model?nathan.rs
• GenAI Handbookgenai-handbook.github.io
• Large Language Models explained briefly | 3Blue1Brown3blue1brown.com
• Generalized Language Models | Lil'Loglilianweng.github.io
• An Introduction to N-grams: What Are They and Why Do We Need Them? - XRDSXRDSblog.xrds.acm.org
• How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com