Introduction to Large Language Models  |  Machine Learning  |  Google for Developers

A language model is a machine learning model that aims to predict and generate plausible language. Autocomplete is a language model, for example. These models work by estimating the probability of a token or sequence of tokens occurring within a longer sequence of tokens. Consider the following sentence: If you assume that a token is a word, then a language model determines the probabilities of different words or sequences of words to replace that underscore. For example, a language model might determine the following probabilities: A "sequence of tokens" could be an entire sentence or a series of sentences. That is, a language model could calculate the likelihood of different entire sentences or blocks of text. Estimating the probability of what comes next in a sequence is useful for all kinds of things: generating text, translating languages, and answering questions, to name a few. Modeling human language at scale is a highly complex and resource-intensive endeavor. The path to reach

Home Products Machine Learning ML Concepts Crash Course Send feedback Introduction to Large Language Models Stay organized with collections Save and categorize content based on your preferences. Page Summary outlined_flag This module explores language models, which estimate the probability of a token or sequence of tokens occurring within a longer sequence, enabling tasks like text generation, translation, and summarization. Language models utilize context, the surrounding information of a target token, to enhance prediction accuracy, with recurrent neural networks offering more context than t

saved by

• Linda
• Nguyen Hoang Thao

related reading

• Language Modelinglena-voita.github.io
• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
• A History of Large Language Modelsgregorygundersen.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Large Language Model: world models or surface statistics?thegradient.pub
• Language Modeling Without Neural Networksnathan.rs
• Large Language Models explained briefly | 3Blue1Brown3blue1brown.com
• Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• GitHub - brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4.github.com
• How LLMs Actually Work | 0xkato0xkato.xyz
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub