An Introduction to N-grams: What Are They and Why Do We Need Them? - XRDSXRDS

In this post I am going to talk about N-grams, a concept found in Natural Language Processing ( aka NLP). First of all, let’s see what the term ‘N-gram’ means. Turns out that is the simplest bit, an N-gram is simply … Continue reading →

In this post I am going to talk about N-grams, a concept found in Natural Language Processing ( aka NLP). First of all, let’s see what the term ‘N-gram’ means. Turns out that is the simplest bit, an N-gram is simply a sequence of N words. For instance, let us take a look at the following examples. San Francisco (is a 2-gram) The Three Musketeers (is a 3-gram) She stood up slowly (is a 4-gram) Now which of these three N-grams have you seen quite frequently? Probably, “San Francisco” and “The Three Musketeers”. On the other hand, you might not have seen “She stood up slowly” that frequently. Bas

saved by

• Kevin Lee

related reading

• Introduction to Large Language Models | Machine Learning | Google for Developersdevelopers.google.com
• Language Modelinglena-voita.github.io
• Home | infini-graminfini-gram.io
• What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Language Modeling Without Neural Networksnathan.rs
• [2001.04063] ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingarxiv.org
• Can gzip be a language model?nathan.rs
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• 讓電腦聽懂人話: 直觀理解 Word2Vec 模型. Word2Vec 是 Google 於 2013 年由 Tomas… | by TengYuan Chang | Mediumtengyuanchang.medium.com
• Large Language Models explained briefly | 3Blue1Brown3blue1brown.com
• Bag-of-words model - Wikipediaen.wikipedia.org