Word Embeddings

The way machine learning models "see" data is different from how we (humans) do. For example, we can easily understand the text "I saw a cat", but our models can not - they need vectors of features. Such vectors, or word embeddings, are representations of words which can be fed into your model. In practice, you have a vocabulary of allowed words; you choose this vocabulary in advance. For each vocabulary word, a look-up table contains its embedding. This embedding can be found using the word index in the vocabulary (i.e., you to look up the embedding in the table using word index). To account for unknown words (the ones which are not in the vocabulary), usually a vocabulary contains a special token UNK. Alternatively, unknown tokens can be ignored or assigned a zero vector. The easiest you can do is to represent words as one-hot vectors: for the i-th word in the vocabulary, the vector has 1 on the i-th dimension and 0 on the rest. In Machine Learning, this is the most simple way to rep

Word Embeddings p { text-align: justify; } ⇤ NLP Course | For You Word Embeddings One-Hot Vectors Distributional Semantics Count-Based Methods Word2Vec • Idea • Objective Function • Training Procedure • Negative Sampling • Skip-Gram vs CBOW • Additional Notes GloVe Evaluation Analysis and Interpretability Research Thinking --> Related Papers --> Have Fun! ☰ --> --> Word Embeddings The way machine learning models " see " data is different from how we (humans) do. For example, we can easily understand the text "I saw a cat" , but our models can not - they need

saved by

• Fernando Silva

related reading

• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Transfer Learninglena-voita.github.io
• SenseBERT: Driving Some Sense into BERT - ACL Anthologyaclanthology.org
• Language Modelinglena-voita.github.io
• Glossary of Deep Learning: Word Embedding | by Jaron Collis | Deeper Learning | Mediummedium.com
• Comparison of different Word Embeddings on Text Similarity — A use case in NLP | by Intellica.AI | Mediumintellica-ai.medium.com
• 讓電腦聽懂人話: 直觀理解 Word2Vec 模型. Word2Vec 是 Google 於 2013 年由 Tomas… | by TengYuan Chang | Mediumtengyuanchang.medium.com
• An intuitive introduction to text embeddings - Stack Overflowstackoverflow.blog
• An intuitive introduction to text embeddings - Stack Overflowstackoverflow.blog
• Sentence Embeddings. Introduction to Sentence Embeddings – hackerllamaosanseviero.github.io
• [2602.15029] Symmetry in language statistics shapes the geometry of model representationsarxiv.org