Semantic Word Embeddings – Off the convex path

Algorithms off the convex path.

This post can be seen as an introduction to how nonconvex problems arise naturally in practice, and also the relative ease with which they are often solved. I will talk about word embeddings, a geometric way to capture the “meaning” of a word via a low-dimensional vector. They are useful in many tasks in Information Retrieval (IR) and Natural Language Processing (NLP), such as answering search queries or translating from one language to another. You may wonder: how can a 300-dimensional vector capture the many nuances of word meaning? And what the heck does it mean to “capture meaning?”…

saved by

• Didier Lampard

related reading

• Foundations of Vector Retrievalarxiv.org
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Word Embeddingslena-voita.github.io
• A History of Large Language Modelsgregorygundersen.com
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• 1301.3781arxiv.org
• Glossary of Deep Learning: Word Embedding | by Jaron Collis | Deeper Learning | Mediummedium.com
• Comparison of different Word Embeddings on Text Similarity — A use case in NLP | by Intellica.AI | Mediumintellica-ai.medium.com
• Embeddings: What they are and why they mattersimonwillison.net
• [2602.15029] Symmetry in language statistics shapes the geometry of model representationsarxiv.org
• Vector embeddings | OpenAI APIplatform.openai.com
• Vector embeddings | OpenAI APIdevelopers.openai.com