Untangling the Moons: A Visual History of Contrastive Learning

Eight contrastive losses, twenty years of history, one interactive playground. Watch pair, triplet, InfoNCE, CLIP, SupCon, SigLIP, alignment+uniformity, and cosine→0 organize 2D points — and see which ones know when to stop.

⌨ Runnable JAX companion Organizing Randomness: Contrastive Learning in JAX Prefer to read the code? This post has a hands-on JAX / Flax NNX implementation. Open the JAX companion → Twenty years of contrastive learning, eight losses, eight datasets, and the question of when a loss should stop pushing. Contrastive learning is the standard recipe for turning raw data into a usable embedding space, and the recipe has been rewritten roughly every three years. As a framing device, and it is a framing device, the real history is messier, each one reads as a response to the previous one’s failure mod

saved by

• Sean Lee

related reading

• Contrastive Representation Learning | Lil'Loglilianweng.github.io
• Understanding self-supervised and contrastive learning with "Bootstrap Your Own Latent" (BYOL) - Imbueimbue.com
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• [2002.05709] A Simple Framework for Contrastive Learning of Visual Representationsarxiv.org
• The Platonic Representation Hypothesisphillipi.github.io
• Feature Visualizationdistill.pub
• State of RL for reasoning LLMs | A. Weersaweers.de
• Self-Supervised Representation Learning | Lil'Loglilianweng.github.io
• arxiv.org/pdf/2511.08544arxiv.org
• CS231n Deep Learning for Computer Visioncs231n.github.io
• Understanding Memorization via Loss Curvaturegoodfire.ai
• Loss Landscape | A.I deep learning explorations of morphology & dynamicslosslandscape.com