Self-supervised learning: The dark matter of intelligence
How can we build machines with human-level intelligence? There’s a limit to how far the field of AI can go with supervised learning alone. Here's why self-supervised learning is one of the most promising ways to make significant progress in AI.
In recent years, the AI field has made tremendous progress in developing AI systems that can learn from massive amounts of carefully labeled data. This paradigm of supervised learning has a proven track record for training specialist models that perform extremely well on the task they were trained to do. Unfortunately, there’s a limit to how far the field of AI can go with supervised learning alone. Supervised learning is a bottleneck for building more intelligent generalist models that can do multiple tasks and acquire new skills without massive amounts of labeled data. Practically…
saved by
related reading
- Learning with not Enough Data Part 1: Semi-Supervised Learning | Lil'Loglilianweng.github.io
- Self-supervised learning - Wikipediaen.wikipedia.org
- Yann LeCun: AI Doesn’t Need Our Supervisionspectrum.ieee.org
- Automated Weak-to-Strong Researcheralignment.anthropic.com
- Just Ask for Generalization | Eric Jangevjang.com
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Ilya Sutskever — We're moving from the age of scaling to the age of researchdwarkesh.com
- pdfopenreview.net
- It Looks Like You’re Trying To Take Over The World · Gwern.net (reader mode)gwern.net
- Big Self-Supervised Models are Strong Semi-Supervised Learnersarxiv.org
- [2103.03230] Barlow Twins: Self-Supervised Learning via Redundancy Reductionarxiv.org
- [2506.09985] V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningarxiv.org