Self-supervised learning: The dark matter of intelligence
How can we build machines with human-level intelligence? There’s a limit to how far the field of AI can go with supervised learning alone. Here's why self-supervised learning is one of the most promising ways to make significant progress in AI.
In recent years, the AI field has made tremendous progress in developing AI systems that can learn from massive amounts of carefully labeled data. This paradigm of supervised learning has a proven track record for training specialist models that perform extremely well on the task they were trained to do. Unfortunately, there’s a limit to how far the field of AI can go with supervised learning alone. Supervised learning is a bottleneck for building more intelligent generalist models that can do multiple tasks and acquire new skills without massive amounts of labeled data. Practically…
saved by
related reading
- Learning with not Enough Data Part 1: Semi-Supervised Learning | Lil'Loglilianweng.github.io
- Using Artificial Intelligence to Augment Human Intelligencedistill.pub
- Oversight Assistants: Turning Compute into Understandingbounded-regret.ghost.io
- pdfopenreview.net
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- Self-supervised learning - Wikipediaen.wikipedia.org
- You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
- An Overview of Deep Semi-Supervised Learningarxiv.org
- Import AI 455: AI systems are about to start building themselves.importai.substack.com
- The data black hole at the center of AIdwarkesh.com
- Large Language Model: world models or surface statistics?thegradient.pub
- What is Contrastive Learning? (Contrastive Learning/Self-supervised Learning Explained)youtube.com