Growing or Compressing Datasets · Introduction to Data-Centric AI
dcai.csail.mit.edu · 1,826 words · saved by 1 readers
Learn cost-effective ways to grow datasets and cost-effective ways to train models on large datasets.
Growing or Compressing Datasets · Introduction to Data-Centric AI Growing or Compressing Datasets Slides: [ pdf , pptx ] For supervised learning applications where training data have been annotated by humans, labeling is time-consuming and expensive. This lecture focuses on ways to more carefully select what examples to label and reduce the labeling burden of creating modern ML systems. Specifically, we look at the following approaches: Active learning as a way to intelligently select examples to label and grow datasets Core-set selection to compress datasets down to a representative su
related reading
- Learning with not Enough Data Part 2: Active Learninglilianweng.github.io
- Practical Deep Learning for Coders - Practical Deep Learningcourse.fast.ai
- Learning with not Enough Data Part 1: Semi-Supervised Learning | Lil'Loglilianweng.github.io
- Introduction to Data-Centric AIdcai.csail.mit.edu
- Just Ask for Generalization | Eric Jangevjang.com
- The Little Book of Deep Learningfleuret.org
- An Overview of Deep Semi-Supervised Learningarxiv.org
- A Course in Machine Learningciml.info
- Compression and Intelligencegreene.sh
- Why you need to improve your training data, and how to do it << Pete Warden's blogpetewarden.com
- [2309.10668] Language Modeling Is Compressionarxiv.org
- [2605.12715] Scaling Laws for Mixture Pretraining Under Data Constraintsarxiv.org