Why you need to improve your training data, and how to do it « Pete Warden's blog
Photo by Lisha Li Andrej Karpathy showed this slide as part of his talk at Train AI and I loved it! It captures the difference between deep learning research and production perfectly. Academic pape…
May 28, 2018 By Pete Warden in Uncategorized 21 Comments Photo by Lisha Li Andrej Karpathy showed this slide as part of his talk at Train AI and I loved it! It captures the difference between deep learning research and production perfectly. Academic papers are almost entirely focused on new and improved models, with datasets usually chosen from a small set of public archives. Everyone I know who uses deep learning as part of an actual application spends most of their time worrying about the training data instead. There are lots of good reasons why researchers are so fixated on model architectu
saved by
related reading
- A Recipe for Training Neural Networkskarpathy.github.io
- Making Deep Learning go Brrrr From First Principleshorace.io
- A Recipe for Training Neural Networkskarpathy.github.io
- The Only Important Technology Is The Internet - Kevin Lukevinlu.ai
- Thinking about High-Quality Human Data | Lil'Loglilianweng.github.io
- Just Ask for Generalization | Eric Jangevjang.com
- Pretraining progress is mostly coming from datadwarkesh.com
- Training Data: What Is It? All About Machine Learning Training Dataappen.com
- The Little Book of Deep Learningfleuret.org
- Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
- GitHub - google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models.github.com
- Deep Dive into LLMs like ChatGPTyoutube.com