Ablations are really important – Non_Interactive – Software & ML
I don’t read as many papers as I once did. I find this surprising as I always assumed that when I made ML my full-time job, I would spend a lot more time reading up on all of the things that other folks in the field are up to. To some extent, this is a weakness. There is a healthy balance one should strike between reading and writing and I’m definitely skewing a bit too far towards the writing side of things (code, not papers). With that said, I have the honor of working with some of the people I respect most in the field, and they don’t read much more than I do. There are several good reasons for this, but the one I’d like to talk about today is the importance of slow progress and ablations in the field. These two things work harmoniously to make papers either too boring to read past a quick glance or completely unusable for any future work. Let’s talk about why. One of the most important things that I’ve learned over the last few years is the surprising capacity of neural networks to
I don’t read as many papers as I once did. I find this surprising as I always assumed that when I made ML my full-time job, I would spend a lot more time reading up on all of the things that other folks in the field are up to. To some extent, this is a weakness. There is a healthy balance one should strike between reading and writing and I’m definitely skewing a bit too far towards the writing side of things (code, not papers). With that said, I have the honor of working with some of the people I respect most in the field, and they don’t read much more than I do. There are several good reasons
Explore this link on the map →related reading
- An Opinionated Guide to ML Researchjoschu.net
- Highly Opinionated Advice on How to Write ML Papers — LessWronglesswrong.com
- How to win a best paper award (or, an opinionated take on how to do important research)nicholas.carlini.com
- A Recipe for Training Neural Networkskarpathy.github.io
- how to read research papers, in 5 minutes. | writingmasonjwang.com
- Home - colah's blogcolah.github.io
- A Recipe for Training Neural Networkskarpathy.github.io
- Tips for Empirical Alignment Research — AI Alignment Forumalignmentforum.org
- Distill — Latest articles about machine learningdistill.pub
- Templates for machine learning research papers | Neel Guhaneelguha.github.io
- Thread by @karpathy on Thread Reader App – Thread Reader Appthreadreaderapp.com
- Heuristics for Scientific Writing (a Machine Learning Perspective) – Approximately Correctapproximatelycorrect.com