A foolproof way to shrink deep learning models | MIT News | Massachusetts Institute of Technology

MIT researchers have proposed a technique for shrinking deep learning models that they say is simpler and produces more accurate results than state-of-the-art methods. It works by retraining the smaller, pruned model at its faster, initial learning rate.

​Researchers unveil a pruning algorithm to make artificial intelligence applications run faster. Kim Martineau | MIT Quest for Intelligence Publication Date : April 30, 2020 Press Inquiries Press Contact : Kim Martineau Email: kimmarti@mit.edu Phone: 617-710-5216 MIT Quest for Intelligence Close Caption : MIT researchers have proposed a technique for shrinking deep learning models that they say is simpler and produces more accurate results than state-of-the-art methods. It works by retraining the smaller, pruned model at its faster, initial learning rate. Credits : Image: Alex Renda Previous i

related reading

• The Little Book of Deep Learningfleuret.org
• The Hardest Part of Shrinking a Robotics Model | Haptic Labshapticlabs.ai
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• The Scaling Hypothesis · Gwern.netgwern.net
• The Decade of Deep Learning | Leo Gaobmk.sh
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
• [2309.10668] Language Modeling Is Compressionarxiv.org
• MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• [1905.11946] EfficientNet: Rethinking Model Scaling for Convolutional Neural Networksarxiv.org
• [1503.02531] Distilling the Knowledge in a Neural Networkarxiv.org