The Eighty Five Percent Rule for optimal learning | Nature Communications

Is there an optimum difficulty level for training? In this paper, the authors show that for the widely-used class of stochastic gradient-descent based learning algorithms, learning is fastest when the accuracy during training is 85%.

Download PDF Subjects Computer science Human behaviour Learning algorithms Psychology Abstract Researchers and educators have long wrestled with the question of how best to teach their clients be they humans, non-human animals or machines. Here, we examine the role of a single variable, the difficulty of training, on the rate of learning. In many situations we find that there is a sweet spot in which training is neither too easy nor too hard, and where learning progresses most quickly. We derive conditions for this sweet spot for a broad class of learning algorithms in the context of binary cl

related reading

• The 85% Rule for Learning - Scott H Youngscotthyoung.com
• A Recipe for Training Neural Networkskarpathy.github.io
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• A Recipe for Training Neural Networkskarpathy.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Understanding deep learning requires rethinking generalizationarxiv.org
• A critique of pure learning and what artificial neural networks can learn from animal brains | Nature Communicationsnature.com
• Speeding up RL with high-leverage samples | Applied Computeappliedcompute.com
• arxiv.org/pdf/1805.08522arxiv.org
• Efficiently Estimating Pareto Frontiers with Cyclic Learning Rate Schedules | Databricks Blogmosaicml.com
• Human-like Neural Nets by Catapulting · Gwern.netgwern.net
• nn-notes.pdfboris-hanin.github.io