What Can't Deep Learning Do? – Bharath Ramsundar – Entrepreneur and Scientist

1/ What can’t deep learning do? Worth putting together a list of known failures to guide algorithmic development.

1/ What can’t deep learning do? Worth putting together a list of known failures to guide algorithmic development. 2/ Deep learning methods are known to fail at learning after small jitters to input. Think object recognition breaking when colors are swapped. 3/ Gradient based learning is quite slow. Takes many, many gradient descent steps to pick up patterns. Tough for high dimensional prediction. 4/ Deep learning methods are terrible at handling constraints. Not possible to find solutions satisfying constraints unlike linear programming. 5/ Training for complex models is quite unstable.…

saved by

• Ishan Mukherjee

related reading

• The Decade of Deep Learning | Leo Gaobmk.sh
• A Recipe for Training Neural Networkskarpathy.github.io
• Scaling is subtler than it seemsberen.io
• Debugging Reinforcement Learning Systemsandyljones.com
• Making Deep Learning go Brrrr From First Principleshorace.io
• A Recipe for Training Neural Networkskarpathy.github.io
• Practical Deep Learning for Coders - Practical Deep Learningcourse.fast.ai
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• The Little Book of Deep Learningfleuret.org
• Deep Learning is Not So Mysterious or Differentarxiv.org
• Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com