[2105.02761] Neural Algorithmic Reasoning
Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another. We argue that algorithms possess fundamentally different qualities to deep learning methods, and this strongly suggests that, were deep learning methods better able to mimic algorithms, generalisation of the sort seen with algorithms would become possible with deep learning -- something far out of the reach of current machine learning methods. Furthermore, by representing elements in a continuous space of learnt algorithms, neural networks are able to adapt known algorithms more closely to real-world problems, potentially finding more efficient and pragmatic solutions than those proposed by human computer scientists. Here we present neural algorithmic reasoning -- the art of building neural networks that are able to execute algorithmic computation -- and provide our opinion on its transformative potential for running classical algorithms on inputs previously considered inaccessible to them.
Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another. We argue that algorithms possess fundamentally different qualities to deep learning methods, and this strongly suggests that, were deep learning methods better able to mimic algorithms, generalisation of the sort seen with algorithms would become possible with deep learning -- something far out of the reach of current machine learning methods. Furthermore, by representing elements in a continuous space o
Explore this link on the map →related reading
- Zoom In: An Introduction to Circuitsdistill.pub
- Deep learning as program synthesis — LessWronglesswrong.com
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- As Rocks May Think | Eric Jangevjang.com
- Building machines that learn and think like people | Behavioral and Brain Sciences | Cambridge Corecambridge.org
- 2009.06489arxiv.org
- Supplement to “Why AI alignment could be hard”cold-takes.com
- Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
- The Decade of Deep Learning | Leo Gaobmk.sh
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- ARC progress update: Competing with sampling — LessWronglesswrong.com
- The Little Book of Deep Learningfleuret.org