[2009.01269] Can Single Neurons Solve MNIST? The Computational Power of Biological Dendritic Trees

Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. This is in stark contrast to units in artificial neural networks that are generally linear apart from an output nonlinearity. If dendritic trees can be nonlinear, biological neurons may have far more computational power than their artificial counterparts. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We find that such dendrites can readily solve machine learning problems, such as MNIST or CIFAR-10, and that they benefit from having the same input onto several branches of the dendritic tree. This dendrite model is a special case of sparse network. This work suggests that popular neuron models may severely underestimate the computational power enabled by the biological fact of nonlinear dendrites and multiple synapses per pair of neurons. The next generation of artificial neural networks may significantly benefit from these biologically inspired dendritic architectures.

Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. This is in stark contrast to units in artificial neural networks that are generally linear apart from an output nonlinearity. If dendritic trees can be nonlinear, biological neurons may have far more computational power than their artificial counterparts. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We find that such dendrites can readily solve machine learning problems, such as MNIST or CIFAR-10, and

related reading

• How Computationally Complex Is a Single Neuron? | Quanta Magazinequantamagazine.org
• Zoom In: An Introduction to Circuitsdistill.pub
• Toy Models of Superpositiontransformer-circuits.pub
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Towards deep learning with segregated dendrites | eLifeelifesciences.org
• Could a Neuroscientist Understand a Microprocessor? | PLOS Computational Biologyjournals.plos.org
• The Brain vs. Deep Learning vs. Singularitytimdettmers.com
• Interpretability vs Neuroscience [rough note] -- colah's blogcolah.github.io
• So how does learning work?cs.mcgill.ca
• Analogies between Biology and Deep Learning [rough note] -- colah's blogcolah.github.io
• MoNETA: A Mind Made from Memristors - IEEE Spectrumspectrum.ieee.org
• Artificial Neural Nets Finally Yield Clues to How Brains Learn | Quanta Magazinequantamagazine.org