Simple arithmetic addition with neural networks | Andy Kong

Before we start, I want to clarify that I am not just some neural network script kiddie, and that my blog is not going to be just a personal log of all the neural networks I make. I also enjoy electronics and hardware design. That being said, check out this neural network I made! There is an underlying perception that people new to the topic of machine learning have, and that is this: when a model isn't doing too hot, you need either more data or more hidden layers with more neurons between them. This misconception isn't their fault; it seems everyone in the news is talking about these huge neural networks trained on terabytes of data to decide whether to advertise an eco-friendly backpack or water bottle to an unsuspecting consumer based on their heart rate variance. However, more neurons != better fit, as I will proceed to show. We often think of neural networks as endlessly flexible tools. I mean, can't they approximate any known function? And while that is mathematically proven to

Simple arithmetic addition with neural networks | Andy Kong --> Simple arithmetic addition with neural networks Published July 17, 2019 Before we start, I want to clarify that I am not just some neural network script kiddie, and that my blog is not going to be just a personal log of all the neural networks I make. I also enjoy electronics and hardware design. That being said, check out this neural network I made! There is an underlying perception that people new to the topic of machine learning have, and that is this: when a model isn't doing too hot, you need either more data or more hidden l

related reading

• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Toy Models of Superpositiontransformer-circuits.pub
• A Recipe for Training Neural Networkskarpathy.github.io
• A Geometric Calculator Inside a Neural Networkgoodfire.ai
• Zoom In: An Introduction to Circuitsdistill.pub
• How Computationally Complex Is a Single Neuron? | Quanta Magazinequantamagazine.org
• A Recipe for Training Neural Networkskarpathy.github.io
• Reverse Engineering a Neural Network's Clever Solution to Binary Addition - Casey Primozic's Homepagecprimozic.net
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
• On neural scaling and the quanta hypothesisericjmichaud.com
• Machine Learning, Kolmogorov Complexity, and Squishy Bunniestheorangeduck.com