keras - When to "add" layers and when to "concatenate" in neural networks? - Cross Validated

Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up. Teams Q&A for work Connect and share knowledge within a single location that is structured and easy to search. I am using "add" and "concatenate" as it is defined in keras. Basically, from my understanding, add will sum the inputs (which are the layers, in essence tensors). So if the first layer had a particular weight as 0.4 and another layer with the same exact shape had the corresponding weight being 0.5, then after the add the new weight becomes 0.9. However, with concatenate, let's say the first layer has dimensions 64x128x128 and the second layer had dimensions 32x128x128, then after concatenate, the n

keras - When to "add" layers and when to "concatenate" in neural networks? - Cross Validated Stack Internal Knowledge at work Bring the best of human thought and AI automation together at your work. Explore Stack Internal When to "add" layers and when to "concatenate" in neural networks? Ask Question Asked 7 years, 11 months ago Modified 6 years, 6 months ago Viewed 35k times 28 $\begingroup$ I am using "add" and "concatenate" as it is defined in keras. Basically, from my understanding , add will sum the inputs (which are the layers, in essence tensors). So if the first layer had a particular

related reading

• Zoom In: An Introduction to Circuitsdistill.pub
• A Recipe for Training Neural Networkskarpathy.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
• A Recipe for Training Neural Networkskarpathy.github.io
• CS231n Deep Learning for Computer Visioncs231n.github.io
• Convolutional Neural Networks, Explained | Towards Data Sciencetowardsdatascience.com
• NL.pdfabehrouz.github.io
• Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
• A Geometric Calculator Inside a Neural Networkgoodfire.ai
• A Beginner's Guide To Understanding Convolutional Neural Networks Part 2 – Adit Deshpande – Engineering at Forward | UCLA CS '19adeshpande3.github.io