[1803.05859] Neural Network Quine

Self-replication is a key aspect of biological life that has been largely overlooked in Artificial Intelligence systems. Here we describe how to build and train self-replicating neural networks. The network replicates itself by learning to output its own weights. The network is designed using a loss function that can be optimized with either gradient-based or non-gradient-based methods. We also describe a method we call regeneration to train the network without explicit optimization, by injecting the network with predictions of its own parameters. The best solution for a self-replicating network was found by alternating between regeneration and optimization steps. Finally, we describe a design for a self-replicating neural network that can solve an auxiliary task such as MNIST image classification. We observe that there is a trade-off between the network's ability to classify images and its ability to replicate, but training is biased towards increasing its specialization at image classification at the expense of replication. This is analogous to the trade-off between reproduction and other tasks observed in nature. We suggest that a self-replication mechanism for artificial intelligence is useful because it introduces the possibility of continual improvement through natural selection.

[1803.05859] Neural Network Quine Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Artificial Intelligence arXiv:1803.05859 (cs) [Submitted on 15 Mar 2018 ( v1 ), last revised 24 May 2018 (this version, v4)] Title: Neural Network Quine Authors: Oscar Chang , Hod Lipson View a PDF of the paper titled Neural Network Quine, by Oscar Chang and 1 other authors View PDF Abstract: Self-replication is a key aspect of biological life that has been largely overlooked in Artificial Intelligenc

related reading

• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Zoom In: An Introduction to Circuitsdistill.pub
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• A Recipe for Training Neural Networkskarpathy.github.io
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• Self-Assembly Gets Automated in Reverse of ‘Game of Life’ | Quanta Magazinequantamagazine.org
• Natural Intelligencegreydanus.github.io
• On neural scaling and the quanta hypothesisericjmichaud.com
• Towards self-organized controlavariengien.github.io
• Andrej Karpathy — AGI is still a decade awaydwarkesh.com
• A Recipe for Training Neural Networkskarpathy.github.io
• Human-like Neural Nets by Catapulting · Gwern.netgwern.net