[2206.08896] Evolution through Large Models

This paper pursues the insight that large language models (LLMs) trained to generate code can vastly improve the effectiveness of mutation operators applied to programs in genetic programming (GP). Because such LLMs benefit from training data that includes sequential changes and modifications, they can approximate likely changes that humans would make. To highlight the breadth of implications of such evolution through large models (ELM), in the main experiment ELM combined with MAP-Elites generates hundreds of thousands of functional examples of Python programs that output working ambulating robots in the Sodarace domain, which the original LLM had never seen in pre-training. These examples then help to bootstrap training a new conditional language model that can output the right walker for a particular terrain. The ability to bootstrap new models that can output appropriate artifacts for a given context in a domain where zero training data was previously available carries implications for open-endedness, deep learning, and reinforcement learning. These implications are explored here in depth in the hope of inspiring new directions of research now opened up by ELM.

Evolution through Large Models Joel Lehman Jonathan Gordon OpenAI OpenAI arXiv:2206.08896v1 [cs.NE] 17 Jun 2022 joel@openai.com gordonjo@openai.com Shawn Jain Kamal Ndousse OpenAI Anthropic∗…

saved by

• Darren Zhu

related reading

• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• [2507.19457] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningarxiv.org
• [2602.19362] LLMs Can Learn to Reason Via Off-Policy RLarxiv.org
• [2506.13131] AlphaEvolve: A coding agent for scientific and algorithmic discoveryarxiv.org
• Program Synthesis with Large Language Modelsarxiv.org
• [2112.02969] Jigsaw: Large Language Models meet Program Synthesisarxiv.org
• A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencealphaxiv.org
• [2502.05209] Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiesarxiv.org
• [2606.12016] Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalizationarxiv.org
• ZeroCoder: Can LLMs Improve Code Generation Without Ground-Truth Supervision?alphaxiv.org