✳flâneur — a map of the web's best reading

How DeepMind's Generally Capable Agents Were Trained - LessWrong

lesswrong.com · 6,967 words · saved by 1 readers

INTRO One of DeepMind's latest papers, Open-Ended Learning Leads to Generally Capable Agents, explains how DeepMind produced agents that can successfully play games as complex as hide-and-seek or capture-the-flag without even having trained on or seen these games before. As far as I know, this is an entirely unprecedented level of generality for a reinforcement-learning agent. The following is a high-level summary of the paper, meant to be accessible to non-specialists, that should nevertheless produce something resembling a gears-level model. I want to focus on explaining the optimization process that produced this agent; on what the different parts of the optimization process are; on why each different part is necessary; and on what would happen if different parts of it were missing. After that summary, I'll add a few more comments and questions about design choices within the paper and about future research I'd like to see. I'm far less certain about this second part, however. I was going to include a part on AI timelines -- but whether this paper influences your timelines, and in what direction, depends on a lot of priors that are out-of-scope for what I want to do here. THE ENVIRONMENT Before we get into the optimization process of the agent, I need to talk about the environment within which the agent trained. Core to the project of this paper are the millions of dynamically-generated tasks on which the agent can train. Each task in the XLand Unity-powered 3d environment space is defined by a (1) unique physical world and a (2) unique set of goals / rewards. Thoughout what follows I refer to (1) as the "environment" or "world", to (2) as the "game", and to both of them together as a "task." Note that both of these can be generated programmatically without human intervention. (The show reel of the trained agents operating on hand-made, held-out test tasks is worth watching for at least a few minutes to get a feel for the complexity possible from both worl

x How DeepMind's Generally Capable Agents Were Trained — LessWrong DeepMind AI Frontpage 87 How DeepMind's Generally Capable Agents Were Trained by 1a3orn 20th Aug 2021 AI Alignment Forum 23 min read 6 87 Ω 27 Intro One of DeepMind's latest papers, Open-Ended Learning Leads to Generally Capable Agents , explains how DeepMind produced agents that can successfully play games as complex as hide-and-seek or capture-the-flag without even having trained on or seen these games before. As far as I know, this is an entirely unprecedented level of generality for a reinforcement-learning agent. The follo

Explore this link on the map →

saved by

related reading