flâneur

Specification gaming examples in AI — LessWrong

lesswrong.com · 1,604 words · saved by 1 readers

Various examples (and lists of examples) of unintended behaviors in AI systems have appeared in recent years. One interesting type of unintended behavior is finding a way to game the specified objective: generating a solution that literally satisfies the stated objective but fails to solve the problem according to the human designer’s intent. This occurs when the objective is poorly specified, and includes reinforcement learning agents hacking the reward function, evolutionary algorithms gaming the fitness function, etc. While ‘specification gaming’ is a somewhat vague category, it is particularly referring to behaviors that are clearly hacks, not just suboptimal solutions. Since these examples are currently scattered across several lists, I have put together a master list of examples collected from the various existing sources. This list is intended to be comprehensive and up-to-date, and serve as a resource for AI safety research and discussion. If you know of any interesting example

x Specification gaming examples in AI — LessWrong Best of LessWrong 2018 Goodhart's Law AI Risk AI Frontpage 49 Specification gaming examples in AI by Vika 3rd Apr 2018 AI Alignment Forum 1 min read 9 49 Ω 18 [Cross-posted from personal blog ] Various examples (and lists of examples ) of unintended behaviors in AI systems have appeared in recent years. One interesting type of unintended behavior is finding a way to game the specified objective: generating a solution that literally satisfies the stated objective but fails to solve the problem according to the human designer’s intent. This occur

saved by

related reading