✳flâneur — a map of the web's best reading

Autopoietic systems and difficulty of AGI alignment - AI Alignment Forum

alignmentforum.org · 3,910 words · saved by 1 readers

I have recently come to the opinion that AGI alignment is probably extremely hard. But it's not clear exactly what AGI or AGI alignment are. And there are some forms of aligment of "AI" systems that are easy. Here I operationalize "AGI" and "AGI alignment" in some different ways and evaluate their difficulties. -------------------------------------------------------------------------------- AUTOPOIETIC COGNITIVE SYSTEMS From Wikipedia: The term "autopoiesis" refers to a system capable of reproducing and maintaining itself. This isn't entirely technically crisp. I'll elaborate on my usage of the term: * An autopoietic system expands, perhaps indefinitely. It will feed on other resources and through its activity gain the ability to feed on more things. It can generate complexity that was not present in the original system through e.g. mutation and selection. In some sense, an autopoietic system is like an independent self-sustaining economy. * An autopoietic system, in principle, doesn't need an external source of autopoesis. It can maintain itself and expand regardless of whether the world contains other autopoietic systems. * An autopoietic cognitive system contains intelligent thinking. Some examples: * A group of people on an island that can survive for a long time and develop technology is an autopoietic cognitive system. * Evolution is an autopoietic cognitive system (cognitive because it contains animals). * An economy made of robots that can repair themselves, create new robots, gather resources, develop new technology, etc is an autopoietic cognitive system. * A moon base that necessarily depends on Earth for resources is not autopoietic. * A car is not autopoietic. * A computer with limited memory not connected to the external world can't be autopoietic. FULLY AUTOMATED AUTOPOIETIC COGNITIVE SYSTEMS A fully automated autopoietic cogni

x Autopoietic systems and difficulty of AGI alignment — AI Alignment Forum Personal Blog 10 Autopoietic systems and difficulty of AGI alignment by jessicata 20th Aug 2017 6 min read 14 10 I have recently come to the opinion that AGI alignment is probably extremely hard. But it's not clear exactly what AGI or AGI alignment are. And there are some forms of aligment of "AI" systems that are easy. Here I operationalize "AGI" and "AGI alignment" in some different ways and evaluate their difficulties. Autopoietic cognitive systems From Wikipedia : The term "autopoiesis" refers to a system capable of

Explore this link on the map →

related reading