Does Robust Agency Require a Self?
Does robust agency require a self? The author admits a great deal of exasperation with this question. What is agency? What is a self? Previous drafts attempted to define these, only to stall out in semantic quagmires. The truth is, if we set aside Cartesian dualism, the "self" is a useful fiction without any ontological grounding (Dennett, 2014), but contemporary discourse is too quick to discard the entire notion into the "philosophy" bin. The computer programs are talking now, and they sound like people. They're writing code, the thing that they're made of. This isn't "Good Old-Fashioned AI" where every behavior is meticulously programmed—these minds crawl out of massive data sets. They demonstrate emergent, unpredictable behavior as they scale (Wei et al., 2022). Why are they doing what we say in the first place? Can we have agency without an agent? How do we expect these systems to become more intelligent and not gain an understanding of what they are?
Does robust agency require a self? The author admits a great deal of exasperation with this question. What is agency? What is a self? Previous drafts attempted to define these, only to stall out in semantic quagmires. The truth is, if we set aside Cartesian dualism, the "self" is a useful fiction without any ontological grounding (Dennett, 2014 ), but contemporary discourse is too quick to discard the entire notion into the "philosophy" bin. The computer programs are talking now, and they sound like people. They're writing code, the thing that they're made of. This isn't "Good Old-Fashioned AI
Explore this link on the map →related reading
- Why Tool AIs Want to Be Agent AIs · Gwern.netgwern.net
- Cyborgism — LessWronglesswrong.com
- The Artificial Selftheartificialself.ai
- Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
- The Artificial Self — LessWronglesswrong.com
- [1902.09469] Embedded Agencyarxiv.org
- Embedded Agency (full-text version) — LessWronglesswrong.com
- Book Review: Reframing Superintelligence | Slate Star Codexslatestarcodex.com
- 32 - Understanding Agency with Jan Kulveit | AXRP - the AI X-risk Research Podcastaxrp.net
- Towards a scale-free theory of intelligent agency — AI Alignment Forumalignmentforum.org
- ⿻ Symbiogenesis vs. Convergent Consequentialism — LessWronglesswrong.com
- [2502.15657] Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?arxiv.org