Comments - The Use of Knowledge in (AGI) Society

lukedrago.substack.com · 452 words · saved by 1 readers

This seems too optimistic to me. A lot of the importance of tacit knowledge comes from the need to understand the current state of a system when making changes to it. For instance, it's hard to come in and add a feature to a software system without understanding exactly how it currently works. Trying to interfere without this knowledge risks causing unintended effects through unknown unknowns, and this seems true in central planning more generally. But a superintelligent AI system wouldn't be restricted to modifying existing systems — it could conceivably replace them from scratch once it reaches a certain level of performance. Current AI systems might not be able to replicate Google's search algorithm without the tacit knowledge of its employees, but once AI becomes significantly better than humans at software engineering, it would be able to build its own search algorithm from scratch, making that human tacit knowledge disposable. This reminds me of a strategy I once heard of for bec

I have a few things to say on centralized economies and AGI automation: 1. It doesn't matter if a centralized AI economy could fail to satisfy the needs of most humans, so long as you can change what they want essentially arbitrarily, and this point is a pretty big reason why I believe that conditional on centralized AI systems coming into power and centralizing the economy, they might essentially stick around forever. Another way to say it is that you can make the problem of central planning easy by changing what a consumer wants to be easy to satisfy, which is a huge way to subvert…

Comments - The Use of Knowledge in (AGI) Society

related reading