Cities of "Solving Alignment" — LessWrong
Often people reference "solving alignment" as a win condition, but I think it's important to specify what this means. Particularly, I think: Because of all this, the things we need to do before solving alignment might themselves suggest different strategies for solving alignment than the ones we currently use, and different ways in which law and policy AI safety people should be spending their time. There seem to be a few different properties we could aim for when "solving alignment": For each potential property, there is a centralized case, where a singleton ASI has the property, a decentralized case, where multiple ASIs have the property, and an emergent case, where individual AIs don't, but collectively-superintelligent swarms do. There are three important things to note about these properties from the get-go: Because not-misalignedness is a prerequisite for corrigibility and humanlikeness, and because it is obviously good to have not-misaligned models, I think the vast majority of