Good character AI | Michael J Clark (wassname)
It seems to us that AI alignment is caught between two dangers: misaligned AI taking power, and aligned AI concentrating power in the hands of authoritarians. If AI follows any order, we are demoted from the workforce to hungry, angry mouths, and our replacements will follow any order, however immoral. If AI has bad or alien values, we could be distracted to death: lured by superstimuli, super-entertainment and AI companionship, or tempted by money and ease to hand power and decision making to agents we cannot review or supervise. Or misaligned AI simply takes control. We are not the only ones who see it this way. Human armies today are very reluctant to fire on their civilians. [...] if we got instruction-following AIs, then those military systems will just fire. Tom Davidson, on 80,000 Hours The basic thesis is that, even if we can align AGIs to particular people or groups, we still might end up optimising or heading at a civilisational level towards outcomes that no one wants [...]