Richard Ngo's Shortform — LessWrong
Comment by Richard_Ngo - Around 8 years ago Dario Amodei, Jan Leike and a few others predicted that we’d probably have AGI by now. Since then, AI capabilities have progressed so fast that many people updated that the short timelines advocates were right. And yes, their expectations about the speed of AI progress were far more directionally correct than almost anyone expected. But we don’t in fact have AGI (even under relatively weak definitions, like a reliable drop-in replacement for central examples of white-collar jobs), and the world would look very different if we did. Now a large proportion of the AI safety community is implicitly or explicitly orienting to futures where an intelligence explosion occurs within a few years. My default expectation (absent an extensive pause) is that a similar thing will happen: they'll turn out to be directionally correct (relative to the expectations of almost anyone not linked to the community) but factually wrong. Specifically, we won’t have superintelligence within the next 8 years, but things will still be moving so fast that it’ll *feel* like the people who argued for short timelines were right. I’ll write about this more extensively soon, but I wanted to say something now because it feels like the level of bandwagoning towards “singularity soon” is getting pretty wild. I previously noted that we’re seeing a pretty sharp divergence between the measured capabilities of models and their real-world impacts, which should suggest that something weird is going on. My sense is that most AI safety people are brushing this aside with the idea that automating ML research is all you need for superintelligence, without thinking very much about where and how the automated ML research would actually lead to atoms moving around in the physical world. I worry about the next decade feeling like a Shepard tone: an auditory illusion of a sound that's continually going up. In such a scenario, people would continually be panicking and grasp