A few days ago, Sam Altman committed the cardinal sin of being technically defensible and rhetorically catastrophic. Appearing on a podcast, he talked about the AI memory bottleneck. The chat turned… | Saanya Ojha | 14 comments
A few days ago, Sam Altman committed the cardinal sin of being technically defensible and rhetorically catastrophic. Appearing on a podcast, he talked about the AI memory bottleneck. The chat turned to local inference - the idea that, as models get smaller and open-source gets better, more AI should run on your laptop rather than inside a giant data center. He was unconvinced. From a “global allocation perspective,” he argued, it is “quite terrible” to have everyone own a bunch of the world’s limited supply of DRAM that sits in their computers being used ~5% of the time. Better, in a supply-constrained world, to pool that memory in the cloud and keep it busy. Twitter translated this, almost instantly, into: Sam wants your RAM. And you can understand why. There is something objectively funny about the CEO of a company engaged in one of the largest infra buildouts in human history explaining that your 64GB MacBook is an inefficient use of scarce semiconductor capacity. But underneath the terrible optics is a sound capacity-utilization argument. Imagine 2 worlds: - One where 10m people each buy a bunch of RAM. Their PCs are asleep/ doing light work most of the day, yet that physical RAM is permanently tied to those machines. - Another where the same RAM sits in giant shared data centers. When you aren’t using it, somebody else is. You get more inference-hours out of each physical unit of scarce memory. So from a pure “maximize AI compute produced per DRAM wafer” perspective, Altman’s point is coherent. Then Why The Backlash? The first problem is who is saying it. Imagine your landlord explaining that private homeownership represents a tragic underutilization of the world’s limited housing stock. Think of the efficiencies! Sure, centralization improves utilization. Conveniently, it also improves the economics of the company selling centralized resources. OpenAI, after all, has a massive commercial interest in making sure that AI inference happens in its data