flâneur

AIs are living on my Macbook Pro | Jordan Cooper's Blog: startups, venture capital, etc...

jordancooper.blog · 1,118 words · saved by 1 readers

Hosted inference is expensive. No two ways about it. GPT4 and Claude 2 are amazing, but if you are a developer using them heavily in production, the bills can stack up pretty quickly. There are plenty of startups that are trying to help developers manage those costs, but I’ve come to believe that local inference is going to be an important dimension to the coming bloom of AI powered applications. There are tons of smaller, task specific open source models that can get the job done running locally on my Macbook Pro, Ipad, and at some point my phone. And the beauty of running inference locally is that it’s free. At some point, the venture dollars and free cloud credits companies rely upon to subsidize users’ inference costs will run out, and when they do, apps are going to (and already have started) to pass those costs along to the end user. As a consumer, I’ve become accustomed to the vast majority of my applications being free, so how does that persist when expensive inference is at th

AIs are living on my Macbook Pro | Jordan Cooper's Blog: startups, venture capital, etc... Search: Jordan Cooper's Blog: startups, venture capital, etc… About Pace Capital AIs are living on my Macbook Pro Posted on August 9, 2023 . Filed under: Uncategorized | Hosted inference is expensive. No two ways about it. GPT4 and Claude 2 are amazing, but if you are a developer using them heavily in production, the bills can stack up pretty quickly. There are plenty of startups that are trying to help developers manage those costs, but I’ve come to believe that local inference is going

related reading