✳flâneur — a map of the web's best reading

Can I run AI locally? | Hacker News

news.ycombinator.com · 19,974 words · saved by 1 readers

1. small models like the new qwen3.5:9b can be fantastic for local tool use, information extraction, and many other embedded applications. 2. For coding tools, just use Google Antigravity and gemini-cli, or, Anthropic Claude, or... Now to be clear, I have spent perhaps 100 hours in the last year configuring local models for coding using Emacs, Claude Code (configured for local), etc. However, I am retired and this time was a lot of fun for me: lot's of efforts trying to maximize local only results. I don't recommend it for others. I do recommend getting very good at using embedded local models in small practical applications. Sweet spot. reply What's also new here, is VRAM-context size trade-off: for 25% of it's attention network, they use the regular KV cache for global coherency, but for 75% they use a new KV cache with linear(!!!!) memory-token-context size expansion! which means, eg ~100K token -> 1.5gb VRAM use -meaning for the first time you can do extremely long conversations /

Can I run AI locally? | Hacker News Hacker News new | past | comments | ask | show | jobs | submit login Can I run AI locally? ( canirun.ai ) 1520 points by ricardbejarano 3 months ago | hide | past | favorite | 353 comments mark_l_watson 3 months ago | next [–] I have spent a HUGE amount of time the last two years experimenting with local models. A few lessons learned: 1. small models like the new qwen3.5:9b can be fantastic for local tool use, information extraction, and many other embedded applications. 2. For coding tools, just use Google Antigravity and gemini-cli, or, Anthropic Claude, o

Explore this link on the map →

saved by

related reading