Creepy crawlies — Konstantin Ryabitsev
people.kernel.org · 1,460 words · saved by 1 readers
You've probably heard me complain about the "AI crawlers" before, but now I actually have some hard numbers I can put up to show their im...
You've probably heard me complain about the “AI crawlers” before, but now I actually have some hard numbers I can put up to show their impact. In a few words, it's bad enough to create a constant “background radiation” of system load, permanently tying up a chunk of capacity spent on producing output that is only useful for a single purpose — feeding a learning model. TL;DR: we spend more CPU cycles rendering commits for scrapers than we spend on all other kinds of legitimate access, including git clones. At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing…
saved by
related reading
- The Future of Everything is Lies, I Guessaphyr.com
- Git at any scalecursor.com
- Aggressive AI scrapers are making it kinda suck to run wikis | Weird Gloopweirdgloop.org
- The machines are fine. I'm worried about us.ergosphere.blog
- I Will Fucking Piledrive You If You Mention AI Again — Ludicityludic.mataroa.blog
- AI is removing the middle class of software engineeringblog.florianherrengt.com
- Simulated Users & Sad LLMs1a3orn.com
- The Dark Forest and Generative AImaggieappleton.com
- The growing divide between AI hype and software engineering realityoptimizedbyotto.com
- Kernel Recipes 2026 - Security in the LLM ageyoutube.com
- Coding Models Are Doing Too Much | whnrehiew.github.io
- Thoughts on slowing the fuck downmariozechner.at