Hōrōshi バガボンド on X: "Your LLM Doesn't Write Correct Code. It Writes Plausible Code." / X
To view keyboard shortcuts, press question mark View keyboard shortcuts Home Explore Notifications Chat Grok Premium Bookmarks Articles Profile More Post amir @amirbolous Article See new posts Conversation Jeremy Howard reposted Hōrōshi バガボンド @KatanaLarp Your LLM Doesn't Write Correct Code. It Writes Plausible Code. 13 33 220 46K One of the simplest tests you can run on a database: Doing a primary key lookup on 100 rows. SQLite takes 0.09 ms. An LLM-generated Rust rewrite takes 1,815.43 ms. It's not a misplaced comma! The rewrite is 20,171 times slower on one of the most basic database operations. The thing is though: The code compiles. It passes all its tests. It reads and writes the correct SQLite file format. Its README claims MVCC concurrent writers, file compatibility, and a drop-in C API. On first glance it reads like a working database engine. But it is not! LLMs optimize for plausibility over correctness. In this case, plausible is about 20,000 times slower than correct. I writ
One of the simplest tests you can run on a database: Doing a primary key lookup on 100 rows. SQLite takes 0.09 ms. An LLM-generated Rust rewrite takes 1,815.43 ms. It's not a misplaced comma! The rewrite is 20,171 times slower on one of the most basic database operations. The thing is though: The code compiles. It passes all its tests. It reads and writes the correct SQLite file format. Its README claims MVCC concurrent writers, file compatibility, and a drop-in C API. At first glance it reads like a working database engine. But it is not! EDIT: Several readers have confused this…
saved by
related reading
- Agent swarms and the new model economics · Cursorcursor.com
- Here’s how I use LLMs to help me write codesimonwillison.net
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- There's no reason for software to be slow anymoredanluu.com
- Malleable software in the age of LLMsgeoffreylitt.com
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- crawshaw - 2025-01-06crawshaw.io
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io
- There's no reason for software to be slow anymoredanluu.com
- Tao on “blue team” vs. “red team” LLMs | Hacker Newsnews.ycombinator.com
- Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQLthinkingmachines.ai
- Lean Software Scaling Laws · Gwern.netgwern.net