flâneur

Hōrōshi バガボンド on X: "Your LLM Doesn't Write Correct Code. It Writes Plausible Code." / X

x.com · 4,348 words · saved by 1 readers

To view keyboard shortcuts, press question mark View keyboard shortcuts Home Explore Notifications Chat Grok Premium Bookmarks Articles Profile More Post amir @amirbolous Article See new posts Conversation Jeremy Howard reposted Hōrōshi バガボンド @KatanaLarp Your LLM Doesn't Write Correct Code. It Writes Plausible Code. 13 33 220 46K One of the simplest tests you can run on a database: Doing a primary key lookup on 100 rows. SQLite takes 0.09 ms. An LLM-generated Rust rewrite takes 1,815.43 ms. It's not a misplaced comma! The rewrite is 20,171 times slower on one of the most basic database operations. The thing is though: The code compiles. It passes all its tests. It reads and writes the correct SQLite file format. Its README claims MVCC concurrent writers, file compatibility, and a drop-in C API. On first glance it reads like a working database engine. But it is not! LLMs optimize for plausibility over correctness. In this case, plausible is about 20,000 times slower than correct. I writ

One of the simplest tests you can run on a database: Doing a primary key lookup on 100 rows. SQLite takes 0.09 ms. An LLM-generated Rust rewrite takes 1,815.43 ms. It's not a misplaced comma! The rewrite is 20,171 times slower on one of the most basic database operations. The thing is though: The code compiles. It passes all its tests. It reads and writes the correct SQLite file format. Its README claims MVCC concurrent writers, file compatibility, and a drop-in C API. At first glance it reads like a working database engine. But it is not! EDIT: Several readers have confused this…

saved by

related reading