✳flâneur — a map of the web's best reading

Sakana, Strawberry, and Scary AI - by Scott Alexander

astralcodexten.com · 2,263 words · saved by 1 readers

Sakana (website, paper) is supposed to be “an AI scientist”. Since it can’t access the physical world, it can only do computer science. Its human handlers give it a computer program. It prompts itself to generate hypotheses about the program (“if I change this number, the program will run faster”). Then it uses an AI coding submodule to test its hypotheses. Then it uses a language model to write them up in typical scientific paper format. Is it good? Not really. Experts who read its papers say they’re trivial, poorly reasoned, and occasionally make things up (the creators defend themselves by saying that “less than ten percent” of the AI’s output is hallucinations). Its writing is meandering, repetitive, and often self-contradictory. Like the proverbial singing dog, we’re not supposed to be impressed that it’s good, we’re supposed to be impressed that it can do it at all. The creators - a Japanese startup with academic collaborators - try to defend their singing dog. They say its AI pa

Sakana, Strawberry, and Scary AI ... Scott Alexander Sep 18, 2024 356 546 33 Share I. Sakana ( website , paper ) is supposed to be “an AI scientist”. Since it can’t access the physical world, it can only do computer science. Its human handlers give it a computer program. It prompts itself to generate hypotheses about the program (“if I change this number, the program will run faster”). Then it uses an AI coding submodule to test its hypotheses. Finally, it uses a language model to write them up in typical scientific paper format. Is it good? Not really. Experts who read its papers say they’re

Explore this link on the map →

related reading