Tao on “blue team” vs. “red team” LLMs | Hacker News
Having LLMs fix bugs or add features is more fraught, since they are prone to cheating or writing non robust code (eg special code paths to pass tests without solving the actual problem). reply Having worked on legacy codebases this is extremely wrong and harmful. Tests are the source of truth more so than your code - and incorrect tests are even more harmful than incorrect code. Having worked on legacy codebases, some of the hardest problems are determining “why is this broken test here that appears to test a behavior we don’t support”. Do we have a bug? Or do we have a bad test? On the other end, when there are tests for scenarios we don’t actually care about it’s impossible to determine if that test is meaningful or was added because “it’s testing the code as written”. reply Anticipating future responses: yes, a robust test harness allows you to make changes fearlessly. But most big test suites I’ve seen are less “harness” and more “straight-jacket” reply reply reply Change detector
Tao on “blue team” vs. “red team” LLMs | Hacker News Hacker News new | past | comments | ask | show | jobs | submit login Tao on “blue team” vs. “red team” LLMs ( mathstodon.xyz ) 542 points by qsort 10 months ago | hide | past | favorite | 171 comments _alternator_ 10 months ago | next [–] This red vs blue team is a good way to understand the capabilities and current utility of LLMs for expert use. I trust them to add tests almost indiscriminately because tests are usually cheap; if they are wrong it’s easy to remove or modify them; and if they are correct, they adds value. But often they don
saved by
related reading
- Building better AI tools | Hacker Newsnews.ycombinator.com
- Prevent cognitive debt by manually retyping LLM-generated code — Ankur Sethi's Lab Notebookankursethi.com
- Your job is to deliver code you have proven to worksimonwillison.net
- AI is removing the middle class of software engineeringblog.florianherrengt.com
- Here’s how I use LLMs to help me write codesimonwillison.net
- Malleable software in the age of LLMsgeoffreylitt.com
- Simulated Users & Sad LLMs1a3orn.com
- Coding Models Are Doing Too Much | whnrehiew.github.io
- Would you fly on an AI-coded plane | Hackle's bloghacklewayne.com
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io
- There's no reason for software to be slow anymoredanluu.com
- The growing divide between AI hype and software engineering realityoptimizedbyotto.com