Dear Agent: Prove it. • Rijnard van Tonder
Count the number of r's in strawberry. Actually no, prove it instead. Why tests won't save us and how formal verification becomes essential when agents write our code.
Dear Agent: Prove it. Count the number of r's in strawberry. Actually no, prove it instead. At some point it became possible to write the following prompt, and any frontier LLM will give you a correct answer. > Write a Lean program that counts the number of times a character occurs in a string. Prove it's correct. You'll get a program like this, or some variation of it. We'll walk through it in just a minute. /-- Count occurrences of a character in a list. -/ def countCharList (c : Char ) : List Char → Nat | [] => 0 | x :: xs => ( if x = c then 1 else 0 ) + countCharList c xs /-- Count occurre
related reading
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io
- Your job is to deliver code you have proven to worksimonwillison.net
- On the Unreasonable Effectiveness of Property-Based Testing for Validating Formal Specifications | Proofs and Intuitionsproofsandintuitions.net
- lf-lean: The frontier of verified software engineering | Theoremtheorem.dev
- Formally Verifying the Easy Partbrainflow.substack.com
- The Case Against Formal Verification, 50 Years Laterivan-gavran.github.io
- Would you fly on an AI-coded plane | Hackle's bloghacklewayne.com
- Formalizing Fermat's Last Theoremanthropic.com
- Automatic Formal Verification for Code Generationlogicalintelligence.com
- Intent Formalization: A Grand Challenge for Reliable Coding in the Age of AI Agentsalphaxiv.org
- Lean Programming Languagelean-lang.org
- AI Will Write All the Code. Mathematics Will Prove It Works.menlovc.com