Dear Agent: Prove it. • Rijnard van Tonder
Count the number of r's in strawberry. Actually no, prove it instead. Why tests won't save us and how formal verification becomes essential when agents write our code.
Dear Agent: Prove it. Count the number of r's in strawberry. Actually no, prove it instead. At some point it became possible to write the following prompt, and any frontier LLM will give you a correct answer. > Write a Lean program that counts the number of times a character occurs in a string. Prove it's correct. You'll get a program like this, or some variation of it. We'll walk through it in just a minute. /-- Count occurrences of a character in a list. -/ def countCharList (c : Char ) : List Char → Nat | [] => 0 | x :: xs => ( if x = c then 1 else 0 ) + countCharList c xs /-- Count occurre
Explore this link on the map →related reading
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io
- Your job is to deliver code you have proven to worksimonwillison.net
- On the Unreasonable Effectiveness of Property-Based Testing for Validating Formal Specifications | Proofs and Intuitionsproofsandintuitions.net
- lf-lean: The frontier of verified software engineering | Theoremtheorem.dev
- Would you fly on an AI-coded plane | Hackle's bloghacklewayne.com
- Automatic Formal Verification for Code Generationlogicalintelligence.com
- LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
- Human Judgment as a Specificationblog.brownplt.org
- Prediction: AI will make formal verification go mainstream - Martin Kleppmann's blogmartin.kleppmann.com
- Demystifying evals for AI agents \ Anthropicanthropic.com
- Lean (proof assistant) - Wikipediaen.wikipedia.org
- 2510.01346arxiv.org