Dear Agent: Prove it. • Rijnard van Tonder
Count the number of r's in strawberry. Actually no, prove it instead. Why tests won't save us and how formal verification becomes essential when agents write our code.
Dear Agent: Prove it. Count the number of r's in strawberry. Actually no, prove it instead. At some point it became possible to write the following prompt, and any frontier LLM will give you a correct answer. > Write a Lean program that counts the number of times a character occurs in a string. Prove it's correct. You'll get a program like this, or some variation of it. We'll walk through it in just a minute. /-- Count occurrences of a character in a list. -/ def countCharList (c : Char ) : List Char → Nat | [] => 0 | x :: xs => ( if x = c then 1 else 0 ) + countCharList c xs /-- Count occurre
related reading
- A shallow dive into formal verificationvitalik.eth.limo
- On the Unreasonable Effectiveness of Property-Based Testing for Validating Formal Specifications | Proofs and Intuitionsproofsandintuitions.net
- LLM Powered Autonomous Agents | Lil'Loglilianweng.github.io
- Your job is to deliver code you have proven to worksimonwillison.net
- Human Judgment as a Specificationblog.brownplt.org
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io
- lf-lean: The frontier of verified software engineering | Theoremtheorem.dev
- Lean Programming Languagelean-lang.org
- A Lean Syntax Primer — overreactedoverreacted.io
- Formally Verifying the Easy Partbrainflow.substack.com
- Lean Software Scaling Laws · Gwern.netgwern.net
- Formalizing Fermat's Last Theoremanthropic.com