Theorem
Most testing strategies miss rare edge cases until customers find them in production. We’ve developed a system that automatically generates targeted unit tests for rare bugs, including the one that would have caught Anthropic’s recent approximate top-K bug.
Fractional proof decomposition fuses partial evaluation and property-based testing to scale testing compute logarithmically with bug rarity, instead of linearly. Introduction Most large projects have a limited compute budget for testing, lest it slows down the CI pipeline. This means missing rare edge cases until customers find them in production. I love formal verification! And I want to bring the power of reasoning about program structure to testing. In this blog post, I will briefly introduce fractional proof decomposition, our new technique that makes testing compute scale logarithmically
Explore this link on the map →saved by
related reading
- Reading 2: Testingweb.mit.edu
- Reading 3: Testingweb.mit.edu
- On the Unreasonable Effectiveness of Property-Based Testing for Validating Formal Specifications | Proofs and Intuitionsproofsandintuitions.net
- Introduction to Property Based Testing | by Nicolas Dubien | Criteo Tech Blog | Mediummedium.com
- The sad state of property-based testing librariesstevana.github.io
- Software Engineering at Googleabseil.io
- Lightweight property-based testing at Row Zerograntslatton.com
- A shallow dive into formal verificationvitalik.eth.limo
- https://danluu.comdanluu.com
- CSC 151 - Unit Testingeikmeier.sites.grinnell.edu
- Testing robotics systems in fast-paced startups | Michael Jae-Yoon Chungmjyc.github.io
- When AI Writes the World's Software, Who Verifies It? — Leonardo de Mouraleodemoura.github.io