Test suites as classifiers - Made of Bugs
Suppose we have some codebase we’re considering applying some patch to, and which has a robust and maintained test suite. Considering the patch, we may ask, is this patch acceptable to apply and deploy. By this we mean to ask if the patch breaks any important functionality, violates any key properties or invariants of the codebase, or would otherwise cause some unacceptable risk or harm. In principle, we can divide all patches into “acceptable” or “unacceptable” relative to some project-specific notion of what we’re willing to allow.
Suppose we have some codebase we’re considering applying some patch to, and which has a robust and maintained test suite. Considering the patch, we may ask, is this patch acceptable to apply and deploy. By this we mean to ask if the patch breaks any important functionality, violates any key properties or invariants of the codebase, or would otherwise cause some unacceptable risk or harm. In principle, we can divide all patches into “acceptable” or “unacceptable” relative to some project-specific notion of what we’re willing to allow. In practice, it’s not always easy to determine for sure if a
Explore this link on the map →related reading
- Software Engineering at Googleabseil.io
- Reading 2: Testingweb.mit.edu
- Reading 3: Testingweb.mit.edu
- Demystifying evals for AI agents \ Anthropicanthropic.com
- CSC 151 - Unit Testingeikmeier.sites.grinnell.edu
- Tao on “blue team” vs. “red team” LLMs | Hacker Newsnews.ycombinator.com
- The Joel Test: 12 Steps to Better Code – Joel on Softwarejoelonsoftware.com
- Google Testing Blogtesting.googleblog.com
- Catching bugs with fractional proofs | Theoremtheorem.dev
- Introducing FrontierCode | Cognitioncognition.ai
- Would you fly on an AI-coded plane | Hackle's bloghacklewayne.com
- commonware > Change is the Only Constantcommonware.xyz