Mutation testing

A passing test suite is only reassuring if the tests can actually fail. Mutation checking proves it: inject a fault, and confirm a test dies.

How it works

/sdlc-studio mutation run --since <ref>

The gate mutates the changed surface - flips a comparison, drops a guard, changes a return - and re-runs the covering tests per mutation. Each mutant is reported killed (a test caught it) or survived (nothing did). A surviving mutant is a finding: the code path it changed is not really under test.

Honest by construction

• An un-mutatable surface reads un-checked, never as a silent pass.
• A stale report reads STALE.
• A mutation run that was refused is named, not quietly skipped.
• The bytecode cache is purged and the mutation applied to the call site, so a same-length mutant cannot reuse a cached .pyc and report SURVIVED without ever running.

Tip

The discipline in one line: mutate the call site, purge the cache, and assert the patch changed the file. A mutation harness that does not do all three can report a kill it never earned.

Where it fits

Mutation checking is the backstop under executable acceptance criteria. The criteria prove the behaviour is present; the mutation check proves the test would notice if it broke. A guard worth trusting is a guard whose mutant was killed.

See also

• Executable acceptance criteria - the tests mutation checking hardens.
• Reconcile & drift - keeping the evidence honest.