Marius CiepluchandClaude Opus 4.8 b08f372022 docs: retire the SZZ-pool metric, add the valid-benchmark brief, plain-English pass ...

The frozen SZZ-labeled pool is not a valid benchmark. SZZ puts the vulnerability in
the wrong place about half the time, and ~76% of entries are library-internal (not
reachable in-repo), so its catch/false-alarm numbers (record 07's "3/92") are noise.
Retired as a metric and a gate across the roadmap and briefs; no pool-scored number
is quoted anywhere.

Added .kiro/specs/valid-detection-benchmark/brief.md: positives labelled from the
fix (not SZZ), kept only when app-shaped and reachable in-repo, each verified;
ordinary negatives from the same repos; unseen repos; the whole tool measured; plain
metrics (catch rate, false-alarms per 100 pushes, precision, proved-catch).

Ran a plain-English pass (the humanizer rules) on the roadmap, README, and the two
active briefs via Codex, then reviewed the diff: only these files changed, every
number/path/hash/command is unchanged, and the retired-metric decision is intact.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

2026-10-12 16:38:18 +02:00

..

brief.md

docs: retire the SZZ-pool metric, add the valid-benchmark brief, plain-English pass

2026-10-12 16:38:18 +02:00