feat(benchmark): freeze the valid-benchmark pilot, 12 verified positives + 240 negatives

First live run of the valid detection benchmark. Labels come from the fix
site (fix^1), never SZZ.

- 32 held-out Python/JS advisories sourced from the public pool-p1 set,
  CWE + text from OSV. 133 candidate locations -> 125 after dropping
  tests/minified.
- 125 DeepSeek verification calls: 31 locations passed all three checks
  (vulnerable pre-fix, fix addresses it, reachable in-repo) across 12 repos.
- Maintainer spot-check accepted all 12.
- 20 ordinary negatives per positive change from the same repos, 240 total.
- Held-out guard: 0 dropped against the training-only used set.
- Frozen manifest at benchmarks/valid/pilot/ (252 changes), provenance and
  NOTES under benchmarks/measurements/2026-10-12-valid-benchmark-pilot/.

Three fixes found by the live run:
- catalog.py: skip a site the fix renamed or removed instead of aborting.
- assemble.py: negative ratio counts positive changes, not accepted
  functions (malla would have demanded 260 negatives). Test updated.
- held-out guard: run against training sources only; build_used() also
  scans measurements/populations, which flagged the 12 repos via my own
  SZZ-pool analysis outputs (circular).

Replay and metrics not yet run; see NOTES.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

benchmarks/valid/assemble.py

@@ -77,7 +77,11 @@ def freeze(
         head = (history_heads or {}).get(repo) or git.run("rev-parse", "HEAD").strip()
         heads[repo] = head
         history = sorted(extract.ordinary_history(git, head), key=lambda row: row["head"])
-        ordinary, exclusions = negatives(history, fixes, functions, count=ratio * len(positives), seed=seed)
+        # Ratio is per positive CHANGE (a fix commit), not per accepted function. Several
+        # accepted functions in one fix collapse to one vulnerability change below, so the
+        # negative count must track distinct (commit, parent) pairs.
+        change_count = len({(case["commit"], case["parent"]) for case in positives})
+        ordinary, exclusions = negatives(history, fixes, functions, count=ratio * change_count, seed=seed)
         draws[repo] = exclusions
         changes = []
         pairs = defaultdict(list)