test_no_pair_repeats_a_repository_the_benchmarks_already_name failed: advisory-fix
catalog repos also appeared elsewhere under benchmarks/. Root cause split two ways:
1. Genuine leak: 16 catalog repos are also subjects in benchmarks/search/private/
pilot.json (a real private pilot population). Removed those 16 pairs from the
three catalogs and their verification.json rows (totals recomputed, catalog ==
verification preserved).
2. Spurious: the overlap scan greped ~49k vendored files under
benchmarks/search/private/runs/*/checkout/node_modules/, so any catalog repo a
vendored dependency's README names collided. Added a narrow vendored-tree skip
(node_modules/vendor) to the scan rather than deleting labelled pairs over a
dependency mention.
Implemented from Codex's first pass (gpt-6.1-sol), then gated and corrected: Codex
removed 20 pairs; 4 of those (backstage/backstage, koajs/koa, beaugunderson/
ip-address, cakephp/cakephp) were driven only by vendored/non-population mentions
and are kept. Also excluded advisory-fixes-spread.json (catalog-derived report)
and research-2026-09-26.md (selection write-up) as non-population.
tests/test_advisory_fix_pairs.py + tests/test_learn_labels.py: 53 passed.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Real (vulnerable function, fixed function) pointer pairs for the six compiled
families, mined at zero model cost from reviewed GHSA advisories (pip, npm,
maven, composer), newest first: 79 of 98 published after 2026-06-01
(post_cutoff = true). Each pair: one linked one-parent fix commit, <= 4 source
files / 120 lines, the function holding the changed code lines declared once on
both sides with a changed body; every function diff read and kept only when the
named function holds the fix (145 reviewed, 100 kept); 2 dropped by the
reserved-population test; permissive licences only, no repository named
elsewhere under benchmarks/.
New label source advisory-fixes-2; the advisory-fix tests run over both
batches. Memory 937 -> 1133 static examples (98 paired groups added);
positives-only groups stay 99 (vibe-py/dev-php/v2 data gaps), their share of
positive groups in the six families 0.40 -> 0.29. Embeddings: 7 calls,
$0.0014. No chat-model call. Record:
benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
