marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-18 04:29:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(corpus): advisory-pair repo-overlap -- remove real pilot leaks, skip vendored trees

Browse Source 
test_no_pair_repeats_a_repository_the_benchmarks_already_name failed: advisory-fix
catalog repos also appeared elsewhere under benchmarks/. Root cause split two ways:

1. Genuine leak: 16 catalog repos are also subjects in benchmarks/search/private/
   pilot.json (a real private pilot population). Removed those 16 pairs from the
   three catalogs and their verification.json rows (totals recomputed, catalog ==
   verification preserved).
2. Spurious: the overlap scan greped ~49k vendored files under
   benchmarks/search/private/runs/*/checkout/node_modules/, so any catalog repo a
   vendored dependency's README names collided. Added a narrow vendored-tree skip
   (node_modules/vendor) to the scan rather than deleting labelled pairs over a
   dependency mention.

Implemented from Codex's first pass (gpt-6.1-sol), then gated and corrected: Codex
removed 20 pairs; 4 of those (backstage/backstage, koajs/koa, beaugunderson/
ip-address, cakephp/cakephp) were driven only by vendored/non-population mentions
and are kept. Also excluded advisory-fixes-spread.json (catalog-derived report)
and research-2026-09-26.md (selection write-up) as non-population.

tests/test_advisory_fix_pairs.py + tests/test_learn_labels.py: 53 passed.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-12 15:22:55 +02:00
co-authored by Claude Opus 4.8
parent 60ffff0168
commit 9a3250f6c5
8 changed files with 23 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/openultrasast/learn/sources.toml
+10
View File
@@ -141,6 +141,16 @@ reason = "v3 selection research; never opened"
file = "protocol-v3.md"
reason = "v3 protocol; never opened"
# Counts derived from the advisory-fix catalog, not an independent labelled population.
[[excluded]]
file = "advisory-fixes-spread.json"
reason = "advisory-fix catalog spread report; never opened"
# Selection write-up: rejected-candidate mentions are narrative, not labelled cases.
[[excluded]]
file = "research-2026-09-26.md"
reason = "v2 selection research write-up; never opened"
# Repository aliases: forks and renames that are one repository group. Groups sharing an advisory id (CVE/GHSA) or
# a fix commit are merged automatically; list here only what those cannot see.
# [[alias]]