marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(corpus): advisory-pair repo-overlap -- remove real pilot leaks, skip vendored trees

Browse Source 
test_no_pair_repeats_a_repository_the_benchmarks_already_name failed: advisory-fix
catalog repos also appeared elsewhere under benchmarks/. Root cause split two ways:

1. Genuine leak: 16 catalog repos are also subjects in benchmarks/search/private/
   pilot.json (a real private pilot population). Removed those 16 pairs from the
   three catalogs and their verification.json rows (totals recomputed, catalog ==
   verification preserved).
2. Spurious: the overlap scan greped ~49k vendored files under
   benchmarks/search/private/runs/*/checkout/node_modules/, so any catalog repo a
   vendored dependency's README names collided. Added a narrow vendored-tree skip
   (node_modules/vendor) to the scan rather than deleting labelled pairs over a
   dependency mention.

Implemented from Codex's first pass (gpt-6.1-sol), then gated and corrected: Codex
removed 20 pairs; 4 of those (backstage/backstage, koajs/koa, beaugunderson/
ip-address, cakephp/cakephp) were driven only by vendored/non-population mentions
and are kept. Also excluded advisory-fixes-spread.json (catalog-derived report)
and research-2026-09-26.md (selection write-up) as non-population.

tests/test_advisory_fix_pairs.py + tests/test_learn_labels.py: 53 passed.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-12 15:22:55 +02:00
co-authored by Claude Opus 4.8
parent 60ffff0168
commit 9a3250f6c5
8 changed files with 23 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_advisory_fix_pairs.py
+7
View File
@@ -93,6 +93,13 @@ def test_no_pair_repeats_a_repository_the_benchmarks_already_name(source_id: str
ours = {repo_name(item["repo"]) for item in _raw(folder)}
elsewhere: set[str] = set()
for path in (ROOT / "benchmarks").rglob("*"):
# A vendored dependency tree is not the benchmarks NAMING a repository as a population member: it is a
# checked-out application's own `node_modules`, where any dependency's README or package.json mentions
# its upstream repo. benchmarks/search/private/runs/*/checkout/ carries ~49k such files, so grepping
# them deletes a labelled advisory pair the moment a vendored dep happens to name its repo -- corpus
# loss masquerading as hygiene. Skip vendored trees; the leak guard is about our own populations.
if "node_modules" in path.parts or "vendor" in path.parts:
continue
if not path.is_file() or folder in path.parents or path.name in excluded or path.suffix in {".pyc", ".bin", ".gz", ".jsonl"}:
continue
try: