marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

bench(pairs): real fix-commit pairs for the short families

Browse Source 
97 advisory-fix pairs, one repository each (309 advisories examined, 212
rejected with reasons in research-2026-10-01.md): a public fix commit
distinct from its parent, the advisory-named function declared on both
sides with a changed body. Pointer pairs: materialize.py reads one file
per side from a shallow blobless fetch into the pair cache and verifies
them with the label builder's matcher (verification.json, bytes read).

learn/sources.toml lists the catalog as the advisory-fixes source. Label
snapshot: negative repository groups untrusted_destination 8->25, path
3->21, output_encoding 1->22, deserialization 0->21, access_control 0->20;
all five reach the per-family floor. Five pairs on reserved repositories
were dropped after the reserved-population guard flagged them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-01 11:51:28 +02:00
co-authored by Claude Opus 5.5
parent 658f2d5e39
commit 53c9a7cc65
8 changed files with 5081 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/openultrasast/learn/sources.toml
+14
View File
@@ -56,6 +56,20 @@ recorded = "2026-09-30"
status = "development"
exclude_tiers = ["title"] # labels read from advisory titles; a sensitivity arm only (`--include-title`)
# Real fix-commit pairs for the families the pair corpus left without negatives (untrusted_destination, path,
# output_encoding, deserialization, access_control). Each pair is a public advisory fix commit distinct from its parent;
# the advisory-named function is declared on both sides and its body changes. Pointer pairs: both sides are
# materialised into the pair cache by benchmarks/pairs/advisory-fixes/materialize.py (shallow blobless fetch, one
# file per side); `evaluation` is the committed per-pair verification record (bytes read, declared both sides).
[[source]]
id = "advisory-fixes"
kind = "pairs"
file = "benchmarks/pairs/advisory-fixes/catalog.toml"
evaluation = ["benchmarks/pairs/advisory-fixes/verification.json"]
recorded = "2026-10-01"
status = "development"
exclude_tiers = ["title"]
[[source]]
id = "adjudications"
kind = "adjudications"