mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 19:59:26 +00:00
learn: 98 advisory-fix pairs whose function survives the fix (advisory-fixes-2)
Real (vulnerable function, fixed function) pointer pairs for the six compiled families, mined at zero model cost from reviewed GHSA advisories (pip, npm, maven, composer), newest first: 79 of 98 published after 2026-06-01 (post_cutoff = true). Each pair: one linked one-parent fix commit, <= 4 source files / 120 lines, the function holding the changed code lines declared once on both sides with a changed body; every function diff read and kept only when the named function holds the fix (145 reviewed, 100 kept); 2 dropped by the reserved-population test; permissive licences only, no repository named elsewhere under benchmarks/. New label source advisory-fixes-2; the advisory-fix tests run over both batches. Memory 937 -> 1133 static examples (98 paired groups added); positives-only groups stay 99 (vibe-py/dev-php/v2 data gaps), their share of positive groups in the six families 0.40 -> 0.29. Embeddings: 7 calls, $0.0014. No chat-model call. Record: benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
432fd2947e
commit
50c93edfb8
@@ -70,6 +70,18 @@ recorded = "2026-10-01"
|
||||
status = "development"
|
||||
exclude_tiers = ["title"]
|
||||
|
||||
# Batch 2: advisory fixes whose changed function survives on the fixed side, for the six compiled families (most
|
||||
# published after 2026-06-01). Same slice, cache layout and checks as batch 1; screening record
|
||||
# benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest/record.json.
|
||||
[[source]]
|
||||
id = "advisory-fixes-2"
|
||||
kind = "pairs"
|
||||
file = "benchmarks/pairs/advisory-fixes-2/catalog.toml"
|
||||
evaluation = ["benchmarks/pairs/advisory-fixes-2/verification.json"]
|
||||
recorded = "2026-10-03"
|
||||
status = "development"
|
||||
exclude_tiers = ["title"]
|
||||
|
||||
[[source]]
|
||||
id = "adjudications"
|
||||
kind = "adjudications"
|
||||
|
||||
Reference in New Issue
Block a user