marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-17 03:59:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

learn: 251 more advisory-fix pairs whose function survives the fix (advisory-fixes-3)

Browse Source 
Batch 3 of real (vulnerable function, fixed function) pointer pairs, mined at
zero model cost from reviewed GHSA advisories (pip, npm, maven, composer,
published since 2023, newest first). Same rules as batch 2, widened three
ways: a merged PR's single commit when the advisory links no commit, commits
up to 6 files / 240 lines, and two-family advisories with the family read
from the diff. Every candidate's function diff was read: 412 reviewed, 255
kept, 3 one-per-repository, 2 withheld by the reserved-population guard
(bisected with output discarded). Permissive licences only; no repository
named elsewhere under benchmarks/; no WordPress plugin.

Kept 251: injection 56, untrusted_destination 26, path 71, output_encoding
69, access_control 20, deserialization 9; python 83, typescript 48, php 46,
java 37, javascript 37. Only 12 post-cutoff: batch 2 had used up the
post-2026-06-01 pool. Memory 1133 -> 1635 static examples; complete pairs in
the whole memory 459 -> 710. Embeddings: 15 calls, $0.0030. No chat-model
call, no Docker, no Joern. Record:
benchmarks/measurements/2026-10-04-decision-engine-harvest-3.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-04 11:57:34 +02:00
co-authored by Claude Opus 5.5
parent 20777d8863
commit cbc252553a
5 changed files with 10671 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/openultrasast/learn/sources.toml
+12
View File
@@ -82,6 +82,18 @@ recorded = "2026-10-03"
status = "development"
exclude_tiers = ["title"]
# Batch 3: more advisory fixes whose function survives the fix, weighted toward injection and untrusted_destination.
# Same slice, cache layout and checks; screening record
# benchmarks/measurements/2026-10-04-decision-engine-harvest-3/record.json.
[[source]]
id = "advisory-fixes-3"
kind = "pairs"
file = "benchmarks/pairs/advisory-fixes-3/catalog.toml"
evaluation = ["benchmarks/pairs/advisory-fixes-3/verification.json"]
recorded = "2026-10-04"
status = "development"
exclude_tiers = ["title"]
[[source]]
id = "adjudications"
kind = "adjudications"