marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-11-13 17:58:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

measure(prover): gate 1 on the 23-repo training pool: recall 65.2% (Wilson 44.9-81.2), 15.0 FA/100, 0 instrument failures; 4/5 on the hunt overlap incl. trigger.dev

Browse Source
This commit is contained in:
Marius Ciepluch
2026-10-28 18:59:01 +01:00
parent 03d925fffa
commit c5341d79a2
4 changed files with 1355 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/access-control-prover/tasks.md
+5 -1
View File
@@ -107,10 +107,14 @@
- _Depends: 4.1_
- [ ] 6. Measurement (maintainer, not Codex)
- [ ] 6.1 Run the arm on `benchmarks/valid/train/pool-train.toml` over the training clones; write
- [x] 6.1 Run the arm on `benchmarks/valid/train/pool-train.toml` over the training clones; write
`benchmarks/measurements/<date>-access-control-prover/` with NOTES, summary, coverage table; compare with the
DSPy and hunter arms of the same pool
- _Requirements: 5.2, 5.3, 7.2_
- Done 2026-10-28: `benchmarks/measurements/2026-10-28-access-control-prover/` -- 23-repo
pool (483 changes, 93 locations), recall 65.2% (Wilson 44.9-81.2), 15.0 FA/100, 0
instrument failures; on the 5 repos shared with the 2026-10-14 hunt arms, prover 4/5 vs
DSPy 2/5 and hunter 2/5 (union 4/5), including trigger.dev which neither arm found.
## Implementation Notes