marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-18 04:29:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

bench(learn): harvest the advisory-fix pairs

Browse Source 
The 97 advisory-fix pairs (194 sides) harvested for the decision engine: verify a/b + agree on the 34
untrusted_destination sides and model roles on every side as plane Runs on ax ($0.38 metered, task
budgets $1.49, ceiling $5; DeepSeek balance 17.88 -> 17.73), engine features on the host (frozen
7b17eac, one container, --memory 3g: 54 ran, 58 asked no question, 82 no language model), feature
records for all 194 sides, static memory rebuilt with the merged harvest units: 724 -> 904 examples.
No unit was skipped for the 20 KB inline limit.

Instrument fix: SideReader read only the first pair-kind source, so the first rebuild left all 194
advisory sides `unread` (0 examples). It now reads every pair-kind source; test added.

Leak audit: the static profile flags only engine signals and facts.function_lines (withheld by v1);
the plane profile newly flags verify.flag_b and ms.operation (plane-only, not in any v1 prompt).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 09:40:00 +02:00
co-authored by Claude Opus 5.5
parent 7b17eacd44
commit d9608b9a63
4 changed files with 817 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/learned-decision-engine/tasks.md
+4
View File
@@ -38,6 +38,10 @@ a model or embedding call.
- Done 2026-10-01 (`benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`): verify $2.54 and roles
$1.40 by attribution. The engine job finished the same day (682 of 686 pair sides; the 2 family-unknown pairs
not run) and the feature records were rebuilt; dev-php pins have no engine run (`missing`).
- Advisory-fix pairs harvested 2026-10-02 (`benchmarks/measurements/2026-10-02-decision-engine-harvest-advisory/record.json`):
194 sides, verify a/b on the 34 untrusted_destination sides and model roles on all ($0.38 metered, ceiling $5),
engine on the host (54 ran, 58 asked no question, 82 no language model); static memory 724 -> 904 examples. The
leak audit flags nothing new in the static profile; the plane profile newly flags `verify.flag_b` and `ms.operation`.
- _Requirements: 1.1_
- [ ] 6. Program, local memory and compilation (design sections 4-5)