marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 11:49:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(plane): handover -- ops README points at the ax runbook; replaced scripts marked reference-only (task 9)

Browse Source 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-30 08:39:24 +02:00
co-authored by Claude Opus 5.5
parent 67699e8737
commit df1dfa3b7a
4 changed files with 15 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/independent/score_batched.py
+5 -1
View File
@@ -1,4 +1,8 @@
"""Exploratory score of a batched-verifier run against the population's cases (protocol v2 matching).
"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax
through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/
plane-increment-2.json). It stays as the reference behaviour the plane was measured against.
Exploratory score of a batched-verifier run against the population's cases (protocol v2 matching).
The batched verifier writes one vulnerable-pin result per case (and a fixed-pin recheck with --fixed), not the
six pins protocol v2 scores, so this applies the same matching rule (`evaluate.matches_v2`) to what exists:
benchmarks/independent/verify_batched.py
+5 -1
View File
@@ -1,4 +1,8 @@
"""Stage 2, second design: triage per file, verify per file, report only what two passes agree on.
"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax
through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/
plane-increment-2.json). It stays as the reference behaviour the plane was measured against.
Stage 2, second design: triage per file, verify per file, report only what two passes agree on.
The first verifier (`verify_sinks.py`) asked one 6-step tool hunt per candidate function. Measured on pgAdmin
(verifier-check-2026-09-29.json): $0.014-0.020 per candidate, and on 180 candidates verified twice the two runs