marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-22 14:39:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

bench(harnessx-removal): deterministic baseline without the extra (task 1)

Browse Source 
Equality script (design section 3) records suite outcomes, `ousast pairs --json`,
the python-vulnerable quick benchmark, one standard scan and `improve --dry-run`
from a frozen export with the harnessx extra absent and no model reachable.
Two independent records are byte-identical (determinism.json).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-30 10:42:57 +02:00
co-authored by Claude Opus 5.5
parent 2ed6d9a036
commit ca5caf735f
20 changed files with 22358 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/measurements/2026-09-30-harnessx-removal-baseline/README
+1
View File
@@ -0,0 +1 @@
Excluded from comparison: JSON keys: created_at, finished, generated_at, runtime_seconds, scan_id, started, timestamp; JSON keys ending in: _seconds, _ms; run directory names / run ids (<timestamp>-<8 hex>, benchmark.py create_benchmark_run, run.py create_scan_run) -> <RUN_ID>; trace/events.jsonl as a whole (timings); never copied; the export's absolute root path -> <ROOT>; the children's TMPDIR -> <TMP>; suite: only the outcome per test id is kept (junit durations dropped); env.json is a provenance record (commit, freeze hash, instrument checks), not compared