mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 5.5
800204de87
Reran benchmarks/harnessx_removal_equality.py record from a git archive export ofb9afb66(tasks 1-6 landed; same venv freeze hash as the baseline, harnessx not importable, no API keys, empty config) and compared it against 2026-09-30-harnessx-removal-baseline: equal=True differences=0 All 13 output files (pairs.json, benchmark, standard scan, improve --dry-run) are raw-byte identical to the baseline. The suite map (suite-map.json) accounts for 29 deleted ids, 2 renamed ids and 83 added ids; each deleted or renamed id cites the design's Tests table and commitcbadc38orb9afb66. Every surviving test id keeps its outcome (1740 passed, 38 skipped incl. the README xfail). Instrument note: a first record failed test_push_hook twice because the recording shell's PATH lacked .venv/bin (the hook execs `ousast`); the same two tests fail atb9afb66without it and pass with it. The committed record has .venv/bin on PATH. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Equality proof for harnessx-removal (task 7, Requirement 3.2), recorded 2026-09-30.
candidate/ `harnessx_removal_equality.py record` from a `git archive` export of b9afb66 (tasks 1-6 landed),
same venv (freeze sha256 0e5ad419... identical to the baseline's), no API keys, empty config.
suite-map.json the suite ids deleted (29), renamed (2) and added (83) since the baseline, each deleted or renamed
id justified by the design's Tests table and commit cbadc38 or b9afb66.
comparison.json `compare ../2026-09-30-harnessx-removal-baseline candidate --suite-map suite-map.json`:
equal=True differences=0.
Beyond the comparison: every output file (pairs, benchmark, scan, improve) is raw-byte identical to the baseline
(`diff -rq` differs only in suite.json and env.json; determinism.json is baseline-only), and every surviving test id keeps its outcome.
Instrument note: the first record failed the two test_push_hook cases (exit 1, prior-input never written) because
the recording shell's PATH lacked .venv/bin, so the hook's `exec ousast` could not start. Same failure at b9afb66
in the worktree, gone with .venv/bin on PATH (5 passed). The recorded run has .venv/bin on PATH; the baseline
passed these tests, so its PATH had `ousast` too.