mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 5.5
800204de87
Reran benchmarks/harnessx_removal_equality.py record from a git archive export ofb9afb66(tasks 1-6 landed; same venv freeze hash as the baseline, harnessx not importable, no API keys, empty config) and compared it against 2026-09-30-harnessx-removal-baseline: equal=True differences=0 All 13 output files (pairs.json, benchmark, standard scan, improve --dry-run) are raw-byte identical to the baseline. The suite map (suite-map.json) accounts for 29 deleted ids, 2 renamed ids and 83 added ids; each deleted or renamed id cites the design's Tests table and commitcbadc38orb9afb66. Every surviving test id keeps its outcome (1740 passed, 38 skipped incl. the README xfail). Instrument note: a first record failed test_push_hook twice because the recording shell's PATH lacked .venv/bin (the hook execs `ousast`); the same two tests fail atb9afb66without it and pass with it. The committed record has .venv/bin on PATH. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
17 lines
1.3 KiB
Plaintext
17 lines
1.3 KiB
Plaintext
Equality proof for harnessx-removal (task 7, Requirement 3.2), recorded 2026-09-30.
|
|
|
|
candidate/ `harnessx_removal_equality.py record` from a `git archive` export of b9afb66 (tasks 1-6 landed),
|
|
same venv (freeze sha256 0e5ad419... identical to the baseline's), no API keys, empty config.
|
|
suite-map.json the suite ids deleted (29), renamed (2) and added (83) since the baseline, each deleted or renamed
|
|
id justified by the design's Tests table and commit cbadc38 or b9afb66.
|
|
comparison.json `compare ../2026-09-30-harnessx-removal-baseline candidate --suite-map suite-map.json`:
|
|
equal=True differences=0.
|
|
|
|
Beyond the comparison: every output file (pairs, benchmark, scan, improve) is raw-byte identical to the baseline
|
|
(`diff -rq` differs only in suite.json and env.json; determinism.json is baseline-only), and every surviving test id keeps its outcome.
|
|
|
|
Instrument note: the first record failed the two test_push_hook cases (exit 1, prior-input never written) because
|
|
the recording shell's PATH lacked .venv/bin, so the hook's `exec ousast` could not start. Same failure at b9afb66
|
|
in the worktree, gone with .venv/bin on PATH (5 passed). The recorded run has .venv/bin on PATH; the baseline
|
|
passed these tests, so its PATH had `ousast` too.
|