mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pre-push-safety-net): record admission vetoes and meet M1a on NodeGoat
Add group 9 to the approved task plan as the bounded owner amendment release milestone M1a requires, implement the task-local recorded-veto evaluation flag, and run the three pinned NodeGoat revisions under it. The flag re-runs the production comparison and admission over a lifted copy of the same evidence and records each veto instead of applying it. Recognized completeness vetoes are lifted and logged; failures never are, because silence from a failure is not a quiet fixed twin. It is off by default, accepted only on explicit base/head replay, refused in Git stdin mode, labels every payload it produces, and admits no defect even when a lifted candidate otherwise would. M1a outcome, inside its one-week box and with expectations frozen on disk first: the three entailed <lambda>2 witnesses are present at contributions.js 32-34; the fixed twin was genuinely analyzed and quiet, answering its target question with zero rows and no deadline reason while the same findings appear as base-only; the unchanged revision was analyzed and recorded unchanged. The kill criterion did not trigger. Five vetoes fired and were recorded. M1a also corrected the diagnosis: with all five lifted the detection still stops at operation_correspondence_unresolved, because a newly introduced operation sits on a changed line and has no base anchor. The unchanged revision resolves its base location on the same lines and classifies as unchanged, which isolates correspondence as the mechanism. That fifth veto was not in the earlier four-veto diagnosis and is now M1b's central repair. Diagnosis only. No defect admitted, no capability enabled, no gate changed, no latency claim; the 900-second development budget cannot measure the hook gate. Rollout and task 8.3 remain NO-GO. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5
parent
8b66d5cb75
commit
3224884d8f
@@ -189,6 +189,39 @@ Exit 1 with a complete scorecard is measured NO-GO; a missing scorecard is an in
|
||||
failure. Three samples per workload on development NodeGoat cannot independently qualify
|
||||
a production language/framework/family or establish broad multi-project performance.
|
||||
|
||||
### M1a recorded-veto diagnosis (task 9.2)
|
||||
|
||||
`m1a_record_vetoes.py` replays the pinned NodeGoat vulnerable, fixed and unchanged
|
||||
revisions through the production `ousast pre-push --base/--head` command under
|
||||
`--experimental-record-vetoes`. The flag records each comparison and admission veto per
|
||||
raw finding instead of applying it; it is off by default, refused in Git stdin mode, and
|
||||
enables no hook capability. Expected operation and outcome for all three comparisons are
|
||||
written to `m1a-record.json` before the first scan.
|
||||
|
||||
```sh
|
||||
docker run --rm --network none --memory 3g --entrypoint python \
|
||||
-v "$PWD:/context:ro" \
|
||||
-v "$HOME/.cache/openultrasast/repos:/cache:ro" \
|
||||
-v /absolute/writable/results:/results \
|
||||
openultrasast:dev /context/benchmarks/push/m1a_record_vetoes.py \
|
||||
--manifest /context/benchmarks/push/inputs.json --cache /cache \
|
||||
--out /results/new-m1a --deadline 900
|
||||
```
|
||||
|
||||
The output directory must not exist. `--deadline` is an explicitly named development
|
||||
budget: it obtains witnesses and cannot satisfy or measure the 30-second hook latency
|
||||
gate. Each case retains the full artifact, stdout/stderr, exact base/head pins, source
|
||||
byte receipts and the partition census. The record states, per case, the expectation
|
||||
declared beforehand against the observed outcome, and for every finding the exact list of
|
||||
vetoes that fired with whether each was recorded or still applied.
|
||||
|
||||
Exit **0** requires the raw witness present on the vulnerable revision, the fixed twin
|
||||
genuinely analyzed and quiet, the unchanged revision analyzed and not reported as a new
|
||||
regression, and a recorded evaluation for all three. A timed-out or unanalyzed fixed twin
|
||||
fails M1a and triggers the milestone's kill criterion; silence is never a pass. Lifted
|
||||
vetoes are diagnostic only. This run admits no defect, enables no capability, and cannot
|
||||
qualify Node, PHP or Python.
|
||||
|
||||
### Capability qualification (task 8.3)
|
||||
|
||||
The same `measure.py` replay instrument accepts `--selection development` (the
|
||||
|
||||
Reference in New Issue
Block a user