mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
test(pre-push-safety-net): record the M1b production comparison on NodeGoat
Rerunning the three pinned revisions after tasks 10.1-10.6 shows the production comparison classifying all of them with no lifting, and agreeing exactly with the recorded view. That agreement is the point: the vetoes M1a recorded were repaired rather than bypassed. The vulnerable revision is new with operation_absent_from_comparable_base, and the three eval witnesses at contributions.js lines 32 to 34 are attributed to the change that introduced them. The fixed twin is analyzed and quiet with three base-only findings. The unchanged revision is unchanged. In M1a every one of the 436 questions was demoted on both sides and production reported witness_identity_unresolved on every finding. All three artifacts still report finding_status none with zero admitted defects. Admission rejects on capability_unavailable, the deliberate eligibility gate, and on dependency_unresolved, which the rerun exposed as coarse plumbing: the runner passes every coverage reason as every candidate's dependency gaps. That is now task 10.7, ahead of M4, since it would block admission forever once capabilities qualify. It is not a comparison failure and not an M1b exit blocker. Three comparisons on one inspected teaching repository under a development budget. No capability enabled, no alert emitted, no latency claim. Runtime, independent qualification and rollout all remain NO-GO. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5
parent
bc68000e93
commit
b8b46f1c19
@@ -15,7 +15,7 @@ not a demonstrated useful safety net. Task 8.3 and rollout remain NO-GO.
|
||||
| Order | Milestone | Observable exit | Current state |
|
||||
|---|---|---|---|
|
||||
| M1a | Show the NodeGoat detection with every admission veto recorded, not applied | Raw finding with witness on the vulnerable revision; analyzed fixed twin and unchanged control quiet; per-finding veto list. One-week time box | MET 2026-09-16 |
|
||||
| M1b | Repair only the vetoes M1a showed to matter | Supported evaluation finding through production rules, exact change/base evidence and repair direction; fixed twin and unchanged control stay quiet | IN PROGRESS: 10.1-10.3 verified; 10.4 operation inventory now blocks the rest |
|
||||
| M1b | Repair only the vetoes M1a showed to matter | Supported evaluation finding through production rules, exact change/base evidence and repair direction; fixed twin and unchanged control stay quiet | IN PROGRESS: every recorded veto repaired and the production comparison classifies all three revisions; 10.7-10.9 remain |
|
||||
| M2 | Make that same analysis fast | Equivalent evidence; representative warm changed-code p95 <=30s, cancellation/report <=2s, >=95% supported completion | Pending M1 |
|
||||
| M3 | Prove the shared design transfers | PMPro/PHP and VAmPI/Python vulnerable/fixed evidence; unchanged core rules; Node replay after any PMPro core change | Pending M2 |
|
||||
| M4 | Qualify on independent cases | Reviewed untouched vulnerable/fixed/benign populations pass joint quality, coverage and latency gates per capability | Pending M3 |
|
||||
@@ -23,14 +23,13 @@ not a demonstrated useful safety net. Task 8.3 and rollout remain NO-GO.
|
||||
|
||||
### Immediate next action
|
||||
|
||||
Export the enumerated operation inventory separately from traced flows (task 10.4), then
|
||||
finish the last two vetoes it unblocks and rerun the three NodeGoat revisions through the
|
||||
production rules. Operation correspondence, required comparison scope and ambiguity scoping
|
||||
are already verified and measured. The inventory is needed because a taint answer reports
|
||||
flows: an absent row means either that the operation does not exist or that no traced flow
|
||||
reached it, and the veto that stops those being confused is always true on a real Express
|
||||
application, so it can neither be discharged nor safely removed. Vetoes that did not fire
|
||||
stay unchanged and their families stay experimental. Normal hook eligibility stays disabled.
|
||||
Finish M1b: bind candidate dependency gaps to the candidate (10.7), carry query-to-operation
|
||||
provenance (10.8), then rerun and record the three NodeGoat revisions as the milestone's exit
|
||||
evidence (10.9). Every veto M1a recorded is repaired and the production comparison already
|
||||
classifies all three revisions correctly, with production and the recorded view agreeing. The
|
||||
dependency-gap defect is coarse plumbing rather than a comparison failure, but it would block
|
||||
admission forever once capabilities qualify, so it belongs before M4. Normal hook eligibility
|
||||
stays disabled and no capability is enabled by any of this.
|
||||
Do not start another broad speed campaign before this outcome exists. See the
|
||||
[M1a evidence](../specs/pre-push-safety-net/m1a-recorded-vetoes.md) and the
|
||||
[owner diagnosis](../specs/contributor-scan/release-remediation-research.md).
|
||||
|
||||
Reference in New Issue
Block a user