test(pre-push-safety-net): record the M1b production comparison on NodeGoat

Rerunning the three pinned revisions after tasks 10.1-10.6 shows the production
comparison classifying all of them with no lifting, and agreeing exactly with the
recorded view. That agreement is the point: the vetoes M1a recorded were repaired
rather than bypassed.

The vulnerable revision is new with operation_absent_from_comparable_base, and the
three eval witnesses at contributions.js lines 32 to 34 are attributed to the
change that introduced them. The fixed twin is analyzed and quiet with three
base-only findings. The unchanged revision is unchanged. In M1a every one of the
436 questions was demoted on both sides and production reported
witness_identity_unresolved on every finding.

All three artifacts still report finding_status none with zero admitted defects.
Admission rejects on capability_unavailable, the deliberate eligibility gate, and
on dependency_unresolved, which the rerun exposed as coarse plumbing: the runner
passes every coverage reason as every candidate's dependency gaps. That is now
task 10.7, ahead of M4, since it would block admission forever once capabilities
qualify. It is not a comparison failure and not an M1b exit blocker.

Three comparisons on one inspected teaching repository under a development budget.
No capability enabled, no alert emitted, no latency claim. Runtime, independent
qualification and rollout all remain NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-16 18:06:39 +02:00
co-authored by Claude Opus 5
parent bc68000e93
commit b8b46f1c19
8 changed files with 2327 additions and 31 deletions
+8 -9
View File
@@ -15,7 +15,7 @@ not a demonstrated useful safety net. Task 8.3 and rollout remain NO-GO.
| Order | Milestone | Observable exit | Current state |
|---|---|---|---|
| M1a | Show the NodeGoat detection with every admission veto recorded, not applied | Raw finding with witness on the vulnerable revision; analyzed fixed twin and unchanged control quiet; per-finding veto list. One-week time box | MET 2026-09-16 |
| M1b | Repair only the vetoes M1a showed to matter | Supported evaluation finding through production rules, exact change/base evidence and repair direction; fixed twin and unchanged control stay quiet | IN PROGRESS: 10.1-10.3 verified; 10.4 operation inventory now blocks the rest |
| M1b | Repair only the vetoes M1a showed to matter | Supported evaluation finding through production rules, exact change/base evidence and repair direction; fixed twin and unchanged control stay quiet | IN PROGRESS: every recorded veto repaired and the production comparison classifies all three revisions; 10.7-10.9 remain |
| M2 | Make that same analysis fast | Equivalent evidence; representative warm changed-code p95 <=30s, cancellation/report <=2s, >=95% supported completion | Pending M1 |
| M3 | Prove the shared design transfers | PMPro/PHP and VAmPI/Python vulnerable/fixed evidence; unchanged core rules; Node replay after any PMPro core change | Pending M2 |
| M4 | Qualify on independent cases | Reviewed untouched vulnerable/fixed/benign populations pass joint quality, coverage and latency gates per capability | Pending M3 |
@@ -23,14 +23,13 @@ not a demonstrated useful safety net. Task 8.3 and rollout remain NO-GO.
### Immediate next action
Export the enumerated operation inventory separately from traced flows (task 10.4), then
finish the last two vetoes it unblocks and rerun the three NodeGoat revisions through the
production rules. Operation correspondence, required comparison scope and ambiguity scoping
are already verified and measured. The inventory is needed because a taint answer reports
flows: an absent row means either that the operation does not exist or that no traced flow
reached it, and the veto that stops those being confused is always true on a real Express
application, so it can neither be discharged nor safely removed. Vetoes that did not fire
stay unchanged and their families stay experimental. Normal hook eligibility stays disabled.
Finish M1b: bind candidate dependency gaps to the candidate (10.7), carry query-to-operation
provenance (10.8), then rerun and record the three NodeGoat revisions as the milestone's exit
evidence (10.9). Every veto M1a recorded is repaired and the production comparison already
classifies all three revisions correctly, with production and the recorded view agreeing. The
dependency-gap defect is coarse plumbing rather than a comparison failure, but it would block
admission forever once capabilities qualify, so it belongs before M4. Normal hook eligibility
stays disabled and no capability is enabled by any of this.
Do not start another broad speed campaign before this outcome exists. See the
[M1a evidence](../specs/pre-push-safety-net/m1a-recorded-vetoes.md) and the
[owner diagnosis](../specs/contributor-scan/release-remediation-research.md).