test(pre-push-safety-net): record the M1b production comparison on NodeGoat

Rerunning the three pinned revisions after tasks 10.1-10.6 shows the production
comparison classifying all of them with no lifting, and agreeing exactly with the
recorded view. That agreement is the point: the vetoes M1a recorded were repaired
rather than bypassed.

The vulnerable revision is new with operation_absent_from_comparable_base, and the
three eval witnesses at contributions.js lines 32 to 34 are attributed to the
change that introduced them. The fixed twin is analyzed and quiet with three
base-only findings. The unchanged revision is unchanged. In M1a every one of the
436 questions was demoted on both sides and production reported
witness_identity_unresolved on every finding.

All three artifacts still report finding_status none with zero admitted defects.
Admission rejects on capability_unavailable, the deliberate eligibility gate, and
on dependency_unresolved, which the rerun exposed as coarse plumbing: the runner
passes every coverage reason as every candidate's dependency gaps. That is now
task 10.7, ahead of M4, since it would block admission forever once capabilities
qualify. It is not a comparison failure and not an M1b exit blocker.

Three comparisons on one inspected teaching repository under a development budget.
No capability enabled, no alert emitted, no latency claim. Runtime, independent
qualification and rollout all remain NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-16 18:06:39 +02:00
co-authored by Claude Opus 5
parent bc68000e93
commit b8b46f1c19
8 changed files with 2327 additions and 31 deletions
+11 -3
View File
@@ -316,18 +316,26 @@ Preserve all completed task evidence below.
- _Depends: 10.5_
- _Requirements: 2.2, 2.3, 8.2, 8.4_
- _Verified 2026-09-16:_ 1,365 tests passed, nine skipped; Ruff/format/mypy passed. Graph boundaries are split into integrity failures, which still demote a completed answer, and declared exclusions, which do not. Whether a particular answer needed the excluded semantics is carried by that question's own unresolved call destinations from 10.5, not assumed for every answer. Controls exercise the real partition logic rather than an injected reason: a vendored directory leaves the answer completed with the exclusion still reported, an unresolved symlink still demotes, and the two together still demote. The exclusion stays physical and out of targets and graphs. The NodeGoat rerun showed the same distinction was still missing inside comparison, where any degradation and any boundary blocked: comparison now filters declared exclusions and correspondence ambiguity on both sides, keeping integrity degradations and every other boundary blocking.
- [ ] 10.7 Carry query-to-operation provenance into the reported evidence
- [ ] 10.7 Bind candidate dependency gaps to the candidate
- **Discovered 2026-09-16 in the M1b production rerun.** The runner passes every coverage reason as every candidate's `dependency_gaps`, so admission always answers `dependency_unresolved`. That is not "an unresolved dependency that undermines the claim"; it is every gap anywhere in the transaction.
- Supply the gaps that pertain to this candidate's required comparison scope, the same scope 10.2 established. Unknown ownership still counts against the candidate.
- Not an M1b exit blocker, since eligibility stays disabled and no alert is emitted either way. It would prevent admission from ever passing after qualification, so it belongs before M4.
- Done when an unrelated transaction gap no longer appears as this candidate's unresolved dependency, while a gap in its required scope still does.
- _Boundary: Push runner, Delta and Actionability Policy_
- _Depends: 10.2_
- _Requirements: 3.1, 3.3, 3.4_
- [ ] 10.8 Carry query-to-operation provenance into the reported evidence
- The target is selected through a file-fallback question with `function=None` while rows report method `<lambda>2`. M1a showed this did not block the comparison once the question completed, so it is provenance quality rather than the blocker.
- Record verified query-to-operation provenance on the evidence; do not inject benchmark function names into regions or count coincident sites.
- Done when a reported operation names the question that produced it and the method the engine reported, with unresolved provenance explicit.
- _Boundary: Delta and Actionability Policy, contributor-scan normalization_
- _Requirements: 3.1, 3.4, 6.3_
- [ ] 10.8 Rerun the three revisions through the production rules
- [ ] 10.9 Rerun the three revisions through the production rules
- Replay the same pinned vulnerable, fixed and unchanged revisions **without** the recorded-veto flag under the named development budget. Demonstrate a developer-readable evaluation finding with a supported consequence, exact location, witness, change attribution and repair direction.
- Verify the fixed twin is analyzed and produces no actionable defect and the unchanged revision is not reported as a new regression. Keep qualification eligibility disabled; experimental evaluation output cannot advertise an unqualified hook alert.
- Done when one replayable record states which vetoes were repaired and which remain, with completed required checks and readable source and census receipts.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7_
- _Depends: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8_
- _Requirements: 1.1, 2.1, 2.4, 3.1, 3.4, 6.3, 7.1, 7.3_
## Implementation Notes