feat(pre-push-safety-net): replay immutable revisions through the full transaction

This commit is contained in:
Marius Ciepluch
2026-09-13 14:50:25 +02:00
parent 6b12b73411
commit 31e5ec7430
14 changed files with 4589 additions and 29 deletions
@@ -0,0 +1,66 @@
# Implementation verification: group 5
Group 5 was authorized by the maintainer's “approved” after group 4 completion.
Task 5.1 is verified complete (15/27 total); task 5.2 is next.
Real capability eligibility and rollout readiness remain unproven.
## Execution and review mode
Fresh implementer dispatch for 5.1 returned `agent thread limit reached`.
The [Kiro implementation skill](../../../.opencode/skills/kiro-impl/SKILL.md) explicitly
states: “If multi-agent is not available, fall back to manual mode execution for all tasks.”
Its manual review rule permits review in the main context when a fresh reviewer is unavailable.
This run therefore uses manual implementation and adversarial review, not independent review.
## Task 5.1 brief
Wire explicit immutable base/head replay through the existing mapper, ranker, driver,
base comparison and admission/report contracts. Preserve the shared elapsed deadline,
source/index/ref state, completed evidence on failure, and separate coverage/exit policy.
Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3. Design sections are named:
Snapshot Adapter, Ranker Scope Contract, Delta and Actionability Policy, Result and Hook
Adapter, Cache and Engine Boundary. No new detector, ranking weights, framework facts,
cache or hook installation belongs to this task.
### Integration findings
- Snapshot resolution previously used fresh per-command timeouts. An optional shared
execution budget now covers constructor probes and explicit revision resolution.
- Discovery and provenance are bounded in a Linux child process. The parent drains output
incrementally, checks the deadline, kills a stalled worker and requires a valid receipt;
exit zero with no result is failure. No project code or engine runs in this child.
- The first real controlled PHP replay found the exact request-to-eval witness but novelty
was unknown: comparison policy treated another question's context gap as global. The
owning task 4.1 policy was repaired here with a failing regression first. Only recognized
boundaries attached to a different recorded question can be ignored for that candidate;
aggregate coverage retains every gap. Own-question and global boundaries still prevent
supported novelty. This is policy ownership repair, not a new upstream graph abstraction.
- Artifacts must resolve outside the analyzed repository. The runner refuses a destination
that could overwrite live source/index/ref state and preserves the real enforcement result.
### Evidence in progress
Initial OFF-state tests: `/tmp/ousast-replay-red.log`, six failures before implementation.
Policy ownership regression: `/tmp/ousast-replay-policy-red.log`, one failure before repair.
Live-source overwrite regression: `/tmp/ousast-replay-review-red.log`, reproduced before repair.
Final validation: 1,236 tests passed, nine skipped; Ruff/format, mypy (112 source files), compileall and all three regression gates passed. Final packaged CLI boot and the controlled positive replay passed. See `benchmarks/measurements/2026-09-13-replay-smoke.json`.
Additional integration repairs retain snapshot omissions in engine change context and distinguish
base questions proven tier zero from unfinished queries. The exact required base question still
must have completed, valid evidence; budget-deferred work still prevents a complete comparison.
RED evidence: `/tmp/ousast-replay-tierzero-red.log` and `/tmp/ousast-replay-snapshot-red.log`.
## Task 5.1 review verdict
- VERDICT: APPROVED
- TASK: 5.1
- MECHANICAL_RESULTS: Full suite exit 0 (1,236 passed, 9 skipped); Ruff/format/mypy/compileall and detection/map/pair gates PASS; no new placeholder markers or secrets; boundary WITHIN; behavioral RED VERIFIED.
- FINDINGS: Integration defects above repaired with regressions; no outstanding task-blocking finding.
- SUMMARY: Manual adversarial review approves bounded local replay; this is not independent review or rollout admission.
## Task 5.1 verification
- STATUS: VERIFIED
- CLAIM_TYPE: TASK
- CLAIM: Explicit local replay traverses the production mapper/ranker/arbiter/comparison/report path and retains uncertainty and shared deadlines.
- EVIDENCE: Final packaged controlled security change yields new/source_connection_absent_from_comparable_base. Fixed twin emits no modeled head finding. The packaged deadline control returns incomplete in 30.785 seconds, within 30 + 2 seconds. Exact final package policy/core/facts/query digests match the workspace. Complete commands and captured raw answers live in the measurement artifact.
- GAPS: Aggregate coverage remains incomplete, real capabilities are disabled, and lab timings do not satisfy the latency gate. Source-mounted fixed and earlier packaged deadline provenance is explicitly separated in the artifact. PMPro/Node transfer is task 5.2, not inferred from a small PHP control.
+13 -6
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-13T11:21:07.667199+00:00",
"updated_at": "2026-09-13T12:50:25.249104+00:00",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -19,7 +19,7 @@
}
},
"ready_for_implementation": true,
"implementation_approval_note": "Groups 1\u20134 verified complete. Tasks 4.1/4.2 independently approved; 4.3 used the documented Kiro manual review fallback after agent thread limit. Remaining tasks require review and fresh verification.",
"implementation_approval_note": "Group 5 authorized. Task5.1 verified through manual Kiro fallback review; 5.2 next. Real capabilities remain disabled.",
"depends_on": [
"contributor-scan"
],
@@ -60,6 +60,11 @@
"at": "2026-09-13T10:26:47.307154+00:00",
"source": "move on to group 4 as a whole",
"action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending."
},
{
"at": "2026-09-13T12:40:57.961577+00:00",
"source": "approved",
"action": "Authorized group 5 after the group 4 completion summary: explicit transaction replay, then the first PMPro/frozen-core Node experiment. Review and fresh evidence remain required."
}
],
"task_review": {
@@ -86,15 +91,17 @@
"3.4",
"4.1",
"4.2",
"4.3"
"4.3",
"5.1"
],
"remaining_tasks": 13,
"verification_record": "implementation-group-4.md",
"remaining_tasks": 12,
"verification_record": "implementation-group-5.md",
"verification_records": [
"implementation-group-1.md",
"implementation-group-2.md",
"implementation-group-3.md",
"implementation-group-4.md"
"implementation-group-4.md",
"implementation-group-5.md"
],
"next_group": 5,
"blocked_tasks": []
+3 -2
View File
@@ -3,7 +3,7 @@
Status: task plan approved by the maintainer on 2026-09-12. Groups 1–4 (tasks 1.1–4.3)
are complete after task reviews and fresh verification (14/27 subtasks). Task 4.3 used
the documented manual review fallback after agent dispatch became unavailable.
Groups 5–8 remain pending; no production capability or hook latency is admitted.
Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pending; no production capability or hook latency is admitted.
## Execution contract
@@ -140,10 +140,11 @@ Groups 5–8 remain pending; no production capability or hook latency is admitte
- _Requirements: 3.1, 3.2, 3.3, 5.1, 5.2, 5.5, 7.3_
- [ ] 5. Prove the first end-to-end experiment before cache optimization
- [ ] 5.1 Wire explicit base/head replay through the full transaction
- [x] 5.1 Wire explicit base/head replay through the full transaction
- Connect immutable snapshots, ranker decisions, existing arbiter, targeted comparison, admission diagnostics and compact artifacts through an explicit local replay command.
- Apply the shared budget to preparation and both revisions; no model/service or installed hook is required. All runtime errors preserve coverage state.
- Done when a controlled security change and fixed twin traverse the production runner through the existing ranker and produce replayable results with truthful deadlines.
- _Verified 2026-09-13:_ manual Kiro fallback review APPROVED; 1,236 tests passed, nine skipped; final packaged security replay and fixed/deadline controls verified. See `implementation-group-5.md`.
- _Boundary: Push runner, CLI, Snapshot Adapter, engine and policy integration_
- _Depends: 1.3, 3.1, 3.4, 4.3_
- _Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3_