feat(pre-push-safety-net): render bounded reports and preserve full evidence

This commit is contained in:
Marius Ciepluch
2026-09-13 13:25:44 +02:00
parent d0e0ecad78
commit 6b12b73411
13 changed files with 3105 additions and 35 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Brief: pre-push-safety-net
Status: requirements, design and tasks approved on 2026-09-12; groups 1–3 verified complete (11/27 subtasks), including the approved contributor-scan frontend-retention prerequisite; groups 4–8 pending. Language: English.
Status: requirements, design and tasks approved; groups 1–4 verified complete (14/27 subtasks). Groups 5–8 pending; real capability eligibility and hook latency remain unproven. Language: English.
## Problem
@@ -4,9 +4,10 @@ Date: 2026-09-13. The maintainer authorized group 4 as a whole. Scope is tasks
4.1–4.3: targeted novelty comparison, actionability admission and compact reporting.
Groups 1–3 are verified at `6ba2b06`; full transaction replay remains task 5.1.
Status: implementation in progress. Task completion requires independent review
and fresh evidence. No production capability is enabled for normal hook alerts;
capability admission still requires task 8.3 evaluation.
Status: tasks 4.1–4.3 are VERIFIED (TASK scope), bringing the spec to 14/27 tasks.
Tasks 4.1 and 4.2 passed independent review. Task 4.3 used the Kiro manual review
fallback after the agent service rejected further dispatch. No production capability
is enabled for normal hook alerts; actual admission still requires task 8.3 evaluation.
## Verification boundary
@@ -62,3 +63,64 @@ default registry. Explicitly synthetic PHP/WordPress and JavaScript/Express cont
receive equivalent admission and enforcement behavior. They do not qualify any
real capability; task 8.3 still owns actual eligibility. Task 4.2 brings progress to
13/27 tasks. Compact reporting remains in this authorized group.
## Task 4.3 verification
The agent service rejected the task 4.3 dispatch with `agent thread limit reached`.
The [kiro-impl skill](../../../.opencode/skills/kiro-impl/SKILL.md) explicitly provides:
“If multi-agent is not available, fall back to manual mode execution for all tasks.”
Implementation, review and completion checks therefore ran in the main context.
This is a manual review record, not an independent-review claim.
OFF controls produced seven relevant failures; ON and unflagged reporting tests
passed. Manual review found and repaired an invalid artifact path escaping the
notice path and a complete comparison carrying failed base evidence. A writer
exiting zero without a receipt is rejected. Stalled-writer tests prove cancellation,
preservation of an existing artifact and no surviving child. Other controls cover
all candidate/scan preservation, five defects with only three displayed, separate
coverage/enforcement, escaping and bounded long text, and immutable provenance.
## Review Verdict
- VERDICT: APPROVED
- TASK: 4.3
- REVIEW_MODE: Kiro manual fallback; agent dispatch unavailable.
- MECHANICAL_RESULTS: final full suite 1,222 passed, nine skipped in 102.39 seconds;
Ruff and format (238 files), mypy (111 source files), compileall, diff checks and
detection/map/pair gates PASS. No new placeholders or hardcoded secrets.
- FINDINGS: Invalid destination and failed-base completeness issues repaired and
verified with failing controls followed by passing tests. No outstanding task finding.
- BOUNDARY: Reporting consumes existing admission and engine records; no new
detector, ranker, graph mutation, provider or installed hook.
The image build and CLI boot passed. The packaged integration control rechecked
captured actual PHP/JavaScript answers through current comparison/admission/reporting,
preserving complete scan payloads and default diagnostic-only output. Five explicitly
synthetic admitted defects retain all five records, display three summaries and keep
the block decision. Artifact write failure also retains that decision. Packaged
policy/reporter hashes match the workspace. See
`benchmarks/measurements/2026-09-13-compact-report-smoke.json`.
Publication is atomic from mode-0600 temporary files. A separate Linux process
bounds serialization and filesystem work by the remaining transaction/report
allowance. Hard cancellation can leave a private unpublished partial file; it is
never presented as the completed artifact. Caller-supplied stage timings are
preserved, and the delivery receipt measures publication/rendering separately.
Observed packaged delivery costs (about 0.02–0.04 seconds for these controls) do not
establish whole-hook latency.
## Group integration validation
- DECISION: GO
- VALIDATION_SCOPE: Tasks 4.1–4.3 only; no full-feature or release admission.
- CROSS_TASK_CONTRACTS: Actual captured engine evidence → bound delta → empty-registry
diagnostics → compact report and complete artifact PASS. Explicitly synthetic
calibrated controls → admission → advisory/blocking → capped rendering PASS.
- REQUIREMENTS: Group-owned portions of 2.4, 3.1–3.5, 4.1, 5.1–5.5, 7.3 and 8.1 are
covered by the implementation and controls above. Full transaction wiring and
representative capability/latency evaluation remain later tasks.
- DESIGN_ALIGNMENT: Existing arbiters and ranker retained; policy and report live
in their planned push-adapter boundary. No orphaned module or upstream detector change.
- BLOCKED_TASKS: None in group 4. Automatic dominance change-context projection is
a documented upstream integration limit, not erased by these declared controls.
- NEXT: Group 5 full replay and first bounded experiment. Complete that experiment
before persistent graph reuse in group 6. Groups 5–8 remain pending.
+6 -5
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-13T10:26:47.307154+00:00",
"updated_at": "2026-09-13T11:21:07.667199+00:00",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -19,7 +19,7 @@
}
},
"ready_for_implementation": true,
"implementation_approval_note": "Groups 1\u20133 verified complete. Maintainer explicitly authorized group 4 as a whole; each task requires independent review and fresh verification.",
"implementation_approval_note": "Groups 1\u20134 verified complete. Tasks 4.1/4.2 independently approved; 4.3 used the documented Kiro manual review fallback after agent thread limit. Remaining tasks require review and fresh verification.",
"depends_on": [
"contributor-scan"
],
@@ -85,9 +85,10 @@
"3.3",
"3.4",
"4.1",
"4.2"
"4.2",
"4.3"
],
"remaining_tasks": 14,
"remaining_tasks": 13,
"verification_record": "implementation-group-4.md",
"verification_records": [
"implementation-group-1.md",
@@ -95,7 +96,7 @@
"implementation-group-3.md",
"implementation-group-4.md"
],
"next_group": 4,
"next_group": 5,
"blocked_tasks": []
}
}
+7 -5
View File
@@ -1,8 +1,9 @@
# Implementation Plan: pre-push-safety-net
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1–3 (tasks 1.1–3.4)
are complete after independent review and fresh verification (13/27 subtasks, including tasks 4.1–4.2). Task 4.3 and groups 5–8 remain pending;
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
Status: task plan approved by the maintainer on 2026-09-12. Groups 1–4 (tasks 1.1–4.3)
are complete after task reviews and fresh verification (14/27 subtasks). Task 4.3 used
the documented manual review fallback after agent dispatch became unavailable.
Groups 5–8 remain pending; no production capability or hook latency is admitted.
## Execution contract
@@ -112,7 +113,7 @@ the hook and capability admission are not yet implemented. See `implementation-g
- _Requirements: 2.2, 2.3, 7.3, 8.2, 8.4_
- _Verified 2026-09-13:_ independent review APPROVED; 1,151 passed, nine skipped; Ruff/format/mypy pass. Real PHP/JavaScript graphs retain tests/browser source, omit vendor nodes and share one ranker limit. Named census survives PHP aggregation; missing source names and unreadable discovery inputs remain explicit. Python exercises all three batch census schemas. Evidence: `benchmarks/measurements/2026-09-13-partition-scope-smoke.json`. No hook capability or latency claim.
- [ ] 4. Implement change attribution and actionable output
- [x] 4. Implement change attribution and actionable output
- [x] 4.1 Compare targeted base/head evidence for defect novelty
- Establish stable mechanism/operation identity with rename/span mapping and collect comparable base evidence under the same semantics and remaining deadline.
- Distinguish new, worsened, unchanged and unknown; an absent finding from a failed/truncated base is unknown. Support removed guards and new source connections at unchanged sinks.
@@ -130,10 +131,11 @@ the hook and capability admission are not yet implemented. See `implementation-g
- _Verified 2026-09-13:_ independent review APPROVED after operation/context-binding repair; 1,207 tests passed, nine skipped; default real-evidence controls remain diagnostic and synthetic PHP/JavaScript admission is equivalent. See `implementation-group-4.md`.
- _Boundary: Delta and Actionability Policy_
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 5.3, 5.4, 8.1_
- [ ] 4.3 Render compact results and preserve complete artifacts
- [x] 4.3 Render compact results and preserve complete artifacts
- Render up to three actionable summaries plus remaining count; emit at most one no-finding status and one aggregated incomplete notice with a useful next action.
- Persist all defects, candidate dispositions, exact scope, identities, capability/ranker provenance and timings with separate finding/coverage/push status.
- Done when the display cap does not change the push decision, incomplete analysis cannot read as secure, and artifact-write failure produces a bounded notice while retaining the real decision.
- _Verified 2026-09-13:_ manual Kiro review fallback APPROVED after dispatch thread limit; 1,222 tests passed, nine skipped. Packaged comparison/admission/report smoke and CLI boot passed; full records retained with bounded output/write failure handling. See `implementation-group-4.md`.
- _Boundary: Result and Hook Adapter_
- _Requirements: 3.1, 3.2, 3.3, 5.1, 5.2, 5.5, 7.3_
+10 -8
View File
@@ -46,17 +46,19 @@ engine are out of scope.
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–3 are complete (11/27 tasks), including the resolved frontend-retention prerequisite.
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–4 are complete (14/27 tasks), including the resolved frontend-retention prerequisite.
### Next concrete step
Groups 1–3 have passed independent review: runtime, inputs, scoring, snapshots,
shared deadline, exact ranker scope, change-context evidence and vendor-free language
partitions. The approved contributor-scan prerequisite 2.18 (`51fabef`) repairs both
JavaScript test filters; real graphs retain first-party tests and exclude vendor code.
Task 3.4 also repairs unreadable-input handling and preserves named PHP census evidence.
Group 4 is next: novelty comparison, actionability admission and compact advisory output.
See `pre-push-safety-net/implementation-group-3.md` for evidence and limits.
Groups 1–4 are verified: runtime, inputs, scoring, snapshots, shared deadline,
ranker scope, change context, vendor-free partitions, novelty comparison, actionability
admission and compact reporting. Tasks 4.1/4.2 passed independent review; 4.3 used the
Kiro manual review fallback after the agent service reached its thread limit.
Group 5 is next: wire explicit full transaction replay, then run the first bounded
experiment. Real capabilities remain disabled pending independent evaluation.
Automatic dominance-only change-context projection remains an upstream limit to
retain during replay integration. See `pre-push-safety-net/implementation-group-4.md`
for verification and scope limits.
The first complete experiment is 5.2, before persistent caching and hook
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through
+8 -7
View File
@@ -79,10 +79,11 @@ updates as a shortcut to an unmeasured latency promise.
The product direction above is user-requested. The amended requirements, design and initial
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–3 are now verified
complete (11/27 total). The maintainer approved the frontend-retention prerequisite;
contributor-scan 2.18 and pre-push task 3.4 have both passed independent review and
fresh verification. Group 4 is next: novelty comparison, actionability admission and
compact advisory output. Implementation is authorized under the approved plan;
task completion still requires independent review and fresh evidence. Hook performance
and capability admission remain unproven.
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–4 are now verified
complete (14/27 total), including the approved frontend-retention prerequisite.
Group 4 adds bound novelty, operation-specific admission and compact artifacts;
4.1/4.2 passed independent review and 4.3 used the documented Kiro manual fallback
after agent dispatch became unavailable. Group 5 is next for full replay and the
first bounded experiment. Task completion still requires review and fresh evidence.
Real capability eligibility, automatic dominance context projection and hook
performance remain unproven; the default capability registry is empty.