fix(pre-push-safety-net): stop semantic planning at the shared deadline

This commit is contained in:
Marius Ciepluch
2026-09-13 15:11:42 +02:00
parent 31e5ec7430
commit 8fe3a7e51e
8 changed files with 24687 additions and 11 deletions
@@ -64,3 +64,44 @@ RED evidence: `/tmp/ousast-replay-tierzero-red.log` and `/tmp/ousast-replay-snap
- CLAIM: Explicit local replay traverses the production mapper/ranker/arbiter/comparison/report path and retains uncertainty and shared deadlines.
- EVIDENCE: Final packaged controlled security change yields new/source_connection_absent_from_comparable_base. Fixed twin emits no modeled head finding. The packaged deadline control returns incomplete in 30.785 seconds, within 30 + 2 seconds. Exact final package policy/core/facts/query digests match the workspace. Complete commands and captured raw answers live in the measurement artifact.
- GAPS: Aggregate coverage remains incomplete, real capabilities are disabled, and lab timings do not satisfy the latency gate. Source-mounted fixed and earlier packaged deadline provenance is explicitly separated in the artifact. PMPro/Node transfer is task 5.2, not inferred from a small PHP control.
## Predecessor deadline regression discovered by task 5.2
The first frozen trial completed six CLI invocations, retaining all 11 declared cases.
All three PMPro cases timed out during preparation; both snapshots were fully materialized.
Node hit the engine deadline but then took 34.6–37.4 seconds and could not publish its artifact.
Task 5.2 is blocked pending the owning 3.1/5.1 deadline integration repair and a newly frozen run.
The initial trial overlaps regression testing; it is diagnostic evidence, not the final latency sample.
### Debug report
- ROOT_CAUSE: After an engine timeout, the generic scan driver's failure census reloaded semantic facts and taxonomy for every question. The replay runner then launched an unbudgeted engine-availability subprocess before attempting base comparison. Main-thread stack samples at 30–34 seconds prove both paths.
- CATEGORY: LOGIC_ERROR
- FIX_PLAN: Route failure bookkeeping through the existing driver without loading new semantic facts after a failed build; instantiate the budget-aware Joern backend once rather than running legacy availability probes in replay. Preserve every declared question as deferred and keep ordinary backend selection unchanged outside replay.
- VERIFICATION: Two failing regressions (including 400 declared failed questions), real Node deadline replay, full suite and packaged validation before re-freezing the experiment.
- NEXT_ACTION: RETRY_TASK
- CONFIDENCE: HIGH
- NOTES: This repairs the already-approved generic deadline integration in task 3.1 and replay in 5.1. It changes no detector facts, ranking features or abstract capabilities. No workaround is hidden in the evaluation harness; task 5.2 stays blocked until this contract is repaired. Manual Kiro debug/review fallback applies after the recorded agent thread limit. Runtime trace proves a local logic error; no external dependency/version hypothesis is needed.
RED: `/tmp/ousast-node-deadline-flag-red.log`, two failures with flags OFF. Flags were
removed after GREEN; the focused scope/replay suite passed 30 tests.
The first repair exposed the same repeated fact-read path after evidence planning rather than
build failure. The absolute budget now also reaches `_collect` and `_scope_work`; planning
stops loading facts when time expires but retains the entire question census. No ranking weights,
facts or detector semantics changed. The additional regression failed before repair in
`/tmp/ousast-node-planning-red.log` and then passed.
### Predecessor repair review verdict
- VERDICT: APPROVED
- TASK: 3.1/5.1 deadline integration regression
- MECHANICAL_RESULTS: Full current suite 1,242 passed, nine skipped (including three pending experiment-harness checks), exit 0; Ruff/format/mypy/compileall and all regression gates PASS; behavioral RED VERIFIED; no new placeholders/secrets; scope is the existing driver/deadline integration.
- FINDINGS: Both measured post-deadline paths are repaired, including successful-build/evidence-planning cancellation; all declared questions remain recorded. No outstanding repair blocker.
- SUMMARY: Manual review approves the predecessor repair; task5.2 may resume with a new frozen profile.
### Predecessor repair verification
- STATUS: VERIFIED
- CLAIM_TYPE: FIX
- CLAIM: The reproduced Node cancellation path retains diagnostics and returns within 30 + 2 seconds.
- EVIDENCE: Final packaged real Node replay returned in 30.1255 seconds with one complete diagnostic artifact and incomplete coverage. Final policy/core/facts/query hashes match the workspace. See `benchmarks/measurements/2026-09-13-replay-deadline-repair.json` for initial/partial-repair traces and final raw artifact.
- GAPS: This proves the reproduced cancellation path, not representative warm performance or security coverage. Normal capabilities remain disabled.
+2 -2
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-13T12:50:25.249104+00:00",
"updated_at": "2026-09-13T13:11:42.850127+00:00",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -19,7 +19,7 @@
}
},
"ready_for_implementation": true,
"implementation_approval_note": "Group 5 authorized. Task5.1 verified through manual Kiro fallback review; 5.2 next. Real capabilities remain disabled.",
"implementation_approval_note": "Group5 authorized. Task5.1 and its observed deadline regression repaired and verified with manual Kiro fallback; task5.2 resumes under a newly frozen core identity.",
"depends_on": [
"contributor-scan"
],
+1
View File
@@ -149,6 +149,7 @@ Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pendin
- _Depends: 1.3, 3.1, 3.4, 4.3_
- _Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3_
- [ ] 5.2 Measure PMPro and frozen-core Node transfer
- _Prerequisite repaired 2026-09-13:_ 3.1/5.1 deadline regression verified in the packaged Node replay; re-freeze the experiment after this core identity change.
- Replay each prepared security/fixed/benign case, first using the declared hook deadline; record timeouts and stage costs. A separately named extended lab run may obtain witnesses but cannot satisfy the hook latency gate.
- Freeze the core after PMPro, run Node with existing facts, and classify instrument, mapping, fact, scope and domain gaps without tuning on an untouched holdout.
- Done when one joint artifact reports actual evidence and outcomes for both workloads, with unresolved cases retained and an explicit feasibility decision; no normal capability is enabled here.