feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage

This commit is contained in:
Marius Ciepluch
2026-09-13 11:53:29 +02:00
parent 51fabef9ea
commit 6ba2b06353
29 changed files with 1996 additions and 72 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Brief: pre-push-safety-net
Status: requirements, design and tasks approved on 2026-09-12; groups 1 and 2 plus tasks 3.1–3.3 verified complete (10/27 subtasks); task 3.4 blocked on frontend test retention; groups 4–8 pending. Language: English.
Status: requirements, design and tasks approved on 2026-09-12; groups 1–3 verified complete (11/27 subtasks), including the approved contributor-scan frontend-retention prerequisite; groups 4–8 pending. Language: English.
## Problem
@@ -15,10 +15,10 @@ coverage limits.
## Verification state
Tasks 3.1–3.3 are independently approved and verified (10/27 tasks overall).
Task 3.4 is blocked on a confirmed frontend retention prerequisite; group 3 is
not complete. The proposed foundation repair is in
`../contributor-scan/frontend-retention-prerequisite.md`.
Group 3 is independently approved and VERIFIED (TASK scope): tasks 3.1–3.4 are
complete, bringing the pre-push spec to 11/27 tasks. The approved contributor-scan
frontend-retention prerequisite 2.18 is also verified. Groups 4–8 remain pending;
this is not feature GO, capability admission or a hook latency claim.
## Baseline
@@ -130,7 +130,7 @@ line enforced security, that the defect is new, or that a hook meets its latency
Ten of 27 tasks are now complete; task 3.4 remains for this group.
## Task 3.4 blocked verification
## Historical task 3.4 blocker
Claim type TASK; status NOT_VERIFIED. The implementer returned BLOCKED, and an
independent fresh-context Kiro debug investigation returned STOP_FOR_HUMAN with
@@ -169,3 +169,44 @@ uncommitted pending the foundation repair, review and fresh runtime verification
The packaged failure preceded a comment-only clarification in `cpg/backend.py`
removing the disproven attribution to legacy frontend autodetection. No executable
backend behavior changed after that measurement.
## Task 3.4 final verification
Status VERIFIED, claim type TASK; independent review APPROVED after two concrete
error-path repairs. The final suite passed 1,151 tests with nine skipped in
100.55 seconds. Ruff, formatting (230 files), mypy (109 source files), compileall,
diff and all three regression gates passed.
The approved foundation repair is committed as `51fabef`. Physically filtered
frontend inputs retain original source paths and bytes; the existing layout facts
mark JavaScript/TypeScript test conventions unshipped. One existing ranker decision
and region/deadline budget cover supported partitions. Browser-path source remains
available, while runtime origin stays unspecified without declarations. Vendor,
cross-language, TypeScript-property and C-bounds limits remain explicit.
Exact source names now travel in taint, dominance and configuration batch censuses.
PHP aggregation preserves them only when every shard provides named evidence.
Synthetic `<unknown>` nodes cannot mask missing real files. Unreadable unknown
extensions produce `source_unreadable` coverage instead of raising PermissionError.
Legacy callbacks that omit census entirely retain their existing compatibility
behavior; production queries emit the named census, and unsupported/partial
sharded evidence cannot establish a complete negative.
`benchmarks/measurements/2026-09-13-partition-scope-smoke.json` records the final
image and ten matching source/query/fact hashes. The real mixed graph retains
seven JavaScript files (296 bytes), including tests and browser source, plus two
PHP files (97 bytes), with vendor nodes absent. One question is selected and three
deferred; unsupported Go is reported. No false empty/unavailable census remains.
PHP and JavaScript change-context controls both complete with unchanged-helper
witnesses, preserving dirty checkout/index state and cleaning snapshot scratch.
A real Python graph supplies named census from all three production query kinds.
Those unchanged Scala sources were verified before the separate PHP transport and
discovery repairs; the artifact records that narrower provenance explicitly. A
real filesystem permission probe verifies the repaired error path, using a clearly
labeled synthetic graph callback.
The mixed smoke took 197.76 seconds under a 900-second lab allowance; PHP/JavaScript
context checks took 117.3/95.9 seconds. These manually declared controls do not
establish automatic scope discovery, production hook latency or alert quality.
Group 4 now owns novelty comparison, actionability admission and compact output.
+12 -1
View File
@@ -6,7 +6,7 @@
- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism;
third-party vendor code must remain outside targets and graphs.
- Requirements, design and tasks are now approved, and groups 1 and 2 plus tasks 3.1–3.3 are verified complete; task 3.4 is blocked on JavaScript frontend test retention; see
- Requirements, design and tasks are now approved, and groups 1–3 are verified complete after the approved frontend-retention repair; see
`tasks.md` for verified completion. The discovery log below preserves the earlier drafting state
and measurements. No hook latency claim is made.
@@ -167,3 +167,14 @@ Direct frontend invocation reproduced the omission. Two unconditional upstream
test filters require an owning foundation repair; see
`../contributor-scan/frontend-retention-prerequisite.md`. The runtime failure is
not overridden by passing mocks, static checks or a populated server graph.
### Group 3 completed after frontend and coverage repairs
The approved contributor-scan 2.18 adaptation preserves test directories and
filename suffixes through both pinned JavaScript frontend filters. Task 3.4 then
passed independent review with exact graph-path census, including PHP aggregation.
Review also repaired unreadable discovery inputs throwing rather than reporting
incomplete coverage. A synthetic graph file must never compensate for an omitted
real source, and a wrapper must not discard evidence supplied by a frontend.
See `implementation-group-3.md` for final evidence. Eleven of 27 tasks are complete;
change attribution, actionability admission and compact output are next.
+7 -13
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-13T09:17:01.788332+00:00",
"updated_at": "2026-09-13T09:50:01.821464+00:00",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -19,7 +19,7 @@
}
},
"ready_for_implementation": true,
"implementation_approval_note": "Groups 1 and 2 verified complete. Maintainer authorized continuing group 3; remaining approved tasks execute in dependency order with independent review and fresh verification.",
"implementation_approval_note": "Groups 1\u20133 verified complete. Contributor-scan frontend-retention prerequisite 2.18 approved and verified. Remaining approved tasks require independent review and fresh verification.",
"depends_on": [
"contributor-scan"
],
@@ -77,23 +77,17 @@
"2.3",
"3.1",
"3.2",
"3.3"
"3.3",
"3.4"
],
"remaining_tasks": 17,
"remaining_tasks": 16,
"verification_record": "implementation-group-3.md",
"verification_records": [
"implementation-group-1.md",
"implementation-group-2.md",
"implementation-group-3.md"
],
"next_group": 3,
"blocked_tasks": [
{
"task": "3.4",
"reason": "Pinned JavaScript frontend silently excludes first-party tests; missing contributor-scan prerequisite.",
"record": "../contributor-scan/frontend-retention-prerequisite.md",
"next_action": "implement_approved_contributor_scan_2.18"
}
]
"next_group": 4,
"blocked_tasks": []
}
}
+6 -5
View File
@@ -1,7 +1,7 @@
# Implementation Plan: pre-push-safety-net
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1 and 2 (tasks 1.1–2.3)
are complete after independent review and fresh verification. Group 3 has 3.1–3.3 verified; 3.4 is blocked on frontend test retention. Groups 4–8 remain pending;
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1–3 (tasks 1.1–3.4)
are complete after independent review and fresh verification (11/27 subtasks). Groups 4–8 remain pending;
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
## Execution contract
@@ -78,7 +78,7 @@ the hook and capability admission are not yet implemented. See `implementation-g
- _Requirements: 1.4, 2.1, 2.4, 3.4_
- _Verified 2026-09-12:_ independent review APPROVED; fresh final suite 1,069 passed, 9 skipped; focused 148 passed. Global Ruff/format, touched mypy and regression gates passed. PHP/JavaScript fixtures retain removed spans and unchanged-operation anchors across declaration renames; uncertain moves, repeated lines, binary and unavailable inputs remain unresolved. Forty independent real-Git comparisons checked exact spans/anchors. Network-disabled image smoke preserved committed PHP/JavaScript bytes and dirty non-HEAD state, with cleanup verified; see `implementation-group-2.md`. Lexical anchors do not establish semantic impact or novelty.
- [ ] 3. Integrate bounded execution and ranker scope with the existing engine
- [x] 3. Integrate bounded execution and ranker scope with the existing engine
- [x] 3.1 (P) Enforce one remaining-time budget in the backend
- Thread the absolute deadline through readability probes, frontend builds, overlays, census, query batches, fallbacks and retries; no phase gets a fresh full budget.
- Bound process-group termination and lease/scratch cleanup by the cancellation allowance, replacing any longer fixed cleanup waits on the push path.
@@ -103,13 +103,14 @@ the hook and capability admission are not yet implemented. See `implementation-g
- _Depends: 2.3, 3.2_
- _Requirements: 2.1, 2.2, 2.4, 8.1, 8.2_
- _Verified 2026-09-13:_ independent review APPROVED after an unknown-call boundary repair; fresh suite 1,128 passed, nine skipped; global Ruff/format/mypy passed. Real PHP/JavaScript Git guard-removal snapshots reached unchanged helper-file witnesses with local dirty state preserved. Production negative regression reproduced then fixed unknown(req.body) being pruned as tier zero: modeled arguments no longer summarize unknown consumers. Existing ranking and detector semantics unchanged; unsupported projections remain explicit. Evidence: `benchmarks/measurements/2026-09-13-change-context-smoke.json`. Lexical attribution is not semantic novelty or alert admission.
- [ ] 3.4 Apply vendor exclusion and explicit language partitions
- _Blocked:_ Joern 4.0.625/ASTGen 3.50.1 omit first-party JavaScript tests despite readable filtered inputs. Independent debug returned STOP_FOR_HUMAN; review the owning `contributor-scan/frontend-retention-prerequisite.md` before resuming. Implementation remains uncommitted and unapproved; runtime evidence: `benchmarks/measurements/2026-09-13-partition-retention-blocker.json`.
- [x] 3.4 Apply vendor exclusion and explicit language partitions
- _Prerequisite resolved 2026-09-13:_ contributor-scan 2.18 independently approved and verified at `51fabef`; both JavaScript test filters repaired by explicit versioned input policy. Task 3.4 subsequently passed its own independent review.
- Reuse declared layout exclusions before target enumeration and graph build; retain unshipped first-party tests and separate browser/server or language capability coverage.
- Build supported frontend partitions explicitly; preserve unknown cross-language and vendor semantics without restoring excluded sources.
- Done when PHP and Node graphs/targets contain no declared vendor code, first-party callers remain visible, and unsupported partitions appear as coverage limits rather than clean results.
- _Boundary: Ranker Scope Contract, Cache and Engine Boundary integration_
- _Requirements: 2.2, 2.3, 7.3, 8.2, 8.4_
- _Verified 2026-09-13:_ independent review APPROVED; 1,151 passed, nine skipped; Ruff/format/mypy pass. Real PHP/JavaScript graphs retain tests/browser source, omit vendor nodes and share one ranker limit. Named census survives PHP aggregation; missing source names and unreadable discovery inputs remain explicit. Python exercises all three batch census schemas. Evidence: `benchmarks/measurements/2026-09-13-partition-scope-smoke.json`. No hook capability or latency claim.
- [ ] 4. Implement change attribution and actionable output
- [ ] 4.1 Compare targeted base/head evidence for defect novelty
+8 -7
View File
@@ -27,7 +27,7 @@ engine are out of scope.
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
- `contributor-scan` has 24/40 checked tasks; historical checkboxes are not rewritten by this review.
- `contributor-scan` has 25/41 checked tasks, including the approved and verified frontend-retention prerequisite 2.18.
`flow-aware-ranking` is still an unapproved brief despite implementation history.
## Existing Spec Updates
@@ -46,15 +46,16 @@ engine are out of scope.
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1 and 2 and tasks 3.1–3.3 are complete (10/27 tasks); 3.4 is blocked.
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–3 are complete (11/27 tasks), including the resolved frontend-retention prerequisite.
### Next concrete step
Groups 1 and 2 and tasks 3.1–3.3 have passed independent review: runtime, inputs,
scoring, snapshots, shared deadline, exact ranker scope and change-context evidence.
Task 3.4 is blocked: real Joern graphs omit first-party JavaScript tests even though
filtered inputs contain them. Review `contributor-scan/frontend-retention-prerequisite.md`,
repair that foundation contract, then rerun the partition census and independent review.
Groups 1–3 have passed independent review: runtime, inputs, scoring, snapshots,
shared deadline, exact ranker scope, change-context evidence and vendor-free language
partitions. The approved contributor-scan prerequisite 2.18 (`51fabef`) repairs both
JavaScript test filters; real graphs retain first-party tests and exclude vendor code.
Task 3.4 also repairs unreadable-input handling and preserves named PHP census evidence.
Group 4 is next: novelty comparison, actionability admission and compact advisory output.
See `pre-push-safety-net/implementation-group-3.md` for evidence and limits.
The first complete experiment is 5.2, before persistent caching and hook
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
+7 -5
View File
@@ -79,8 +79,10 @@ updates as a shortcut to an unmeasured latency promise.
The product direction above is user-requested. The amended requirements, design and initial
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1 and 2 are now verified
complete, as are group 3 tasks 3.1–3.3 (10/27 total). Task 3.4 is blocked on a
confirmed frontend test-retention gap; review the proposed contributor-scan
frontend-retention prerequisite before resuming it. Implementation is authorized under that plan; task completion still requires
independent review and fresh evidence. Hook performance and capability admission remain unproven.
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–3 are now verified
complete (11/27 total). The maintainer approved the frontend-retention prerequisite;
contributor-scan 2.18 and pre-push task 3.4 have both passed independent review and
fresh verification. Group 4 is next: novelty comparison, actionability admission and
compact advisory output. Implementation is authorized under the approved plan;
task completion still requires independent review and fresh evidence. Hook performance
and capability admission remain unproven.