mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 20:19:29 +00:00
feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Brief: pre-push-safety-net
|
||||
|
||||
Status: requirements, design and tasks approved on 2026-09-12; groups 1 and 2 plus tasks 3.1–3.3 verified complete (10/27 subtasks); task 3.4 blocked on frontend test retention; groups 4–8 pending. Language: English.
|
||||
Status: requirements, design and tasks approved on 2026-09-12; groups 1–3 verified complete (11/27 subtasks), including the approved contributor-scan frontend-retention prerequisite; groups 4–8 pending. Language: English.
|
||||
|
||||
## Problem
|
||||
|
||||
|
||||
@@ -15,10 +15,10 @@ coverage limits.
|
||||
|
||||
## Verification state
|
||||
|
||||
Tasks 3.1–3.3 are independently approved and verified (10/27 tasks overall).
|
||||
Task 3.4 is blocked on a confirmed frontend retention prerequisite; group 3 is
|
||||
not complete. The proposed foundation repair is in
|
||||
`../contributor-scan/frontend-retention-prerequisite.md`.
|
||||
Group 3 is independently approved and VERIFIED (TASK scope): tasks 3.1–3.4 are
|
||||
complete, bringing the pre-push spec to 11/27 tasks. The approved contributor-scan
|
||||
frontend-retention prerequisite 2.18 is also verified. Groups 4–8 remain pending;
|
||||
this is not feature GO, capability admission or a hook latency claim.
|
||||
|
||||
## Baseline
|
||||
|
||||
@@ -130,7 +130,7 @@ line enforced security, that the defect is new, or that a hook meets its latency
|
||||
|
||||
Ten of 27 tasks are now complete; task 3.4 remains for this group.
|
||||
|
||||
## Task 3.4 blocked verification
|
||||
## Historical task 3.4 blocker
|
||||
|
||||
Claim type TASK; status NOT_VERIFIED. The implementer returned BLOCKED, and an
|
||||
independent fresh-context Kiro debug investigation returned STOP_FOR_HUMAN with
|
||||
@@ -169,3 +169,44 @@ uncommitted pending the foundation repair, review and fresh runtime verification
|
||||
The packaged failure preceded a comment-only clarification in `cpg/backend.py`
|
||||
removing the disproven attribution to legacy frontend autodetection. No executable
|
||||
backend behavior changed after that measurement.
|
||||
|
||||
## Task 3.4 final verification
|
||||
|
||||
Status VERIFIED, claim type TASK; independent review APPROVED after two concrete
|
||||
error-path repairs. The final suite passed 1,151 tests with nine skipped in
|
||||
100.55 seconds. Ruff, formatting (230 files), mypy (109 source files), compileall,
|
||||
diff and all three regression gates passed.
|
||||
|
||||
The approved foundation repair is committed as `51fabef`. Physically filtered
|
||||
frontend inputs retain original source paths and bytes; the existing layout facts
|
||||
mark JavaScript/TypeScript test conventions unshipped. One existing ranker decision
|
||||
and region/deadline budget cover supported partitions. Browser-path source remains
|
||||
available, while runtime origin stays unspecified without declarations. Vendor,
|
||||
cross-language, TypeScript-property and C-bounds limits remain explicit.
|
||||
|
||||
Exact source names now travel in taint, dominance and configuration batch censuses.
|
||||
PHP aggregation preserves them only when every shard provides named evidence.
|
||||
Synthetic `<unknown>` nodes cannot mask missing real files. Unreadable unknown
|
||||
extensions produce `source_unreadable` coverage instead of raising PermissionError.
|
||||
Legacy callbacks that omit census entirely retain their existing compatibility
|
||||
behavior; production queries emit the named census, and unsupported/partial
|
||||
sharded evidence cannot establish a complete negative.
|
||||
|
||||
`benchmarks/measurements/2026-09-13-partition-scope-smoke.json` records the final
|
||||
image and ten matching source/query/fact hashes. The real mixed graph retains
|
||||
seven JavaScript files (296 bytes), including tests and browser source, plus two
|
||||
PHP files (97 bytes), with vendor nodes absent. One question is selected and three
|
||||
deferred; unsupported Go is reported. No false empty/unavailable census remains.
|
||||
PHP and JavaScript change-context controls both complete with unchanged-helper
|
||||
witnesses, preserving dirty checkout/index state and cleaning snapshot scratch.
|
||||
|
||||
A real Python graph supplies named census from all three production query kinds.
|
||||
Those unchanged Scala sources were verified before the separate PHP transport and
|
||||
discovery repairs; the artifact records that narrower provenance explicitly. A
|
||||
real filesystem permission probe verifies the repaired error path, using a clearly
|
||||
labeled synthetic graph callback.
|
||||
|
||||
The mixed smoke took 197.76 seconds under a 900-second lab allowance; PHP/JavaScript
|
||||
context checks took 117.3/95.9 seconds. These manually declared controls do not
|
||||
establish automatic scope discovery, production hook latency or alert quality.
|
||||
Group 4 now owns novelty comparison, actionability admission and compact output.
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
|
||||
- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism;
|
||||
third-party vendor code must remain outside targets and graphs.
|
||||
- Requirements, design and tasks are now approved, and groups 1 and 2 plus tasks 3.1–3.3 are verified complete; task 3.4 is blocked on JavaScript frontend test retention; see
|
||||
- Requirements, design and tasks are now approved, and groups 1–3 are verified complete after the approved frontend-retention repair; see
|
||||
`tasks.md` for verified completion. The discovery log below preserves the earlier drafting state
|
||||
and measurements. No hook latency claim is made.
|
||||
|
||||
@@ -167,3 +167,14 @@ Direct frontend invocation reproduced the omission. Two unconditional upstream
|
||||
test filters require an owning foundation repair; see
|
||||
`../contributor-scan/frontend-retention-prerequisite.md`. The runtime failure is
|
||||
not overridden by passing mocks, static checks or a populated server graph.
|
||||
|
||||
### Group 3 completed after frontend and coverage repairs
|
||||
|
||||
The approved contributor-scan 2.18 adaptation preserves test directories and
|
||||
filename suffixes through both pinned JavaScript frontend filters. Task 3.4 then
|
||||
passed independent review with exact graph-path census, including PHP aggregation.
|
||||
Review also repaired unreadable discovery inputs throwing rather than reporting
|
||||
incomplete coverage. A synthetic graph file must never compensate for an omitted
|
||||
real source, and a wrapper must not discard evidence supplied by a frontend.
|
||||
See `implementation-group-3.md` for final evidence. Eleven of 27 tasks are complete;
|
||||
change attribution, actionability admission and compact output are next.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-13T09:17:01.788332+00:00",
|
||||
"updated_at": "2026-09-13T09:50:01.821464+00:00",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -19,7 +19,7 @@
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true,
|
||||
"implementation_approval_note": "Groups 1 and 2 verified complete. Maintainer authorized continuing group 3; remaining approved tasks execute in dependency order with independent review and fresh verification.",
|
||||
"implementation_approval_note": "Groups 1\u20133 verified complete. Contributor-scan frontend-retention prerequisite 2.18 approved and verified. Remaining approved tasks require independent review and fresh verification.",
|
||||
"depends_on": [
|
||||
"contributor-scan"
|
||||
],
|
||||
@@ -77,23 +77,17 @@
|
||||
"2.3",
|
||||
"3.1",
|
||||
"3.2",
|
||||
"3.3"
|
||||
"3.3",
|
||||
"3.4"
|
||||
],
|
||||
"remaining_tasks": 17,
|
||||
"remaining_tasks": 16,
|
||||
"verification_record": "implementation-group-3.md",
|
||||
"verification_records": [
|
||||
"implementation-group-1.md",
|
||||
"implementation-group-2.md",
|
||||
"implementation-group-3.md"
|
||||
],
|
||||
"next_group": 3,
|
||||
"blocked_tasks": [
|
||||
{
|
||||
"task": "3.4",
|
||||
"reason": "Pinned JavaScript frontend silently excludes first-party tests; missing contributor-scan prerequisite.",
|
||||
"record": "../contributor-scan/frontend-retention-prerequisite.md",
|
||||
"next_action": "implement_approved_contributor_scan_2.18"
|
||||
}
|
||||
]
|
||||
"next_group": 4,
|
||||
"blocked_tasks": []
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Implementation Plan: pre-push-safety-net
|
||||
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1 and 2 (tasks 1.1–2.3)
|
||||
are complete after independent review and fresh verification. Group 3 has 3.1–3.3 verified; 3.4 is blocked on frontend test retention. Groups 4–8 remain pending;
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1–3 (tasks 1.1–3.4)
|
||||
are complete after independent review and fresh verification (11/27 subtasks). Groups 4–8 remain pending;
|
||||
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
|
||||
|
||||
## Execution contract
|
||||
@@ -78,7 +78,7 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Requirements: 1.4, 2.1, 2.4, 3.4_
|
||||
- _Verified 2026-09-12:_ independent review APPROVED; fresh final suite 1,069 passed, 9 skipped; focused 148 passed. Global Ruff/format, touched mypy and regression gates passed. PHP/JavaScript fixtures retain removed spans and unchanged-operation anchors across declaration renames; uncertain moves, repeated lines, binary and unavailable inputs remain unresolved. Forty independent real-Git comparisons checked exact spans/anchors. Network-disabled image smoke preserved committed PHP/JavaScript bytes and dirty non-HEAD state, with cleanup verified; see `implementation-group-2.md`. Lexical anchors do not establish semantic impact or novelty.
|
||||
|
||||
- [ ] 3. Integrate bounded execution and ranker scope with the existing engine
|
||||
- [x] 3. Integrate bounded execution and ranker scope with the existing engine
|
||||
- [x] 3.1 (P) Enforce one remaining-time budget in the backend
|
||||
- Thread the absolute deadline through readability probes, frontend builds, overlays, census, query batches, fallbacks and retries; no phase gets a fresh full budget.
|
||||
- Bound process-group termination and lease/scratch cleanup by the cancellation allowance, replacing any longer fixed cleanup waits on the push path.
|
||||
@@ -103,13 +103,14 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Depends: 2.3, 3.2_
|
||||
- _Requirements: 2.1, 2.2, 2.4, 8.1, 8.2_
|
||||
- _Verified 2026-09-13:_ independent review APPROVED after an unknown-call boundary repair; fresh suite 1,128 passed, nine skipped; global Ruff/format/mypy passed. Real PHP/JavaScript Git guard-removal snapshots reached unchanged helper-file witnesses with local dirty state preserved. Production negative regression reproduced then fixed unknown(req.body) being pruned as tier zero: modeled arguments no longer summarize unknown consumers. Existing ranking and detector semantics unchanged; unsupported projections remain explicit. Evidence: `benchmarks/measurements/2026-09-13-change-context-smoke.json`. Lexical attribution is not semantic novelty or alert admission.
|
||||
- [ ] 3.4 Apply vendor exclusion and explicit language partitions
|
||||
- _Blocked:_ Joern 4.0.625/ASTGen 3.50.1 omit first-party JavaScript tests despite readable filtered inputs. Independent debug returned STOP_FOR_HUMAN; review the owning `contributor-scan/frontend-retention-prerequisite.md` before resuming. Implementation remains uncommitted and unapproved; runtime evidence: `benchmarks/measurements/2026-09-13-partition-retention-blocker.json`.
|
||||
- [x] 3.4 Apply vendor exclusion and explicit language partitions
|
||||
- _Prerequisite resolved 2026-09-13:_ contributor-scan 2.18 independently approved and verified at `51fabef`; both JavaScript test filters repaired by explicit versioned input policy. Task 3.4 subsequently passed its own independent review.
|
||||
- Reuse declared layout exclusions before target enumeration and graph build; retain unshipped first-party tests and separate browser/server or language capability coverage.
|
||||
- Build supported frontend partitions explicitly; preserve unknown cross-language and vendor semantics without restoring excluded sources.
|
||||
- Done when PHP and Node graphs/targets contain no declared vendor code, first-party callers remain visible, and unsupported partitions appear as coverage limits rather than clean results.
|
||||
- _Boundary: Ranker Scope Contract, Cache and Engine Boundary integration_
|
||||
- _Requirements: 2.2, 2.3, 7.3, 8.2, 8.4_
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; 1,151 passed, nine skipped; Ruff/format/mypy pass. Real PHP/JavaScript graphs retain tests/browser source, omit vendor nodes and share one ranker limit. Named census survives PHP aggregation; missing source names and unreadable discovery inputs remain explicit. Python exercises all three batch census schemas. Evidence: `benchmarks/measurements/2026-09-13-partition-scope-smoke.json`. No hook capability or latency claim.
|
||||
|
||||
- [ ] 4. Implement change attribution and actionable output
|
||||
- [ ] 4.1 Compare targeted base/head evidence for defect novelty
|
||||
|
||||
@@ -27,7 +27,7 @@ engine are out of scope.
|
||||
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
|
||||
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
|
||||
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
|
||||
- `contributor-scan` has 24/40 checked tasks; historical checkboxes are not rewritten by this review.
|
||||
- `contributor-scan` has 25/41 checked tasks, including the approved and verified frontend-retention prerequisite 2.18.
|
||||
`flow-aware-ranking` is still an unapproved brief despite implementation history.
|
||||
|
||||
## Existing Spec Updates
|
||||
@@ -46,15 +46,16 @@ engine are out of scope.
|
||||
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
|
||||
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
|
||||
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1 and 2 and tasks 3.1–3.3 are complete (10/27 tasks); 3.4 is blocked.
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–3 are complete (11/27 tasks), including the resolved frontend-retention prerequisite.
|
||||
|
||||
### Next concrete step
|
||||
|
||||
Groups 1 and 2 and tasks 3.1–3.3 have passed independent review: runtime, inputs,
|
||||
scoring, snapshots, shared deadline, exact ranker scope and change-context evidence.
|
||||
Task 3.4 is blocked: real Joern graphs omit first-party JavaScript tests even though
|
||||
filtered inputs contain them. Review `contributor-scan/frontend-retention-prerequisite.md`,
|
||||
repair that foundation contract, then rerun the partition census and independent review.
|
||||
Groups 1–3 have passed independent review: runtime, inputs, scoring, snapshots,
|
||||
shared deadline, exact ranker scope, change-context evidence and vendor-free language
|
||||
partitions. The approved contributor-scan prerequisite 2.18 (`51fabef`) repairs both
|
||||
JavaScript test filters; real graphs retain first-party tests and exclude vendor code.
|
||||
Task 3.4 also repairs unreadable-input handling and preserves named PHP census evidence.
|
||||
Group 4 is next: novelty comparison, actionability admission and compact advisory output.
|
||||
See `pre-push-safety-net/implementation-group-3.md` for evidence and limits.
|
||||
The first complete experiment is 5.2, before persistent caching and hook
|
||||
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
|
||||
|
||||
@@ -79,8 +79,10 @@ updates as a shortcut to an unmeasured latency promise.
|
||||
The product direction above is user-requested. The amended requirements, design and initial
|
||||
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
|
||||
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
|
||||
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1 and 2 are now verified
|
||||
complete, as are group 3 tasks 3.1–3.3 (10/27 total). Task 3.4 is blocked on a
|
||||
confirmed frontend test-retention gap; review the proposed contributor-scan
|
||||
frontend-retention prerequisite before resuming it. Implementation is authorized under that plan; task completion still requires
|
||||
independent review and fresh evidence. Hook performance and capability admission remain unproven.
|
||||
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–3 are now verified
|
||||
complete (11/27 total). The maintainer approved the frontend-retention prerequisite;
|
||||
contributor-scan 2.18 and pre-push task 3.4 have both passed independent review and
|
||||
fresh verification. Group 4 is next: novelty comparison, actionability admission and
|
||||
compact advisory output. Implementation is authorized under the approved plan;
|
||||
task completion still requires independent review and fresh evidence. Hook performance
|
||||
and capability admission remain unproven.
|
||||
|
||||
Reference in New Issue
Block a user