mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 03:59:28 +00:00
feat(pre-push-safety-net): retain bounded change and rename context
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Brief: pre-push-safety-net
|
||||
|
||||
Status: requirements, design and tasks approved on 2026-09-12; group 1 and task 2.1 verified complete; group 2 implementation underway. Language: English.
|
||||
Status: requirements, design and tasks approved on 2026-09-12; groups 1 and 2 verified complete (7/27 subtasks); groups 3–8 pending. Language: English.
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -11,8 +11,8 @@ maintainer explicitly rejects noisy advisory output as well as noisy blocking ou
|
||||
## Current State
|
||||
|
||||
The scan driver and Joern arbiters exist, with demonstrated PHP/Python cases. Immutable push-tip
|
||||
resolution is implemented and verified. Isolated source materialization and change context are
|
||||
under implementation; persistent graph caching and an end-to-end deadline remain pending.
|
||||
resolution, isolated source materialization and lexical change context are implemented and verified.
|
||||
Ranker integration, persistent graph caching and an end-to-end deadline remain pending.
|
||||
The driver builds a whole-root graph before cutting the region budget. The latest PMPro ranking
|
||||
experiment alone spends about 130 seconds building and 296 seconds extracting evidence. A manually
|
||||
selected two-file slice finding a CVE does not demonstrate automatic incremental analysis.
|
||||
|
||||
@@ -192,6 +192,12 @@ automatic fetch inside the hook. Peel annotated tags to commits; other objects a
|
||||
Deletion updates require no target scan. Deduplicate identical target graphs, but retain separate
|
||||
comparisons when the same target has different bases. Multiple refs share one deadline.
|
||||
|
||||
The snapshot implementation uses `SnapshotAdapter.resolve_updates`, the owned `materialize`
|
||||
context manager, and `compare`. Preparation has explicit file, source-byte, tree/diff-output and
|
||||
line-mapping limits. Materialization verifies blob identity and records omitted symlinks, gitlinks
|
||||
and LFS pointers. Its completeness flag describes source preparation, not security coverage.
|
||||
The shipped implementation uses Linux directory-relative cleanup that does not follow links.
|
||||
|
||||
Materialize tracked blobs directly into isolated scratch storage; do not checkout over the user's
|
||||
tree or run filters/build scripts. Use NUL-delimited tree/diff records and argument arrays. Include
|
||||
deletions as impact seeds and retain rename attribution. Symlinks are not followed outside the
|
||||
@@ -206,6 +212,17 @@ an alternative inclusion policy. The existing ranker produces `ScopeDecision`: s
|
||||
IDs, their priority/evidence, deferred question IDs with reasons, and unresolved context boundaries.
|
||||
Identifiers include unit/language/path/function/family to avoid collisions across partitions.
|
||||
|
||||
Adapter-produced contexts declare `path_encoding="filesystem-bytes-hex"` for structural paths
|
||||
(changed/deleted/declaration paths, spans, renames and line correspondences). Use
|
||||
`ChangeContext.decode_path` when matching canonical question paths; semantic relationship
|
||||
`QuestionIdentity` paths retain their normal encoding. Existing text-context construction remains
|
||||
compatible. This preserves unusual and non-UTF-8 filesystem names without ambiguous display escaping.
|
||||
The caller supplies known declaration/configuration paths as raw bytes; an unavailable declaration
|
||||
inventory is explicitly unresolved. Unique identical lines outside edit hunks provide lexical
|
||||
correspondence even after a declaration rename or guard removal. Repeated lines, unmatched moves
|
||||
and ambiguous rename attribution remain unresolved; these anchors do not prove semantic function
|
||||
identity, guard effects or defect novelty. Those decisions remain with tasks 3.3 and 4.1.
|
||||
|
||||
Third-party vendor exclusions apply before both preprocessing targets and graph construction.
|
||||
Tests remain available but unshipped, consistent with the approved layout decision. The ranker
|
||||
cannot reintroduce vendor sources. Known external summaries are versioned facts; without a relevant
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# Group 2 implementation verification
|
||||
|
||||
Date: 2026-09-12. Scope: tasks 2.1–2.3. All three tasks received independent
|
||||
APPROVED reviews after task-local verification. The specification remains in
|
||||
implementation: 7/27 executable subtasks are complete, with groups 3–8 pending.
|
||||
|
||||
## Verification result
|
||||
|
||||
- STATUS: VERIFIED
|
||||
- CLAIM_TYPE: TASK
|
||||
- CLAIM: Group 2 resolves immutable pushed revisions, materializes isolated tracked
|
||||
source, and supplies bounded lexical change evidence to the shared contract.
|
||||
- EVIDENCE: Fresh final `.venv/bin/pytest -q`: 1,069 passed, 9 skipped (exit 0).
|
||||
Focused snapshot/contracts/audit suite: 148 passed. Global Ruff check and format
|
||||
check passed (218 files formatted); touched-module mypy and compileall passed.
|
||||
Detection, split-sink map and local pair regression gates each passed.
|
||||
- GAPS: This is not feature-level GO. Ranker integration, graph/vendor partitions,
|
||||
semantic change attribution, persistent caching and the operational hook remain
|
||||
scheduled work. No hook detection, precision or latency claim follows.
|
||||
|
||||
## Task evidence
|
||||
|
||||
| Task | Verified result |
|
||||
| --- | --- |
|
||||
| 2.1 | Exact supplied base/head comparisons, configured unique merge bases, tag peeling, special-update dispositions and all ref associations. SHA-1/SHA-256 tests include replacement/graft suppression and missing promisor objects without fetching. Review caught Unicode-aware protocol splitting; six regression cases now retain Git-valid Unicode refs. Full suite at completion: 997 passed, 9 skipped. Commit `05da202`. |
|
||||
| 2.2 | Direct bounded blob reads with object hashes, reversible binary paths, explicit symlink/gitlink/LFS limits, and owned scratch cleanup. Dirty/non-HEAD success and failure paths preserve live files, index, refs and hooks. Independent hanging-child cancellation removed scratch in 0.202 s. Full suite: 1,037 passed, 9 skipped. Commit `97fd633`. |
|
||||
| 2.3 | NUL-safe comparison metadata, both-side changed spans, declared configuration paths, renamed-path attribution and unique unchanged-line anchors. PHP/JavaScript declaration-renaming and removed-guard fixtures preserve the unchanged operation's lexical anchor. Repeated lines, unmatched/ambiguous moves, binary and unavailable inputs retain explicit uncertainty. Independent review additionally checked 40 real-Git edits with 4,880 base bytes read. |
|
||||
|
||||
The initial disabled implementations failed the relevant tests before implementation;
|
||||
expanded tests passed after flag removal. Final normalization also fixed formatting
|
||||
in the earlier push-foundation files without changing their behavior.
|
||||
|
||||
## Real repository and packaged checks
|
||||
|
||||
`benchmarks/measurements/2026-09-12-snapshot-materialization.json` records a lab
|
||||
materialization of commit `05da202`: 1,157 tracked files and 8,737,198 source bytes.
|
||||
Every materialized file was reopened and checked for length and SHA256; the pinned
|
||||
2,547-byte `pyproject.toml` also matched its Git blob directly. Scratch removal and
|
||||
unchanged live status/index were verified. Preparation took 5.783 seconds under an
|
||||
explicit 120-second lab budget. This is preparation cost, not complete hook latency.
|
||||
The artifact's source hashes match committed task 2.2 implementation `97fd633`.
|
||||
|
||||
After final source formatting, `docker compose build ousast` exited 0. Image:
|
||||
`sha256:2a0dd7943ca9b9c84adc821990d38bff13a7824c7d5a9f94d07c38cb04ce333f`.
|
||||
Seven packaged source-file hashes matched the workspace. The non-root,
|
||||
network-disabled `ops/smoke_snapshot.py` run verified 103 JavaScript bytes and 46
|
||||
PHP bytes, exact pushed comparisons, renamed/removed spans and declared configuration
|
||||
context in a dirty non-HEAD checkout. Scratch was removed and live state preserved.
|
||||
The packaged CLI help also exited 0. Retained evidence:
|
||||
`benchmarks/measurements/2026-09-12-packaged-snapshot-smoke.json`.
|
||||
Reproduction instructions are in `ops/README.md`.
|
||||
|
||||
## Remaining limits and next group
|
||||
|
||||
Full-tree `.venv/bin/mypy src/openultrasast` still exits 1 with the same three
|
||||
pre-existing errors in `model/scan.py` at 299, 302 and 310, checking 108 source files.
|
||||
No additional type errors were reported. The nine skipped tests remain outside
|
||||
demonstrated coverage.
|
||||
|
||||
Snapshot completeness describes materialized source only. The snapshot contains the
|
||||
tracked population; task 3.4 applies vendor exclusion and supported language partitions
|
||||
before graph construction and target selection. Cleanup currently targets the shipped
|
||||
Linux environment. The resolver's transaction-wide deadline integration remains pending.
|
||||
|
||||
Change-context structural paths use `filesystem-bytes-hex`; consumers must decode them
|
||||
with `ChangeContext.decode_path` before matching canonical question paths. Declaration
|
||||
paths come from the caller's project discovery; an unavailable inventory is recorded.
|
||||
Lexical correspondence does not prove semantic function identity, a guard's effect,
|
||||
or defect novelty. Tasks 3.3 and 4.1 own those decisions.
|
||||
|
||||
Next: group 3 integrates the shared execution budget, exact existing-ranker scope,
|
||||
affected first-party relationships and vendor/language boundaries. The first complete
|
||||
PMPro/Node experiment remains task 5.2, before persistent caching in group 6.
|
||||
@@ -6,7 +6,7 @@
|
||||
- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
|
||||
- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism;
|
||||
third-party vendor code must remain outside targets and graphs.
|
||||
- Requirements, design and tasks are now approved, and group 1 implementation is verified complete; see
|
||||
- Requirements, design and tasks are now approved, and groups 1 and 2 are verified complete; see
|
||||
`tasks.md` for verified completion. The discovery log below preserves the earlier drafting state
|
||||
and measurements. No hook latency claim is made.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-12T18:32:19.033933Z",
|
||||
"updated_at": "2026-09-12T19:09:25.547910Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -19,7 +19,7 @@
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true,
|
||||
"implementation_approval_note": "Groups 1 and 2 explicitly authorized by the maintainer. Group 1 verified complete; this run implements group 2 (2.1\u20132.3). Remaining task plan retains approval, with execution sequenced by the spec.",
|
||||
"implementation_approval_note": "Groups 1 and 2 explicitly authorized by the maintainer and verified complete after independent review. Remaining task plan retains approval, with execution sequenced by the spec. Implementation of the complete hook is still pending.",
|
||||
"depends_on": [
|
||||
"contributor-scan"
|
||||
],
|
||||
@@ -61,11 +61,17 @@
|
||||
"1.1",
|
||||
"1.2",
|
||||
"1.3",
|
||||
"1.4"
|
||||
"1.4",
|
||||
"2.1",
|
||||
"2.2",
|
||||
"2.3"
|
||||
],
|
||||
"remaining_tasks": 23,
|
||||
"verification_record": "implementation-group-1.md",
|
||||
"next_group": 2,
|
||||
"active_group": 2
|
||||
"remaining_tasks": 20,
|
||||
"verification_record": "implementation-group-2.md",
|
||||
"verification_records": [
|
||||
"implementation-group-1.md",
|
||||
"implementation-group-2.md"
|
||||
],
|
||||
"next_group": 3
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Implementation Plan: pre-push-safety-net
|
||||
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Group 1 (tasks 1.1–1.4)
|
||||
is complete after independent review and fresh verification. Remaining groups are pending;
|
||||
the hook and capability admission are not yet implemented. See `implementation-group-1.md`.
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1 and 2 (tasks 1.1–2.3)
|
||||
are complete after independent review and fresh verification. Groups 3–8 remain pending;
|
||||
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
|
||||
|
||||
## Execution contract
|
||||
|
||||
@@ -55,7 +55,7 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Requirements: 4.1, 4.3, 5.3, 5.4, 5.5, 7.1, 7.3_
|
||||
- _Verified 2026-09-12:_ independent review APPROVED; 965 tests passed, 9 skipped; touched-source Ruff/mypy passed. Parent rebuilt the local image and verified non-root, network-disabled CLI startup, all contract imports, an 83-byte configuration round trip and invalid-budget rejection. Comparison-owned completion preserves different bases for a shared head. Contracts are structural; runtime snapshot/cache/runner/admission implementations remain in later tasks.
|
||||
|
||||
- [ ] 2. Resolve immutable pushed content
|
||||
- [x] 2. Resolve immutable pushed content
|
||||
- [x] 2.1 Resolve every supplied ref update and comparison
|
||||
- Parse Git's complete update input using object-format-aware validation; resolve existing tips, new-branch configured bases, force-pushes, merges, peeled tags, deleted refs and unsupported objects.
|
||||
- Deduplicate equal targets without merging different base comparisons; retain every ref association. Do not silently substitute HEAD or fetch missing objects.
|
||||
@@ -70,12 +70,13 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Boundary: Snapshot Adapter_
|
||||
- _Requirements: 1.1, 1.2, 1.3, 2.3, 7.1, 7.3_
|
||||
- _Verified 2026-09-12:_ independent review APPROVED; 1,037 tests passed, 9 skipped; focused 116 passed; touched Ruff/mypy passed. Real SHA-1/SHA-256 fixtures verify binary-safe paths, exact blobs, no filters, explicit symlink/gitlink/LFS limits, cancellation and unchanged live state. Independent hanging-child cleanup took 0.202 s. Parent reopened and checked all 1,157 materialized repository files (8,737,198 bytes), with scratch removed and live state preserved; evidence `benchmarks/measurements/2026-09-12-snapshot-materialization.json` records a 5.783 s preparation-only lab run, not end-to-end hook latency.
|
||||
- [ ] 2.3 Attach changed spans and deletion/rename context
|
||||
- [x] 2.3 Attach changed spans and deletion/rename context
|
||||
- Compare recorded base/head objects using NUL-safe path handling; retain removed spans, renamed identities and declaration/config changes.
|
||||
- Supply change context to the shared ranker contract rather than selecting an independent changed-files-only scope.
|
||||
- Done when renamed functions retain correspondence and a removed guard remains an impact seed for an unchanged operation, with missing correspondence explicitly unresolved.
|
||||
- _Boundary: Snapshot Adapter_
|
||||
- _Requirements: 1.4, 2.1, 2.4, 3.4_
|
||||
- _Verified 2026-09-12:_ independent review APPROVED; fresh final suite 1,069 passed, 9 skipped; focused 148 passed. Global Ruff/format, touched mypy and regression gates passed. PHP/JavaScript fixtures retain removed spans and unchanged-operation anchors across declaration renames; uncertain moves, repeated lines, binary and unavailable inputs remain unresolved. Forty independent real-Git comparisons checked exact spans/anchors. Network-disabled image smoke preserved committed PHP/JavaScript bytes and dirty non-HEAD state, with cleanup verified; see `implementation-group-2.md`. Lexical anchors do not establish semantic impact or novelty.
|
||||
|
||||
- [ ] 3. Integrate bounded execution and ranker scope with the existing engine
|
||||
- [ ] 3.1 (P) Enforce one remaining-time budget in the backend
|
||||
|
||||
@@ -46,13 +46,14 @@ engine are out of scope.
|
||||
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
|
||||
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
|
||||
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; group 1 is complete (4/27 tasks).
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1 and 2 are complete (7/27 tasks).
|
||||
|
||||
### Next concrete step
|
||||
|
||||
Group 1 has passed independent review: reproducible PHP/JavaScript runtime, pinned PMPro/Node inputs,
|
||||
honest scoring and shared contracts. Continue with group 2's immutable push snapshots under the
|
||||
approved task plan; see `pre-push-safety-net/implementation-group-1.md` for evidence and limits.
|
||||
Groups 1 and 2 have passed independent review: reproducible runtime, pinned inputs, honest scoring,
|
||||
shared contracts, immutable push snapshots and lexical change context. Continue with group 3: one
|
||||
execution budget, exact ranker scope, affected relationships and vendor/language partitions. See
|
||||
`pre-push-safety-net/implementation-group-2.md` for evidence and limits.
|
||||
The first complete experiment is 5.2, before persistent caching and hook
|
||||
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
|
||||
prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through
|
||||
|
||||
@@ -79,5 +79,6 @@ updates as a shortcut to an unmeasured latency promise.
|
||||
The product direction above is user-requested. The amended requirements, design and initial
|
||||
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
|
||||
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
|
||||
("approve."). Implementation is authorized under that plan; task completion still requires
|
||||
("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1 and 2 are now verified
|
||||
complete. Implementation is authorized under that plan; task completion still requires
|
||||
independent review and fresh evidence. Hook performance and capability admission remain unproven.
|
||||
|
||||
Reference in New Issue
Block a user