feat(pre-push-safety-net): establish shared push and engine contracts

This commit is contained in:
Marius Ciepluch
2026-09-12 19:21:31 +02:00
parent 051f3b0058
commit 14a2a60e52
18 changed files with 810 additions and 25 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Brief: pre-push-safety-net
Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation started. Language: English.
Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation verified complete; groups 2–8 pending. Language: English.
## Problem
+16 -2
View File
@@ -132,8 +132,8 @@ Concrete workload roles and required paired changes are in `evaluation.md`.
| Storage | Local content-addressed directories and JSON manifests | Atomic reusable artifacts, no database |
| Presentation | Existing finding records plus compact hook renderer | Separate findings and coverage |
Align Docker, Compose and provisioning with the evaluated Joern version before any deployment
claim. The old defaults are 4.0.623. Pin alignment is predecessor packaging work, not completed here.
Docker, Compose and provisioning defaults were aligned to 4.0.625 in task 1.1, with a verified
local PHP/JavaScript image smoke. Host deployment and hook performance remain separate claims.
## File Structure Plan
@@ -141,6 +141,10 @@ New files (proposed, not created by this documentation change):
| Path | Responsibility |
|---|---|
| `src/openultrasast/contracts.py` | Strict immutable manifest serialization shared by engine and adapters |
| `src/openultrasast/model/contracts.py` | Generic deadline, change context and exact selected/deferred scope contracts |
| `src/openultrasast/cpg/artifact.py` | Backend-owned graph identity, census and live lease protocol |
| `src/openultrasast/push/contracts.py` | Immutable ref/comparison identities and comparison-owned completion/status records |
| `src/openultrasast/push/snapshot.py` | Parse ref updates, resolve bases, materialize blobs, produce change context |
| `src/openultrasast/push/cache.py` | Artifact keys, validity, atomic publication and eviction |
| `src/openultrasast/push/policy.py` | Delta identity, actionability admission and push disposition |
@@ -149,6 +153,16 @@ New files (proposed, not created by this documentation change):
| `benchmarks/push/README.md` and `benchmarks/push/measure.py` | Labeled push recipes and joint quality/latency measurement |
| `tests/test_push_snapshot.py`, `tests/test_push_cache.py`, `tests/test_push_policy.py`, `tests/test_push_runner.py` | Contract and integration checks below |
Contract configuration (task 1.4) uses `[push]`: optional `comparison_base`, positive
`deadline_seconds` (30), positive `cancellation_allowance_seconds` (2), positive integer
`cache_max_bytes` (2 GiB), `mode` (`advisory` by default, or `blocking`) and independent
`incomplete_coverage_policy` (`allow` by default, or `block`). Unknown keys, malformed
sections and coerced/nonfinite numeric values are rejected. These settings are inert until
runner integration; they do not enable a hook or admit experimental capabilities. The cache
limit is a local storage budget, not a latency claim. Completion records are owned by an
exact comparison so reused head graphs cannot merge evidence from different bases. Live
leases have no JSON constructor; only the backend can validate and issue them in task 6.1.
Modified files: `cli.py` and `config.py` add explicit push command/configuration;
`model/regions.py` carries change context without Git-specific logic; `model/scan.py` consumes the
ranker's selected questions and returns exact selected/deferred IDs, deadline and reusable-graph
@@ -0,0 +1,60 @@
# Group 1 implementation verification
Date: 2026-09-12. Scope: tasks 1.1–1.4 only. All four tasks received independent
review with an APPROVED verdict and fresh task-local verification. Requirements,
design and tasks remain approved; implementation remains in progress (4/27 subtasks).
## Verification result
- STATUS: VERIFIED
- CLAIM_TYPE: TASK
- CLAIM: Group 1 establishes the reproducible runtime, evaluation inputs, honest
scoring and shared push/engine contracts.
- EVIDENCE: Final independent review ran the full suite: 965 passed, 9 skipped.
Focused contracts/configuration tests: 53 passed. Ruff and mypy passed for the
seven touched contract/configuration modules. `git diff --check` passed.
- GAPS: This is not feature-level GO. The pre-push runtime, persistent cache and
capability admission are not implemented. No end-to-end latency or precision
claim follows from these tests.
## Task evidence
| Task | Result and retained evidence |
| --- | --- |
| 1.1 | Joern 4.0.625 packaged and exercised with native PHP and JavaScript. Actual source bytes, graph census and query witnesses were verified; unreadable input failed explicitly. See `benchmarks/measurements/2026-09-12-engine-runtime-smoke.json`. Suite at review: 888 passed, 9 skipped. Commit `56cfc9d`. |
| 1.2 | Pinned PMPro and NodeGoat vulnerable, repaired and benign recipes; explicit authored-repair provenance. Offline validation read 506,502 bytes: 8 snapshots ready, 1 missing; 7 cases ready, 4 missing. Missing prerequisites cause exit 1. See `benchmarks/push/input-validation.json`. Suite: 902 passed, 9 skipped. Commit `8acada0`. |
| 1.3 | Frozen evaluation identity and uncertainty-preserving scorer. VAmPI retains all three known defects; unmatched SQL injection is not silently called transitively covered. Empty runs remain unmeasured. See `benchmarks/ranking/baselines/2026-09-12-vampi-position-v2.json` and `benchmarks/push/diagnostic-profile-v1.json`. Suite: 925 passed, 9 skipped. Commit `051f3b0`. |
| 1.4 | Strict immutable configuration, budget, question, scope, graph artifact and comparison-owned result contracts. Incomplete/unexecuted work cannot become complete through serialization. Existing scan configuration remains compatible; runtime enforcement is pending. Final suite: 965 passed, 9 skipped. |
## Packaged runtime and limitations
After the final source changes, `docker compose build ousast` exited 0. Final local
image: `sha256:92d311097075a2092a2e362b5470d706734f9de35de7820c3de5eb1ad91b17d5`.
A network-disabled container imported the new contracts, read an 83-byte TOML
configuration, round-tripped its push settings, rejected a NaN deadline, and ran
`python -m openultrasast.cli --help` successfully. Runtime: UID 1000, Python 3.12.3.
The earlier graph smoke records its own image identity; the backend did not change
after that smoke. The final packaging check does not repeat or replace graph evidence.
Full-tree `.venv/bin/mypy src/openultrasast` still exits 1 with three pre-existing
errors in `model/scan.py` at lines 299, 302 and 310 (candidate redefinition and
incompatible candidate types). It checked 107 source files with no additional errors.
Skipped tests and the unavailable evaluation prerequisites remain outside demonstrated
coverage. Ghost is a reserved workload, not a scanned or admitted positive population;
independent paired/holdout evidence remains required per capability.
## Design direction and next work
The ranker remains the scope mechanism. Vendor exclusion applies to graph inputs
as well as targets. PMPro guides core development and JavaScript/Express tests
transfer; Python regressions and C-family abstraction limits remain explicit.
Persist validated immutable graphs first. Separate graph, query/evidence and ranked
result identities so ranking-only changes can reuse compatible graphs and evidence.
Differential graph mutation is a future experiment requiring equivalence to cold
rebuilds, including deletions, renames, dependent edges and overlays. No stable
Joern node-ID assumption or partial-file update is accepted as proof of correctness.
Next: group 2 immutable push snapshots and comparison resolution, then the first
complete PMPro/Node experiment at task 5.2. Persistent caching remains group 6;
normal advisory capability admission remains gated by task 8.3.
+3 -1
View File
@@ -6,7 +6,9 @@
- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism;
third-party vendor code must remain outside targets and graphs.
- Requirements and design are proposals, not approved or implemented. No hook latency claim is made.
- Requirements, design and tasks are now approved, and group 1 implementation is verified complete; see
`tasks.md` for verified completion. The discovery log below preserves the earlier drafting state
and measurements. No hook latency claim is made.
## Research Log
+12 -1
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-12T16:20:40.814596Z",
"updated_at": "2026-09-12T17:21:05.783007Z",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -50,5 +50,16 @@
"executable_subtasks": 27,
"requirements_covered": 35,
"record": "task-review.md"
},
"implementation_progress": {
"completed_tasks": [
"1.1",
"1.2",
"1.3",
"1.4"
],
"remaining_tasks": 23,
"verification_record": "implementation-group-1.md",
"next_group": 2
}
}
+6 -4
View File
@@ -1,7 +1,8 @@
# Implementation Plan: pre-push-safety-net
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Implementation has
started with group 1 (tasks 1.1–1.4); each task remains pending until independent review and fresh verification pass.
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Group 1 (tasks 1.1–1.4)
is complete after independent review and fresh verification. Remaining groups are pending;
the hook and capability admission are not yet implemented. See `implementation-group-1.md`.
## Execution contract
@@ -21,7 +22,7 @@ started with group 1 (tasks 1.1–1.4); each task remains pending until independ
- Sequential order is the default. Only 3.1 may run alongside group 2 after its explicit prerequisite;
shared backend/driver, cache and report edits otherwise remain serial. No optional acceptance tests.
- [ ] 1. Establish the runtime, evaluation inputs and shared contracts
- [x] 1. Establish the runtime, evaluation inputs and shared contracts
- [x] 1.1 Reproduce the evaluated engine environment
- Align the shipped image, Compose and host-install defaults with Joern 4.0.625; preserve version override and checksum verification.
- Exercise PHP with the native interpreter and the installed JavaScript frontend; open a source and the frontend parser where applicable, recording byte counts and graph census.
@@ -45,13 +46,14 @@ started with group 1 (tasks 1.1–1.4); each task remains pending until independ
- _Boundary: Evaluation harness, repository benchmark integration_
- _Requirements: 6.2, 6.3, 6.4, 6.5, 7.3_
- _Verified 2026-09-12:_ independent review APPROVED; 925 tests passed, 9 skipped. Parent reproduced the frozen profile and actual VAmPI static rerun (15,942 pinned bytes, all 3 targets retained, SQLI unmatched, no transitive/detection claim, exit 1). Empty-run scoring retains 11 unresolved cases and explicitly reports unmeasured/0 recorded cases. Historical baselines remain unchanged with a caveat; new scorer consumes associated execution/review evidence and does not implement replay or admission.
- [ ] 1.4 Establish typed push and engine integration contracts
- [x] 1.4 Establish typed push and engine integration contracts
- Add importable push components and validated configuration for comparison base, shared deadline/cancellation allowance, local cache limit, advisory/blocking and separate coverage policy.
- Define ownership of update/comparison, change context, selected/deferred scope, graph identity/lease and separate finding/coverage/push statuses as described in design.
- Keep generic engine contracts outside the push package; leave ordinary scan defaults compatible and capabilities experimental until evaluated.
- Done when contract/config round trips preserve all identities/statuses, invalid values are rejected, and the engine has no dependency on the push package.
- _Boundary: Push runner, Cache and Engine Boundary, CLI/config integration_
- _Requirements: 4.1, 4.3, 5.3, 5.4, 5.5, 7.1, 7.3_
- _Verified 2026-09-12:_ independent review APPROVED; 965 tests passed, 9 skipped; touched-source Ruff/mypy passed. Parent rebuilt the local image and verified non-root, network-disabled CLI startup, all contract imports, an 83-byte configuration round trip and invalid-budget rejection. Comparison-owned completion preserves different bases for a shared head. Contracts are structural; runtime snapshot/cache/runner/admission implementations remain in later tasks.
- [ ] 2. Resolve immutable pushed content
- [ ] 2.1 Resolve every supplied ref update and comparison