feat(pre-push-safety-net): establish shared push and engine contracts

This commit is contained in:
Marius Ciepluch
2026-09-12 19:21:31 +02:00
parent 051f3b0058
commit 14a2a60e52
18 changed files with 810 additions and 25 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# Brief: pre-push-safety-net
Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation started. Language: English.
Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation verified complete; groups 2–8 pending. Language: English.
## Problem
+16 -2
View File
@@ -132,8 +132,8 @@ Concrete workload roles and required paired changes are in `evaluation.md`.
| Storage | Local content-addressed directories and JSON manifests | Atomic reusable artifacts, no database |
| Presentation | Existing finding records plus compact hook renderer | Separate findings and coverage |
Align Docker, Compose and provisioning with the evaluated Joern version before any deployment
claim. The old defaults are 4.0.623. Pin alignment is predecessor packaging work, not completed here.
Docker, Compose and provisioning defaults were aligned to 4.0.625 in task 1.1, with a verified
local PHP/JavaScript image smoke. Host deployment and hook performance remain separate claims.
## File Structure Plan
@@ -141,6 +141,10 @@ New files (proposed, not created by this documentation change):
| Path | Responsibility |
|---|---|
| `src/openultrasast/contracts.py` | Strict immutable manifest serialization shared by engine and adapters |
| `src/openultrasast/model/contracts.py` | Generic deadline, change context and exact selected/deferred scope contracts |
| `src/openultrasast/cpg/artifact.py` | Backend-owned graph identity, census and live lease protocol |
| `src/openultrasast/push/contracts.py` | Immutable ref/comparison identities and comparison-owned completion/status records |
| `src/openultrasast/push/snapshot.py` | Parse ref updates, resolve bases, materialize blobs, produce change context |
| `src/openultrasast/push/cache.py` | Artifact keys, validity, atomic publication and eviction |
| `src/openultrasast/push/policy.py` | Delta identity, actionability admission and push disposition |
@@ -149,6 +153,16 @@ New files (proposed, not created by this documentation change):
| `benchmarks/push/README.md` and `benchmarks/push/measure.py` | Labeled push recipes and joint quality/latency measurement |
| `tests/test_push_snapshot.py`, `tests/test_push_cache.py`, `tests/test_push_policy.py`, `tests/test_push_runner.py` | Contract and integration checks below |
Contract configuration (task 1.4) uses `[push]`: optional `comparison_base`, positive
`deadline_seconds` (30), positive `cancellation_allowance_seconds` (2), positive integer
`cache_max_bytes` (2 GiB), `mode` (`advisory` by default, or `blocking`) and independent
`incomplete_coverage_policy` (`allow` by default, or `block`). Unknown keys, malformed
sections and coerced/nonfinite numeric values are rejected. These settings are inert until
runner integration; they do not enable a hook or admit experimental capabilities. The cache
limit is a local storage budget, not a latency claim. Completion records are owned by an
exact comparison so reused head graphs cannot merge evidence from different bases. Live
leases have no JSON constructor; only the backend can validate and issue them in task 6.1.
Modified files: `cli.py` and `config.py` add explicit push command/configuration;
`model/regions.py` carries change context without Git-specific logic; `model/scan.py` consumes the
ranker's selected questions and returns exact selected/deferred IDs, deadline and reusable-graph
@@ -0,0 +1,60 @@
# Group 1 implementation verification
Date: 2026-09-12. Scope: tasks 1.1–1.4 only. All four tasks received independent
review with an APPROVED verdict and fresh task-local verification. Requirements,
design and tasks remain approved; implementation remains in progress (4/27 subtasks).
## Verification result
- STATUS: VERIFIED
- CLAIM_TYPE: TASK
- CLAIM: Group 1 establishes the reproducible runtime, evaluation inputs, honest
scoring and shared push/engine contracts.
- EVIDENCE: Final independent review ran the full suite: 965 passed, 9 skipped.
Focused contracts/configuration tests: 53 passed. Ruff and mypy passed for the
seven touched contract/configuration modules. `git diff --check` passed.
- GAPS: This is not feature-level GO. The pre-push runtime, persistent cache and
capability admission are not implemented. No end-to-end latency or precision
claim follows from these tests.
## Task evidence
| Task | Result and retained evidence |
| --- | --- |
| 1.1 | Joern 4.0.625 packaged and exercised with native PHP and JavaScript. Actual source bytes, graph census and query witnesses were verified; unreadable input failed explicitly. See `benchmarks/measurements/2026-09-12-engine-runtime-smoke.json`. Suite at review: 888 passed, 9 skipped. Commit `56cfc9d`. |
| 1.2 | Pinned PMPro and NodeGoat vulnerable, repaired and benign recipes; explicit authored-repair provenance. Offline validation read 506,502 bytes: 8 snapshots ready, 1 missing; 7 cases ready, 4 missing. Missing prerequisites cause exit 1. See `benchmarks/push/input-validation.json`. Suite: 902 passed, 9 skipped. Commit `8acada0`. |
| 1.3 | Frozen evaluation identity and uncertainty-preserving scorer. VAmPI retains all three known defects; unmatched SQL injection is not silently called transitively covered. Empty runs remain unmeasured. See `benchmarks/ranking/baselines/2026-09-12-vampi-position-v2.json` and `benchmarks/push/diagnostic-profile-v1.json`. Suite: 925 passed, 9 skipped. Commit `051f3b0`. |
| 1.4 | Strict immutable configuration, budget, question, scope, graph artifact and comparison-owned result contracts. Incomplete/unexecuted work cannot become complete through serialization. Existing scan configuration remains compatible; runtime enforcement is pending. Final suite: 965 passed, 9 skipped. |
## Packaged runtime and limitations
After the final source changes, `docker compose build ousast` exited 0. Final local
image: `sha256:92d311097075a2092a2e362b5470d706734f9de35de7820c3de5eb1ad91b17d5`.
A network-disabled container imported the new contracts, read an 83-byte TOML
configuration, round-tripped its push settings, rejected a NaN deadline, and ran
`python -m openultrasast.cli --help` successfully. Runtime: UID 1000, Python 3.12.3.
The earlier graph smoke records its own image identity; the backend did not change
after that smoke. The final packaging check does not repeat or replace graph evidence.
Full-tree `.venv/bin/mypy src/openultrasast` still exits 1 with three pre-existing
errors in `model/scan.py` at lines 299, 302 and 310 (candidate redefinition and
incompatible candidate types). It checked 107 source files with no additional errors.
Skipped tests and the unavailable evaluation prerequisites remain outside demonstrated
coverage. Ghost is a reserved workload, not a scanned or admitted positive population;
independent paired/holdout evidence remains required per capability.
## Design direction and next work
The ranker remains the scope mechanism. Vendor exclusion applies to graph inputs
as well as targets. PMPro guides core development and JavaScript/Express tests
transfer; Python regressions and C-family abstraction limits remain explicit.
Persist validated immutable graphs first. Separate graph, query/evidence and ranked
result identities so ranking-only changes can reuse compatible graphs and evidence.
Differential graph mutation is a future experiment requiring equivalence to cold
rebuilds, including deletions, renames, dependent edges and overlays. No stable
Joern node-ID assumption or partial-file update is accepted as proof of correctness.
Next: group 2 immutable push snapshots and comparison resolution, then the first
complete PMPro/Node experiment at task 5.2. Persistent caching remains group 6;
normal advisory capability admission remains gated by task 8.3.
+3 -1
View File
@@ -6,7 +6,9 @@
- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism;
third-party vendor code must remain outside targets and graphs.
- Requirements and design are proposals, not approved or implemented. No hook latency claim is made.
- Requirements, design and tasks are now approved, and group 1 implementation is verified complete; see
`tasks.md` for verified completion. The discovery log below preserves the earlier drafting state
and measurements. No hook latency claim is made.
## Research Log
+12 -1
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-12T16:20:40.814596Z",
"updated_at": "2026-09-12T17:21:05.783007Z",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -50,5 +50,16 @@
"executable_subtasks": 27,
"requirements_covered": 35,
"record": "task-review.md"
},
"implementation_progress": {
"completed_tasks": [
"1.1",
"1.2",
"1.3",
"1.4"
],
"remaining_tasks": 23,
"verification_record": "implementation-group-1.md",
"next_group": 2
}
}
+6 -4
View File
@@ -1,7 +1,8 @@
# Implementation Plan: pre-push-safety-net
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Implementation has
started with group 1 (tasks 1.1–1.4); each task remains pending until independent review and fresh verification pass.
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Group 1 (tasks 1.1–1.4)
is complete after independent review and fresh verification. Remaining groups are pending;
the hook and capability admission are not yet implemented. See `implementation-group-1.md`.
## Execution contract
@@ -21,7 +22,7 @@ started with group 1 (tasks 1.1–1.4); each task remains pending until independ
- Sequential order is the default. Only 3.1 may run alongside group 2 after its explicit prerequisite;
shared backend/driver, cache and report edits otherwise remain serial. No optional acceptance tests.
- [ ] 1. Establish the runtime, evaluation inputs and shared contracts
- [x] 1. Establish the runtime, evaluation inputs and shared contracts
- [x] 1.1 Reproduce the evaluated engine environment
- Align the shipped image, Compose and host-install defaults with Joern 4.0.625; preserve version override and checksum verification.
- Exercise PHP with the native interpreter and the installed JavaScript frontend; open a source and the frontend parser where applicable, recording byte counts and graph census.
@@ -45,13 +46,14 @@ started with group 1 (tasks 1.1–1.4); each task remains pending until independ
- _Boundary: Evaluation harness, repository benchmark integration_
- _Requirements: 6.2, 6.3, 6.4, 6.5, 7.3_
- _Verified 2026-09-12:_ independent review APPROVED; 925 tests passed, 9 skipped. Parent reproduced the frozen profile and actual VAmPI static rerun (15,942 pinned bytes, all 3 targets retained, SQLI unmatched, no transitive/detection claim, exit 1). Empty-run scoring retains 11 unresolved cases and explicitly reports unmeasured/0 recorded cases. Historical baselines remain unchanged with a caveat; new scorer consumes associated execution/review evidence and does not implement replay or admission.
- [ ] 1.4 Establish typed push and engine integration contracts
- [x] 1.4 Establish typed push and engine integration contracts
- Add importable push components and validated configuration for comparison base, shared deadline/cancellation allowance, local cache limit, advisory/blocking and separate coverage policy.
- Define ownership of update/comparison, change context, selected/deferred scope, graph identity/lease and separate finding/coverage/push statuses as described in design.
- Keep generic engine contracts outside the push package; leave ordinary scan defaults compatible and capabilities experimental until evaluated.
- Done when contract/config round trips preserve all identities/statuses, invalid values are rejected, and the engine has no dependency on the push package.
- _Boundary: Push runner, Cache and Engine Boundary, CLI/config integration_
- _Requirements: 4.1, 4.3, 5.3, 5.4, 5.5, 7.1, 7.3_
- _Verified 2026-09-12:_ independent review APPROVED; 965 tests passed, 9 skipped; touched-source Ruff/mypy passed. Parent rebuilt the local image and verified non-root, network-disabled CLI startup, all contract imports, an 83-byte configuration round trip and invalid-budget rejection. Comparison-owned completion preserves different bases for a shared head. Contracts are structural; runtime snapshot/cache/runner/admission implementations remain in later tasks.
- [ ] 2. Resolve immutable pushed content
- [ ] 2.1 Resolve every supplied ref update and comparison
+8 -2
View File
@@ -1,17 +1,23 @@
# Overview — what this is, and what is actually true of it
**Current direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated
development across supported ecosystems. See `safety-net.md` and the proposed `pre-push-safety-net`
development across supported ecosystems. See `safety-net.md` and the approved `pre-push-safety-net`
spec. The ranker remains the scope mechanism; third-party vendor code stays outside targets and
graphs. Advisory must meet the same actionability bar as blocking.
**Historical scoreboard below:** it was written September 9. Later commits record a completing
whole-PMPro scan and Joern 4.0.625 ranking measurements (82/91/54 on the three plugins), while
default installation pins remain 4.0.623. The pre-push workflow, changed-code latency and actionable
default installation pins are now aligned to 4.0.625 with verified PHP/JavaScript image smoke.
The pre-push workflow, changed-code latency and actionable
precision are not demonstrated by those results. In particular, the manually selected two-file
slice below is a useful experiment, not evidence that an automatic push scope would select the
same context. Current scope/status corrections live in `flow-aware-ranking/brief.md` and the roadmap.
The new scoring instrument retains missing targets and unanswered queries. Historical rank positions
do not establish detection recall; VAmPI's old budget of 16 omitted its unmatched SQLI target. The
corrected positional run keeps that target unresolved. Labeled PMPro/Node inputs and a frozen
diagnostic profile are committed; independent capability evaluation is still required.
**Written 2026-09-09.** Fourteen specs have accumulated and the roadmap still described a pre-Joern world, so
this is the synthesis above them: what the system does, what has been measured, and what is not true yet.
Every number here comes from a committed artifact in `benchmarks/measurements/`.
+8 -7
View File
@@ -21,8 +21,9 @@ engine are out of scope.
- Whole-PMPro analysis has completed with its known CVE; the earlier "does not finish" diagnosis
is superseded. The recorded 72-minute scan is not a pre-push performance result.
- The latest Joern 4.0.625 + layout ranking budgets are PMPro 82, WP Statistics 91, MW WP Form 54,
VAmPI 16. `<50` remains unmet; VAmPI's transitive SQLI is not measured by this position metric.
- Historical Joern 4.0.625 + layout position budgets are PMPro 82, WP Statistics 91, MW WP Form 54.
`<50` remains unmet. VAmPI's old 16 omitted SQLI; the corrected instrument retains the unmatched
target and reports no full-target budget or detection recall without the required evidence.
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
@@ -45,14 +46,14 @@ engine are out of scope.
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
(8 groups, 27 executable subtasks), independently reviewed and approved; group 1 implementation started.
(8 groups, 27 executable subtasks), independently reviewed and approved; group 1 is complete (4/27 tasks).
### Next concrete step
The maintainer approved `pre-push-safety-net/tasks.md` and starting group 1. Requirements and design
were accepted when the maintainer requested the next Kiro phase; the 27-subtask plan passed independent
sanity review. Complete group 1 (reproducible runtime, readable PMPro/Node inputs, honest scoring
and shared contracts). The first complete experiment is 5.2, before persistent caching and hook
Group 1 has passed independent review: reproducible PHP/JavaScript runtime, pinned PMPro/Node inputs,
honest scoring and shared contracts. Continue with group 2's immutable push snapshots under the
approved task plan; see `pre-push-safety-net/implementation-group-1.md` for evidence and limits.
The first complete experiment is 5.2, before persistent caching and hook
installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through
the same ranker, with a real deadline and vendor-free graph. Include benign pushes and retain Python
+3 -2
View File
@@ -78,5 +78,6 @@ updates as a shortcut to an unmeasured latency promise.
The product direction above is user-requested. The amended requirements, design and initial
evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
("ok, next kiro phase"). Task approval remains pending. This is not authorization to implement
or a claim that hook performance or capability admission has been demonstrated.
("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
("approve."). Implementation is authorized under that plan; task completion still requires
independent review and fresh evidence. Hook performance and capability admission remain unproven.
@@ -1,10 +1,10 @@
{
"counts": {
"load_bearing": 92,
"load_bearing": 97,
"standalone_capability": 9
},
"modules": 101,
"note": "Inventory refreshed for pre-push-safety-net task 1.3 standalone scorer; original artifact filename retained for audit gate.",
"modules": 106,
"note": "Refreshed 2026-09-12 after pre-push-safety-net task 1.4: immutable push and generic engine contracts are spec-owned; runtime integration remains pending.",
"orphaned": [],
"rows": [
{
@@ -110,6 +110,17 @@
"module": "config",
"reason": "reachable from gate, map_gate, pair_gate or the CLI"
},
{
"classification": "load_bearing",
"importers": [
"config",
"cpg.artifact",
"model.contracts",
"push.contracts"
],
"module": "contracts",
"reason": "pre-push-safety-net Req 7.3: strict shared manifest serialization, independent of push"
},
{
"classification": "load_bearing",
"importers": [
@@ -125,6 +136,12 @@
"module": "cpg",
"reason": "model-grounded-detection Req 4: the CPG seam package"
},
{
"classification": "load_bearing",
"importers": [],
"module": "cpg.artifact",
"reason": "pre-push-safety-net Req 4.3/7.3: backend graph identity and live lease contract; validation integration pending"
},
{
"classification": "load_bearing",
"importers": [
@@ -333,7 +350,8 @@
"model.scan",
"model.specs",
"model.taint",
"preprocess"
"preprocess",
"push.contracts"
],
"module": "model",
"reason": "reachable from gate, map_gate, pair_gate or the CLI"
@@ -368,6 +386,14 @@
"module": "model.config_value",
"reason": "model-grounded-detection Req 6.3: constant abstraction for the configuration families"
},
{
"classification": "load_bearing",
"importers": [
"push.contracts"
],
"module": "model.contracts",
"reason": "pre-push-safety-net Req 4.1/7.3: generic deadline, change and exact scope contracts; integration pending"
},
{
"classification": "load_bearing",
"importers": [
@@ -605,6 +631,18 @@
"module": "provider.openrouter",
"reason": "reachable from gate, map_gate, pair_gate or the CLI"
},
{
"classification": "load_bearing",
"importers": [],
"module": "push",
"reason": "pre-push-safety-net Req 5.5/7.3: experimental push contract package; no hook installed"
},
{
"classification": "load_bearing",
"importers": [],
"module": "push.contracts",
"reason": "pre-push-safety-net Req 5.5/7.3: immutable comparisons and independent result statuses; runner pending"
},
{
"classification": "standalone_capability",
"importers": [],
+33 -1
View File
@@ -5,6 +5,9 @@ import os
import tomllib
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Literal
from openultrasast.contracts import Contract
DEFAULT_VECTOR_STORE = "json-local"
@@ -163,6 +166,28 @@ class ObligationsConfig:
policy_path: str = ".openultrasast/obligations.toml"
@dataclass(frozen=True)
class PushConfig(Contract):
"""Inert experimental push settings; configuring them does not install a hook.
The 2 GiB local cache cap is a storage budget, not a performance promise. The
eventual cache evicts unleased entries; deadline and cleanup are separate costs.
No endpoint, fetch, engine download or capability admission is enabled here.
"""
comparison_base: str | None = None
deadline_seconds: float = 30.0
cancellation_allowance_seconds: float = 2.0
cache_max_bytes: int = 2 * 1024 ** 3
mode: Literal["advisory", "blocking"] = "advisory"
incomplete_coverage_policy: Literal["allow", "block"] = "allow"
def __post_init__(self) -> None:
super().__post_init__()
if self.deadline_seconds <= 0 or self.cancellation_allowance_seconds <= 0 or self.cache_max_bytes <= 0:
raise ValueError("push deadlines, cancellation allowance and cache limit must be positive")
@dataclass(frozen=True)
class ResolvedConfig:
models: ModelConfig = ModelConfig()
@@ -181,6 +206,7 @@ class ResolvedConfig:
variants: VariantsConfig = VariantsConfig()
obligations: ObligationsConfig = ObligationsConfig()
model: ModelLayerConfig = ModelLayerConfig()
push: PushConfig = PushConfig()
runs_dir: str = ".openultrasast/runs"
@@ -208,13 +234,19 @@ def load_config(config_path: Path | None = None) -> ResolvedConfig:
variants=_load_variants(data.get("variants", {})),
obligations=_load_obligations(data.get("obligations", {})),
model=_load_model(data.get("model", {})),
push=PushConfig.from_payload(data.get("push", {})),
runs_dir=os.environ.get("OPENULTRASAST_RUNS_DIR", ".openultrasast/runs"),
)
def config_payload(config: ResolvedConfig) -> dict[str, object]:
"""Reproducible resolved settings, including the separate experimental push policy."""
return asdict(config)
def write_resolved_config(config: ResolvedConfig, path: Path) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(asdict(config), indent=2, sort_keys=True) + "\n")
path.write_text(json.dumps(config_payload(config), indent=2, sort_keys=True) + "\n")
def _section(value: object) -> dict[str, object]:
+92
View File
@@ -0,0 +1,92 @@
"""Strict immutable manifest values shared by engine and adapters (pre-push Req 7.3).
These schemas validate structure, not source readability, Git object existence or alert
eligibility. Live resources deliberately have no manifest decoder.
"""
from __future__ import annotations
import math
from dataclasses import fields
from types import UnionType
from typing import Any, Literal, Self, get_args, get_origin, get_type_hints
_HINTS: dict[type, dict[str, Any]] = {}
def _hints(cls: type) -> dict[str, Any]:
if cls not in _HINTS:
_HINTS[cls] = get_type_hints(cls)
return _HINTS[cls]
def _value(value: object, kind: Any, *, decode: bool) -> object:
origin, args = get_origin(kind), get_args(kind)
if origin is UnionType:
for candidate in args:
try:
return _value(value, candidate, decode=decode)
except ValueError:
pass
raise ValueError(f"invalid union value: {value!r}")
if origin is Literal:
if any(type(value) is type(option) and value == option for option in args):
return value
elif origin is tuple:
if type(value) is tuple or (decode and type(value) is list):
items = tuple(value) # type: ignore[arg-type]
if len(args) == 2 and args[1] is Ellipsis:
return tuple(_value(item, args[0], decode=decode) for item in items)
if len(items) == len(args):
return tuple(_value(item, item_type, decode=decode) for item, item_type in zip(items, args, strict=True))
elif isinstance(kind, type) and issubclass(kind, Contract):
if decode:
return kind.from_payload(value)
if isinstance(value, kind):
return value
elif kind is float:
if type(value) in (int, float):
try:
if math.isfinite(value): # type: ignore[arg-type]
return value
except OverflowError:
pass
elif kind is str:
if type(value) is str and value.strip():
return value
elif type(value) is kind:
return value
raise ValueError(f"invalid {kind}: {value!r}")
def _payload(value: object) -> object:
if isinstance(value, Contract):
return value.to_payload()
if isinstance(value, tuple):
return [_payload(item) for item in value]
return value
class Contract:
"""Base for frozen dataclasses with strict construction and JSON round trips."""
def __post_init__(self) -> None:
for name, kind in _hints(type(self)).items():
try:
_value(getattr(self, name), kind, decode=False)
except ValueError as error:
raise ValueError(f"{type(self).__name__}.{name}: {error}") from error
def to_payload(self) -> dict[str, object]:
return {item.name: _payload(getattr(self, item.name)) for item in fields(self)} # type: ignore[arg-type]
@classmethod
def from_payload(cls, payload: object) -> Self:
if not isinstance(payload, dict) or not all(isinstance(key, str) for key in payload):
raise ValueError(f"{cls.__name__} requires an object")
hints = _hints(cls)
if unknown := payload.keys() - hints.keys():
raise ValueError(f"unknown {cls.__name__} fields: {sorted(unknown)}")
try:
return cls(**{name: _value(value, hints[name], decode=True) for name, value in payload.items()})
except TypeError as error:
raise ValueError(f"invalid {cls.__name__}: {error}") from error
+82
View File
@@ -0,0 +1,82 @@
"""Backend-owned graph provenance and live resource boundary (pre-push Req 4.3, 7.3).
Structural validity is necessary but insufficient for reuse. Task 6.1 will validate
actual source bytes, graph digest/census and engine compatibility before issuing leases.
"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Literal, Protocol
from openultrasast.contracts import Contract
@dataclass(frozen=True)
class GraphIdentity(Contract):
source_digest: str
declarations_digest: str
exclusions_digest: str
unit: str
language: str
engine_version: str
frontend: str
frontend_version: str
overlay_semantics: tuple[str, ...]
overlay_options: tuple[tuple[str, str], ...]
frontend_options: tuple[tuple[str, str], ...] = ()
def __post_init__(self) -> None:
super().__post_init__()
for options in (self.overlay_options, self.frontend_options):
if len({key for key, _ in options}) != len(options):
raise ValueError("graph option names must be unique")
@dataclass(frozen=True)
class GraphCensus(Contract):
files: int
methods: int
calls: int
def __post_init__(self) -> None:
super().__post_init__()
if min(self.files, self.methods, self.calls) < 0:
raise ValueError("graph census counts must be nonnegative")
@dataclass(frozen=True)
class GraphArtifact(Contract):
identity: GraphIdentity
graph_digest: str
source_bytes: int
census: GraphCensus
completeness: Literal["complete", "partial"]
diagnostics: tuple[str, ...]
def __post_init__(self) -> None:
super().__post_init__()
if self.source_bytes < 0:
raise ValueError("source byte count must be nonnegative")
if self.completeness == "complete" and (not self.source_bytes or not self.census.files or self.diagnostics):
raise ValueError("complete graph requires readable source, a file census and no incomplete diagnostics")
if self.completeness == "partial" and not self.diagnostics:
raise ValueError("partial graph requires a recorded reason")
class GraphLease(Protocol):
"""A backend-issued live lease; never reconstructed from a JSON manifest.
The backend validates the artifact before issuing the lease and isolates loading
when it may mutate graph bytes. The consumer owns releasing it exactly once; the
backend's release is idempotent and bounded by the supplied monotonic deadline.
A cached artifact survives release. Implementations belong to task 6.1.
"""
@property
def artifact(self) -> GraphArtifact: ...
@property
def path(self) -> Path: ...
def close(self, *, deadline_monotonic: float) -> None: ...
+5
View File
@@ -28,6 +28,11 @@ ROOTS = ("gate", "map_gate", "pair_gate", "cli", "__main__")
# Modules that stand alone: their own CLI subcommand or external protocol, deliberately off the scan path.
SPEC_OWNED = {
"contracts": "pre-push-safety-net Req 7.3: strict shared manifest serialization, independent of push",
"model.contracts": "pre-push-safety-net Req 4.1/7.3: generic deadline, change and exact scope contracts; integration pending",
"cpg.artifact": "pre-push-safety-net Req 4.3/7.3: backend graph identity and live lease contract; validation integration pending",
"push": "pre-push-safety-net Req 5.5/7.3: experimental push contract package; no hook installed",
"push.contracts": "pre-push-safety-net Req 5.5/7.3: immutable comparisons and independent result statuses; runner pending",
"model.specs": "model-grounded-detection Req 7.2-7.4: the security vocabularies the CPG queries are parameterised by",
"model.endpoint": "model-grounded-detection Req 8: the judge client, wired by model/judge.py in group 3",
"model.audit": "model-grounded-detection Req 3: this classifier; run by the maintainer, not by the scan",
+115
View File
@@ -0,0 +1,115 @@
"""Language-agnostic change evidence and exact ranker scope (pre-push Req 7.3).
These are integration contracts; constructing them neither selects nor executes work.
Revision identities are opaque to the engine. The snapshot adapter resolves Git objects.
"""
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from typing import Literal
from openultrasast.contracts import Contract
@dataclass(frozen=True)
class ExecutionBudget(Contract):
"""One absolute monotonic deadline, with a distinct bounded cleanup allowance.
Process-local: do not carry the absolute clock value across process restarts.
Manifests should record elapsed stage costs and configured durations instead.
"""
deadline_monotonic: float
cancellation_allowance_seconds: float
def __post_init__(self) -> None:
super().__post_init__()
if self.deadline_monotonic < 0 or self.cancellation_allowance_seconds <= 0:
raise ValueError("deadline must be nonnegative and cancellation allowance positive")
@dataclass(frozen=True)
class QuestionIdentity(Contract):
unit: str
language: str
path: str
function: str | None
family: str
@property
def question_id(self) -> str:
encoded = json.dumps(self.to_payload(), sort_keys=True, separators=(",", ":")).encode()
return hashlib.sha256(encoded).hexdigest()
@dataclass(frozen=True)
class ChangedSpan(Contract):
path: str
start_line: int
end_line: int
side: Literal["base", "head"]
def __post_init__(self) -> None:
super().__post_init__()
if self.start_line < 1 or self.end_line < self.start_line:
raise ValueError("changed spans require positive inclusive ordered lines")
@dataclass(frozen=True)
class PathRename(Contract):
base_path: str
head_path: str
@dataclass(frozen=True)
class AffectedRelationship(Contract):
source: QuestionIdentity
target: QuestionIdentity
kind: str
evidence: tuple[str, ...]
@dataclass(frozen=True)
class ChangeContext(Contract):
base_revision: str | None
head_revision: str
changed_paths: tuple[str, ...]
deleted_paths: tuple[str, ...]
spans: tuple[ChangedSpan, ...]
renames: tuple[PathRename, ...]
declaration_paths: tuple[str, ...]
relationships: tuple[AffectedRelationship, ...]
unresolved_boundaries: tuple[str, ...]
@dataclass(frozen=True)
class RankedQuestion(Contract):
identity: QuestionIdentity
priority: float
evidence: tuple[str, ...]
@dataclass(frozen=True)
class DeferredQuestion(Contract):
identity: QuestionIdentity
reason: str
evidence: tuple[str, ...]
@dataclass(frozen=True)
class ScopeDecision(Contract):
ranker_version: str
ranking_mode: str
# Explicit census assertion: an empty list by itself is never proof of no work.
population_complete: bool
selected: tuple[RankedQuestion, ...]
deferred: tuple[DeferredQuestion, ...]
unresolved_boundaries: tuple[str, ...]
def __post_init__(self) -> None:
super().__post_init__()
identities = [item.identity for item in self.selected] + [item.identity for item in self.deferred]
if len(set(identities)) != len(identities):
raise ValueError("question identities must be unique across selected and deferred scope")
+1
View File
@@ -0,0 +1 @@
"""Experimental pre-push contracts; no hook or admitted capability is enabled here."""
+86
View File
@@ -0,0 +1,86 @@
"""Push-owned immutable identities and independent result statuses (Req 5.5, 7.3).
Git resolution, admission policy and execution are separate later implementations.
No default result status can turn an unexecuted contract into a complete negative.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal
from openultrasast.contracts import Contract
from openultrasast.model.contracts import QuestionIdentity, ScopeDecision
@dataclass(frozen=True)
class PushUpdate(Contract):
local_ref: str
local_oid: str
remote_ref: str
remote_oid: str
@dataclass(frozen=True)
class PushComparison(Contract):
head_oid: str
base_oid: str | None
base_reason: str
refs: tuple[str, ...]
def __post_init__(self) -> None:
super().__post_init__()
if not self.refs or len(set(self.refs)) != len(self.refs):
raise ValueError("comparison must retain distinct associated refs")
@dataclass(frozen=True)
class ComparisonAnalysis(Contract):
"""Scope and completion belong to this exact base/head comparison.
A question completed for one comparison never completes it for another base,
even when the target graph is shared. This record does not assert admission.
"""
comparison: PushComparison
scopes: tuple[ScopeDecision, ...]
completed_questions: tuple[QuestionIdentity, ...]
coverage_status: Literal["complete_within_scope", "incomplete", "unavailable"]
def __post_init__(self) -> None:
super().__post_init__()
selected_list = [item.identity for scope in self.scopes for item in scope.selected]
selected = set(selected_list)
completed = set(self.completed_questions)
if len(selected) != len(selected_list):
raise ValueError("a comparison cannot select the same question across multiple scopes")
if len(completed) != len(self.completed_questions) or not completed <= selected:
raise ValueError("completed questions must be unique members of this comparison's selected scope")
if self.coverage_status == "complete_within_scope":
if self.comparison.base_oid is None or not self.scopes:
raise ValueError("complete coverage requires compared revisions and an explicit scope census")
if completed != selected or any(
not scope.population_complete or scope.deferred or scope.unresolved_boundaries for scope in self.scopes
):
raise ValueError("deferred, unexecuted or unresolved work cannot be complete coverage")
@dataclass(frozen=True)
class PushResult(Contract):
analyses: tuple[ComparisonAnalysis, ...]
finding_status: Literal["actionable", "none"]
coverage_status: Literal["complete_within_scope", "incomplete", "unavailable", "not_applicable"]
push_disposition: Literal["allow", "block"]
actionable_defect_ids: tuple[str, ...]
def __post_init__(self) -> None:
super().__post_init__()
if (self.finding_status == "actionable") != bool(self.actionable_defect_ids):
raise ValueError("finding status must match admitted defect identities")
if len(set(self.actionable_defect_ids)) != len(self.actionable_defect_ids):
raise ValueError("admitted defect identities must be unique")
if self.coverage_status == "complete_within_scope" and (
not self.analyses or any(item.coverage_status != "complete_within_scope" for item in self.analyses)
):
raise ValueError("aggregate complete coverage requires completed comparison analyses")
if self.coverage_status == "not_applicable" and (self.analyses or self.actionable_defect_ids):
raise ValueError("not applicable cannot carry analyzed scope or findings")
+237
View File
@@ -0,0 +1,237 @@
"""Approved task 1.4 contracts: persist identities without inventing completed analysis."""
import ast
import json
from dataclasses import FrozenInstanceError, replace
from pathlib import Path
import pytest
from openultrasast.config import PushConfig, config_payload, load_config
from openultrasast.contracts import Contract
from openultrasast.cpg.artifact import GraphArtifact, GraphCensus, GraphIdentity, GraphLease
from openultrasast.model.contracts import (
ChangeContext,
ChangedSpan,
DeferredQuestion,
QuestionIdentity,
RankedQuestion,
ScopeDecision,
)
from openultrasast.push.contracts import ComparisonAnalysis, PushComparison, PushResult, PushUpdate
def question(language: str = "javascript", path: str = "src/api.js") -> QuestionIdentity:
return QuestionIdentity("api", language, path, "handler", "injection")
def scope() -> ScopeDecision:
return ScopeDecision("core-v1", "evidence", True, (RankedQuestion(question(), 2.5, ("source→operation",)),), (), ())
def comparison() -> PushComparison:
return PushComparison("a" * 64, "b" * 64, "supplied_remote_tip", ("refs/heads/main",))
def test_config_round_trip_and_ordinary_scan_compatibility(tmp_path: Path) -> None:
baseline = load_config()
path = tmp_path / "settings.toml"
path.write_text('[push]\ncomparison_base = "refs/heads/main"\ndeadline_seconds = 11.5\n'
'cancellation_allowance_seconds = 1\ncache_max_bytes = 4096\nmode = "blocking"\n'
'incomplete_coverage_policy = "block"\n')
configured = load_config(path)
assert configured.push == PushConfig("refs/heads/main", 11.5, 1, 4096, "blocking", "block")
assert PushConfig.from_payload(json.loads(json.dumps(config_payload(configured)["push"]))) == configured.push
before, after = config_payload(baseline), config_payload(configured)
before.pop("push")
after.pop("push")
assert before == after
assert baseline.push == PushConfig()
assert baseline.push.deadline_seconds == 30
assert baseline.push.cancellation_allowance_seconds == 2
assert baseline.push.cache_max_bytes == 2 * 1024 ** 3
assert baseline.push.mode == "advisory"
assert baseline.push.incomplete_coverage_policy == "allow"
@pytest.mark.parametrize("setting", [
'push = "bad"', 'push = []', '[push]\ndeadline_seconds = true', '[push]\ndeadline_seconds = "30"',
'[push]\ndeadline_seconds = nan', '[push]\ndeadline_seconds = inf', '[push]\ndeadline_seconds = 0',
'[push]\ncancellation_allowance_seconds = -1', '[push]\ncache_max_bytes = 0',
'[push]\ncache_max_bytes = 1.5', '[push]\ncache_max_bytes = true', '[push]\nmode = "nag"',
'[push]\nincomplete_coverage_policy = "ignore"', '[push]\ncomparison_base = ""',
'[push]\ncomparison_base = 1', '[push]\ndeadline_second = 60',
])
def test_push_configuration_rejects_unsafe_coercion(tmp_path: Path, setting: str) -> None:
path = tmp_path / "settings.toml"
path.write_text(setting)
with pytest.raises(ValueError):
load_config(path)
def test_contract_round_trips_preserve_partition_ref_and_comparison_identity() -> None:
update = PushUpdate("refs/heads/topic", "a" * 64, "refs/heads/main", "b" * 64)
context = ChangeContext("b" * 64, "a" * 64, ("src/api.js",), ("src/guard.js",),
(ChangedSpan("src/guard.js", 3, 8, "base"),), (), ("package.json",), (), ("dynamic dispatch",))
for value in (update, comparison(), context, scope()):
assert type(value).from_payload(json.loads(json.dumps(value.to_payload()))) == value
assert question() != question("php", "src/api.js")
assert question().question_id != question("php", "src/api.js").question_id
other_base = replace(comparison(), base_oid="c" * 64)
assert other_base != comparison()
with pytest.raises(FrozenInstanceError):
update.local_oid = "other" # type: ignore[misc]
@pytest.mark.parametrize("finding,coverage,disposition", [
("none", "incomplete", "allow"), ("none", "unavailable", "block"),
("actionable", "incomplete", "allow"), ("actionable", "complete_within_scope", "block"),
])
def test_result_statuses_are_independent(finding: str, coverage: str, disposition: str) -> None:
payload = {
"analyses": [ComparisonAnalysis(comparison(), (scope(),), (question(),), coverage).to_payload()],
"finding_status": finding,
"coverage_status": coverage, "push_disposition": disposition,
"actionable_defect_ids": ["defect-1"] if finding == "actionable" else [],
}
result = PushResult.from_payload(payload)
assert PushResult.from_payload(json.loads(json.dumps(result.to_payload()))) == result
assert result.finding_status == finding
assert result.coverage_status == coverage
assert result.push_disposition == disposition
def test_incomplete_scope_cannot_be_serialized_as_complete_negative() -> None:
result = ComparisonAnalysis(comparison(), (scope(),), (question(),), "complete_within_scope")
assert ComparisonAnalysis.from_payload(result.to_payload()) == result
deferred = DeferredQuestion(question("php", "index.php"), "deadline", ("unanswered",))
for changed in (replace(scope(), deferred=(deferred,)), replace(scope(), unresolved_boundaries=("vendor call",)),
replace(scope(), population_complete=False)):
with pytest.raises(ValueError):
replace(result, scopes=(changed,))
with pytest.raises(ValueError):
replace(result, completed_questions=())
with pytest.raises(ValueError):
replace(result, scopes=())
with pytest.raises(ValueError):
replace(result, comparison=replace(comparison(), base_oid=None, base_reason="missing"))
with pytest.raises(ValueError):
replace(scope(), deferred=(DeferredQuestion(question(), "deadline", ()),))
@pytest.mark.parametrize("value", [
{"local_ref": "x", "local_oid": "a", "remote_ref": "y", "remote_oid": "b", "extra": 1},
{"local_ref": "x", "local_oid": 1, "remote_ref": "y", "remote_oid": "b"},
{"local_ref": "x", "local_oid": "a", "remote_ref": "y"},
])
def test_malformed_identity_payloads_fail(value: dict[str, object]) -> None:
with pytest.raises(ValueError):
PushUpdate.from_payload(value)
def test_graph_provenance_round_trip_and_live_lease_separation() -> None:
identity = GraphIdentity("source-digest", "declarations", "exclusions", "api", "javascript",
"4.0.625", "jssrc2cpg", "4.0.625", ("dataflowOss-v1",), (("option", "value"),))
artifact = GraphArtifact(identity, "graph-sha256", 128, GraphCensus(1, 2, 3), "complete", ())
assert GraphArtifact.from_payload(json.loads(json.dumps(artifact.to_payload()))) == artifact
assert replace(identity, exclusions_digest="changed") != identity
assert not issubclass(GraphLease, Contract) # a manifest cannot restore a live resource
with pytest.raises(ValueError):
replace(artifact, source_bytes=0)
with pytest.raises(ValueError):
replace(artifact, diagnostics=("file unreadable",))
with pytest.raises(ValueError):
replace(artifact, census=GraphCensus(0, 0, 0))
with pytest.raises(ValueError):
replace(identity, overlay_options=(("x", "1"), ("x", "2")))
def test_strict_runtime_construction_not_only_decoding() -> None:
with pytest.raises(ValueError):
RankedQuestion(question(), float("nan"), ())
with pytest.raises(ValueError):
ChangedSpan("x", True, 3, "head")
with pytest.raises(ValueError):
ChangedSpan("x", 4, 3, "head")
with pytest.raises(ValueError):
PushConfig(deadline_seconds=True)
with pytest.raises(ValueError):
PushConfig.from_payload({"mode": "blocking", "incomplete_coverage_policy": []})
def test_generic_contracts_and_engine_do_not_depend_on_push() -> None:
root = Path("src/openultrasast")
paths = [root / "contracts.py", *(root / "model").glob("*.py"), *(root / "cpg").glob("*.py")]
for path in paths:
for node in ast.walk(ast.parse(path.read_text())):
if isinstance(node, ast.ImportFrom):
assert "push" not in (node.module or "").split("."), path
elif isinstance(node, ast.Import):
assert all("push" not in alias.name.split(".") for alias in node.names), path
def test_shared_head_does_not_merge_different_base_completion() -> None:
completed = ComparisonAnalysis(comparison(), (scope(),), (question(),), "complete_within_scope")
pending = ComparisonAnalysis(replace(comparison(), base_oid="c" * 64), (scope(),), (), "incomplete")
result = PushResult((completed, pending), "none", "incomplete", "allow", ())
restored = PushResult.from_payload(json.loads(json.dumps(result.to_payload())))
assert restored == result
assert restored.analyses[1].completed_questions == ()
with pytest.raises(ValueError):
replace(result, coverage_status="complete_within_scope")
with pytest.raises(ValueError):
replace(pending, coverage_status="complete_within_scope")
def test_deletion_only_result_is_explicitly_not_applicable() -> None:
result = PushResult((), "none", "not_applicable", "allow", ())
assert PushResult.from_payload(result.to_payload()) == result
with pytest.raises(ValueError):
replace(result, coverage_status="complete_within_scope")
@pytest.mark.parametrize("field,value", [
("finding_status", "clean"), ("coverage_status", "secure"), ("push_disposition", "warn"),
("finding_status", True), ("coverage_status", None),
])
def test_result_rejects_unknown_status_vocabulary(field: str, value: object) -> None:
payload = PushResult((), "none", "not_applicable", "allow", ()).to_payload()
payload[field] = value
with pytest.raises(ValueError):
PushResult.from_payload(payload)
def test_graph_partial_provenance_stays_partial_and_options_are_preserved() -> None:
identity = GraphIdentity("sources", "decls", "excludes", "core", "c", "4.0.625", "c2cpg", "4.0.625",
("dataflowOss-v1",), (("depth", "2"),), (("include", "src"),))
artifact = GraphArtifact(identity, "digest", 0, GraphCensus(0, 0, 0), "partial", ("source unreadable",))
restored = GraphArtifact.from_payload(json.loads(json.dumps(artifact.to_payload())))
assert restored == artifact
assert restored.identity.frontend_options == (("include", "src"),)
with pytest.raises(ValueError):
replace(restored, completeness="complete")
def test_execution_budget_requires_finite_clock_and_distinct_positive_allowance() -> None:
from openultrasast.model.contracts import ExecutionBudget
budget = ExecutionBudget(123.0, 2.0)
assert budget.deadline_monotonic == 123
assert budget.cancellation_allowance_seconds == 2
for deadline, allowance in ((float("inf"), 2), (1, float("nan")), (True, 2), (1, 0), (-1, 2)):
with pytest.raises(ValueError):
ExecutionBudget(deadline, allowance)
def test_empty_population_needs_an_explicit_census_assertion() -> None:
empty = ScopeDecision("core-v1", "evidence", True, (), (), ())
analyzed = ComparisonAnalysis(comparison(), (empty,), (), "complete_within_scope")
assert analyzed.completed_questions == ()
with pytest.raises(ValueError):
replace(analyzed, scopes=(replace(empty, population_complete=False),))
def test_direct_mutable_collection_is_not_an_immutable_contract() -> None:
with pytest.raises(ValueError):
PushComparison("head", "base", "supplied", ["refs/heads/main"]) # type: ignore[arg-type]
with pytest.raises(ValueError):
PushResult.from_payload({"finding_status": "none"})