diff --git a/.kiro/specs/pre-push-safety-net/brief.md b/.kiro/specs/pre-push-safety-net/brief.md index d5e60c5..ef56f50 100644 --- a/.kiro/specs/pre-push-safety-net/brief.md +++ b/.kiro/specs/pre-push-safety-net/brief.md @@ -1,6 +1,6 @@ # Brief: pre-push-safety-net -Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation started. Language: English. +Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation verified complete; groups 2–8 pending. Language: English. ## Problem diff --git a/.kiro/specs/pre-push-safety-net/design.md b/.kiro/specs/pre-push-safety-net/design.md index e44905f..8c911f5 100644 --- a/.kiro/specs/pre-push-safety-net/design.md +++ b/.kiro/specs/pre-push-safety-net/design.md @@ -132,8 +132,8 @@ Concrete workload roles and required paired changes are in `evaluation.md`. | Storage | Local content-addressed directories and JSON manifests | Atomic reusable artifacts, no database | | Presentation | Existing finding records plus compact hook renderer | Separate findings and coverage | -Align Docker, Compose and provisioning with the evaluated Joern version before any deployment -claim. The old defaults are 4.0.623. Pin alignment is predecessor packaging work, not completed here. +Docker, Compose and provisioning defaults were aligned to 4.0.625 in task 1.1, with a verified +local PHP/JavaScript image smoke. Host deployment and hook performance remain separate claims. ## File Structure Plan @@ -141,6 +141,10 @@ New files (proposed, not created by this documentation change): | Path | Responsibility | |---|---| +| `src/openultrasast/contracts.py` | Strict immutable manifest serialization shared by engine and adapters | +| `src/openultrasast/model/contracts.py` | Generic deadline, change context and exact selected/deferred scope contracts | +| `src/openultrasast/cpg/artifact.py` | Backend-owned graph identity, census and live lease protocol | +| `src/openultrasast/push/contracts.py` | Immutable ref/comparison identities and comparison-owned completion/status records | | `src/openultrasast/push/snapshot.py` | Parse ref updates, resolve bases, materialize blobs, produce change context | | `src/openultrasast/push/cache.py` | Artifact keys, validity, atomic publication and eviction | | `src/openultrasast/push/policy.py` | Delta identity, actionability admission and push disposition | @@ -149,6 +153,16 @@ New files (proposed, not created by this documentation change): | `benchmarks/push/README.md` and `benchmarks/push/measure.py` | Labeled push recipes and joint quality/latency measurement | | `tests/test_push_snapshot.py`, `tests/test_push_cache.py`, `tests/test_push_policy.py`, `tests/test_push_runner.py` | Contract and integration checks below | +Contract configuration (task 1.4) uses `[push]`: optional `comparison_base`, positive +`deadline_seconds` (30), positive `cancellation_allowance_seconds` (2), positive integer +`cache_max_bytes` (2 GiB), `mode` (`advisory` by default, or `blocking`) and independent +`incomplete_coverage_policy` (`allow` by default, or `block`). Unknown keys, malformed +sections and coerced/nonfinite numeric values are rejected. These settings are inert until +runner integration; they do not enable a hook or admit experimental capabilities. The cache +limit is a local storage budget, not a latency claim. Completion records are owned by an +exact comparison so reused head graphs cannot merge evidence from different bases. Live +leases have no JSON constructor; only the backend can validate and issue them in task 6.1. + Modified files: `cli.py` and `config.py` add explicit push command/configuration; `model/regions.py` carries change context without Git-specific logic; `model/scan.py` consumes the ranker's selected questions and returns exact selected/deferred IDs, deadline and reusable-graph diff --git a/.kiro/specs/pre-push-safety-net/implementation-group-1.md b/.kiro/specs/pre-push-safety-net/implementation-group-1.md new file mode 100644 index 0000000..708fc7e --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/implementation-group-1.md @@ -0,0 +1,60 @@ +# Group 1 implementation verification + +Date: 2026-09-12. Scope: tasks 1.1–1.4 only. All four tasks received independent +review with an APPROVED verdict and fresh task-local verification. Requirements, +design and tasks remain approved; implementation remains in progress (4/27 subtasks). + +## Verification result + +- STATUS: VERIFIED +- CLAIM_TYPE: TASK +- CLAIM: Group 1 establishes the reproducible runtime, evaluation inputs, honest + scoring and shared push/engine contracts. +- EVIDENCE: Final independent review ran the full suite: 965 passed, 9 skipped. + Focused contracts/configuration tests: 53 passed. Ruff and mypy passed for the + seven touched contract/configuration modules. `git diff --check` passed. +- GAPS: This is not feature-level GO. The pre-push runtime, persistent cache and + capability admission are not implemented. No end-to-end latency or precision + claim follows from these tests. + +## Task evidence + +| Task | Result and retained evidence | +| --- | --- | +| 1.1 | Joern 4.0.625 packaged and exercised with native PHP and JavaScript. Actual source bytes, graph census and query witnesses were verified; unreadable input failed explicitly. See `benchmarks/measurements/2026-09-12-engine-runtime-smoke.json`. Suite at review: 888 passed, 9 skipped. Commit `56cfc9d`. | +| 1.2 | Pinned PMPro and NodeGoat vulnerable, repaired and benign recipes; explicit authored-repair provenance. Offline validation read 506,502 bytes: 8 snapshots ready, 1 missing; 7 cases ready, 4 missing. Missing prerequisites cause exit 1. See `benchmarks/push/input-validation.json`. Suite: 902 passed, 9 skipped. Commit `8acada0`. | +| 1.3 | Frozen evaluation identity and uncertainty-preserving scorer. VAmPI retains all three known defects; unmatched SQL injection is not silently called transitively covered. Empty runs remain unmeasured. See `benchmarks/ranking/baselines/2026-09-12-vampi-position-v2.json` and `benchmarks/push/diagnostic-profile-v1.json`. Suite: 925 passed, 9 skipped. Commit `051f3b0`. | +| 1.4 | Strict immutable configuration, budget, question, scope, graph artifact and comparison-owned result contracts. Incomplete/unexecuted work cannot become complete through serialization. Existing scan configuration remains compatible; runtime enforcement is pending. Final suite: 965 passed, 9 skipped. | + +## Packaged runtime and limitations + +After the final source changes, `docker compose build ousast` exited 0. Final local +image: `sha256:92d311097075a2092a2e362b5470d706734f9de35de7820c3de5eb1ad91b17d5`. +A network-disabled container imported the new contracts, read an 83-byte TOML +configuration, round-tripped its push settings, rejected a NaN deadline, and ran +`python -m openultrasast.cli --help` successfully. Runtime: UID 1000, Python 3.12.3. +The earlier graph smoke records its own image identity; the backend did not change +after that smoke. The final packaging check does not repeat or replace graph evidence. + +Full-tree `.venv/bin/mypy src/openultrasast` still exits 1 with three pre-existing +errors in `model/scan.py` at lines 299, 302 and 310 (candidate redefinition and +incompatible candidate types). It checked 107 source files with no additional errors. +Skipped tests and the unavailable evaluation prerequisites remain outside demonstrated +coverage. Ghost is a reserved workload, not a scanned or admitted positive population; +independent paired/holdout evidence remains required per capability. + +## Design direction and next work + +The ranker remains the scope mechanism. Vendor exclusion applies to graph inputs +as well as targets. PMPro guides core development and JavaScript/Express tests +transfer; Python regressions and C-family abstraction limits remain explicit. + +Persist validated immutable graphs first. Separate graph, query/evidence and ranked +result identities so ranking-only changes can reuse compatible graphs and evidence. +Differential graph mutation is a future experiment requiring equivalence to cold +rebuilds, including deletions, renames, dependent edges and overlays. No stable +Joern node-ID assumption or partial-file update is accepted as proof of correctness. + +Next: group 2 immutable push snapshots and comparison resolution, then the first +complete PMPro/Node experiment at task 5.2. Persistent caching remains group 6; +normal advisory capability admission remains gated by task 8.3. diff --git a/.kiro/specs/pre-push-safety-net/research.md b/.kiro/specs/pre-push-safety-net/research.md index cd66c91..e8af0ee 100644 --- a/.kiro/specs/pre-push-safety-net/research.md +++ b/.kiro/specs/pre-push-safety-net/research.md @@ -6,7 +6,9 @@ - User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case. - Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism; third-party vendor code must remain outside targets and graphs. -- Requirements and design are proposals, not approved or implemented. No hook latency claim is made. +- Requirements, design and tasks are now approved, and group 1 implementation is verified complete; see + `tasks.md` for verified completion. The discovery log below preserves the earlier drafting state + and measurements. No hook latency claim is made. ## Research Log diff --git a/.kiro/specs/pre-push-safety-net/spec.json b/.kiro/specs/pre-push-safety-net/spec.json index c074c08..38b4aa4 100644 --- a/.kiro/specs/pre-push-safety-net/spec.json +++ b/.kiro/specs/pre-push-safety-net/spec.json @@ -1,7 +1,7 @@ { "feature_name": "pre-push-safety-net", "created_at": "2026-09-12T15:35:00Z", - "updated_at": "2026-09-12T16:20:40.814596Z", + "updated_at": "2026-09-12T17:21:05.783007Z", "language": "en", "phase": "implementation", "approvals": { @@ -50,5 +50,16 @@ "executable_subtasks": 27, "requirements_covered": 35, "record": "task-review.md" + }, + "implementation_progress": { + "completed_tasks": [ + "1.1", + "1.2", + "1.3", + "1.4" + ], + "remaining_tasks": 23, + "verification_record": "implementation-group-1.md", + "next_group": 2 } } diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md index 3d93336..0f34ac6 100644 --- a/.kiro/specs/pre-push-safety-net/tasks.md +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -1,7 +1,8 @@ # Implementation Plan: pre-push-safety-net -Status: task plan approved by the maintainer's "approve." on 2026-09-12. Implementation has -started with group 1 (tasks 1.1–1.4); each task remains pending until independent review and fresh verification pass. +Status: task plan approved by the maintainer's "approve." on 2026-09-12. Group 1 (tasks 1.1–1.4) +is complete after independent review and fresh verification. Remaining groups are pending; +the hook and capability admission are not yet implemented. See `implementation-group-1.md`. ## Execution contract @@ -21,7 +22,7 @@ started with group 1 (tasks 1.1–1.4); each task remains pending until independ - Sequential order is the default. Only 3.1 may run alongside group 2 after its explicit prerequisite; shared backend/driver, cache and report edits otherwise remain serial. No optional acceptance tests. -- [ ] 1. Establish the runtime, evaluation inputs and shared contracts +- [x] 1. Establish the runtime, evaluation inputs and shared contracts - [x] 1.1 Reproduce the evaluated engine environment - Align the shipped image, Compose and host-install defaults with Joern 4.0.625; preserve version override and checksum verification. - Exercise PHP with the native interpreter and the installed JavaScript frontend; open a source and the frontend parser where applicable, recording byte counts and graph census. @@ -45,13 +46,14 @@ started with group 1 (tasks 1.1–1.4); each task remains pending until independ - _Boundary: Evaluation harness, repository benchmark integration_ - _Requirements: 6.2, 6.3, 6.4, 6.5, 7.3_ - _Verified 2026-09-12:_ independent review APPROVED; 925 tests passed, 9 skipped. Parent reproduced the frozen profile and actual VAmPI static rerun (15,942 pinned bytes, all 3 targets retained, SQLI unmatched, no transitive/detection claim, exit 1). Empty-run scoring retains 11 unresolved cases and explicitly reports unmeasured/0 recorded cases. Historical baselines remain unchanged with a caveat; new scorer consumes associated execution/review evidence and does not implement replay or admission. -- [ ] 1.4 Establish typed push and engine integration contracts +- [x] 1.4 Establish typed push and engine integration contracts - Add importable push components and validated configuration for comparison base, shared deadline/cancellation allowance, local cache limit, advisory/blocking and separate coverage policy. - Define ownership of update/comparison, change context, selected/deferred scope, graph identity/lease and separate finding/coverage/push statuses as described in design. - Keep generic engine contracts outside the push package; leave ordinary scan defaults compatible and capabilities experimental until evaluated. - Done when contract/config round trips preserve all identities/statuses, invalid values are rejected, and the engine has no dependency on the push package. - _Boundary: Push runner, Cache and Engine Boundary, CLI/config integration_ - _Requirements: 4.1, 4.3, 5.3, 5.4, 5.5, 7.1, 7.3_ + - _Verified 2026-09-12:_ independent review APPROVED; 965 tests passed, 9 skipped; touched-source Ruff/mypy passed. Parent rebuilt the local image and verified non-root, network-disabled CLI startup, all contract imports, an 83-byte configuration round trip and invalid-budget rejection. Comparison-owned completion preserves different bases for a shared head. Contracts are structural; runtime snapshot/cache/runner/admission implementations remain in later tasks. - [ ] 2. Resolve immutable pushed content - [ ] 2.1 Resolve every supplied ref update and comparison diff --git a/.kiro/steering/overview.md b/.kiro/steering/overview.md index 7ff6d49..b003955 100644 --- a/.kiro/steering/overview.md +++ b/.kiro/steering/overview.md @@ -1,17 +1,23 @@ # Overview — what this is, and what is actually true of it **Current direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated -development across supported ecosystems. See `safety-net.md` and the proposed `pre-push-safety-net` +development across supported ecosystems. See `safety-net.md` and the approved `pre-push-safety-net` spec. The ranker remains the scope mechanism; third-party vendor code stays outside targets and graphs. Advisory must meet the same actionability bar as blocking. **Historical scoreboard below:** it was written September 9. Later commits record a completing whole-PMPro scan and Joern 4.0.625 ranking measurements (82/91/54 on the three plugins), while -default installation pins remain 4.0.623. The pre-push workflow, changed-code latency and actionable +default installation pins are now aligned to 4.0.625 with verified PHP/JavaScript image smoke. +The pre-push workflow, changed-code latency and actionable precision are not demonstrated by those results. In particular, the manually selected two-file slice below is a useful experiment, not evidence that an automatic push scope would select the same context. Current scope/status corrections live in `flow-aware-ranking/brief.md` and the roadmap. +The new scoring instrument retains missing targets and unanswered queries. Historical rank positions +do not establish detection recall; VAmPI's old budget of 16 omitted its unmatched SQLI target. The +corrected positional run keeps that target unresolved. Labeled PMPro/Node inputs and a frozen +diagnostic profile are committed; independent capability evaluation is still required. + **Written 2026-09-09.** Fourteen specs have accumulated and the roadmap still described a pre-Joern world, so this is the synthesis above them: what the system does, what has been measured, and what is not true yet. Every number here comes from a committed artifact in `benchmarks/measurements/`. diff --git a/.kiro/steering/roadmap.md b/.kiro/steering/roadmap.md index 8ff3eaa..3a14545 100644 --- a/.kiro/steering/roadmap.md +++ b/.kiro/steering/roadmap.md @@ -21,8 +21,9 @@ engine are out of scope. - Whole-PMPro analysis has completed with its known CVE; the earlier "does not finish" diagnosis is superseded. The recorded 72-minute scan is not a pre-push performance result. -- The latest Joern 4.0.625 + layout ranking budgets are PMPro 82, WP Statistics 91, MW WP Form 54, - VAmPI 16. `<50` remains unmet; VAmPI's transitive SQLI is not measured by this position metric. +- Historical Joern 4.0.625 + layout position budgets are PMPro 82, WP Statistics 91, MW WP Form 54. + `<50` remains unmet. VAmPI's old 16 omitted SQLI; the corrected instrument retains the unmatched + target and reports no full-target budget or detection recall without the required evidence. - Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller. - Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence. @@ -45,14 +46,14 @@ engine are out of scope. - [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact reuse, total deadline, change attribution, actionability admission and compact hook behavior. Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated - (8 groups, 27 executable subtasks), independently reviewed and approved; group 1 implementation started. + (8 groups, 27 executable subtasks), independently reviewed and approved; group 1 is complete (4/27 tasks). ### Next concrete step -The maintainer approved `pre-push-safety-net/tasks.md` and starting group 1. Requirements and design -were accepted when the maintainer requested the next Kiro phase; the 27-subtask plan passed independent -sanity review. Complete group 1 (reproducible runtime, readable PMPro/Node inputs, honest scoring -and shared contracts). The first complete experiment is 5.2, before persistent caching and hook +Group 1 has passed independent review: reproducible PHP/JavaScript runtime, pinned PMPro/Node inputs, +honest scoring and shared contracts. Continue with group 2's immutable push snapshots under the +approved task plan; see `pre-push-safety-net/implementation-group-1.md` for evidence and limits. +The first complete experiment is 5.2, before persistent caching and hook installation work. Keep PMPro as the primary core-optimization example, then freeze the core and prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through the same ranker, with a real deadline and vendor-free graph. Include benign pushes and retain Python diff --git a/.kiro/steering/safety-net.md b/.kiro/steering/safety-net.md index 8778520..5c1fe57 100644 --- a/.kiro/steering/safety-net.md +++ b/.kiro/steering/safety-net.md @@ -78,5 +78,6 @@ updates as a shortcut to an unmeasured latency promise. The product direction above is user-requested. The amended requirements, design and initial evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12 -("ok, next kiro phase"). Task approval remains pending. This is not authorization to implement -or a claim that hook performance or capability admission has been demonstrated. +("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1 +("approve."). Implementation is authorized under that plan; task completion still requires +independent review and fresh evidence. Hook performance and capability admission remain unproven. diff --git a/benchmarks/measurements/2026-09-08-module-audit.json b/benchmarks/measurements/2026-09-08-module-audit.json index c8ac2b0..09d4924 100644 --- a/benchmarks/measurements/2026-09-08-module-audit.json +++ b/benchmarks/measurements/2026-09-08-module-audit.json @@ -1,10 +1,10 @@ { "counts": { - "load_bearing": 92, + "load_bearing": 97, "standalone_capability": 9 }, - "modules": 101, - "note": "Inventory refreshed for pre-push-safety-net task 1.3 standalone scorer; original artifact filename retained for audit gate.", + "modules": 106, + "note": "Refreshed 2026-09-12 after pre-push-safety-net task 1.4: immutable push and generic engine contracts are spec-owned; runtime integration remains pending.", "orphaned": [], "rows": [ { @@ -110,6 +110,17 @@ "module": "config", "reason": "reachable from gate, map_gate, pair_gate or the CLI" }, + { + "classification": "load_bearing", + "importers": [ + "config", + "cpg.artifact", + "model.contracts", + "push.contracts" + ], + "module": "contracts", + "reason": "pre-push-safety-net Req 7.3: strict shared manifest serialization, independent of push" + }, { "classification": "load_bearing", "importers": [ @@ -125,6 +136,12 @@ "module": "cpg", "reason": "model-grounded-detection Req 4: the CPG seam package" }, + { + "classification": "load_bearing", + "importers": [], + "module": "cpg.artifact", + "reason": "pre-push-safety-net Req 4.3/7.3: backend graph identity and live lease contract; validation integration pending" + }, { "classification": "load_bearing", "importers": [ @@ -333,7 +350,8 @@ "model.scan", "model.specs", "model.taint", - "preprocess" + "preprocess", + "push.contracts" ], "module": "model", "reason": "reachable from gate, map_gate, pair_gate or the CLI" @@ -368,6 +386,14 @@ "module": "model.config_value", "reason": "model-grounded-detection Req 6.3: constant abstraction for the configuration families" }, + { + "classification": "load_bearing", + "importers": [ + "push.contracts" + ], + "module": "model.contracts", + "reason": "pre-push-safety-net Req 4.1/7.3: generic deadline, change and exact scope contracts; integration pending" + }, { "classification": "load_bearing", "importers": [ @@ -605,6 +631,18 @@ "module": "provider.openrouter", "reason": "reachable from gate, map_gate, pair_gate or the CLI" }, + { + "classification": "load_bearing", + "importers": [], + "module": "push", + "reason": "pre-push-safety-net Req 5.5/7.3: experimental push contract package; no hook installed" + }, + { + "classification": "load_bearing", + "importers": [], + "module": "push.contracts", + "reason": "pre-push-safety-net Req 5.5/7.3: immutable comparisons and independent result statuses; runner pending" + }, { "classification": "standalone_capability", "importers": [], diff --git a/src/openultrasast/config.py b/src/openultrasast/config.py index 805970d..220fd02 100644 --- a/src/openultrasast/config.py +++ b/src/openultrasast/config.py @@ -5,6 +5,9 @@ import os import tomllib from dataclasses import asdict, dataclass from pathlib import Path +from typing import Literal + +from openultrasast.contracts import Contract DEFAULT_VECTOR_STORE = "json-local" @@ -163,6 +166,28 @@ class ObligationsConfig: policy_path: str = ".openultrasast/obligations.toml" +@dataclass(frozen=True) +class PushConfig(Contract): + """Inert experimental push settings; configuring them does not install a hook. + + The 2 GiB local cache cap is a storage budget, not a performance promise. The + eventual cache evicts unleased entries; deadline and cleanup are separate costs. + No endpoint, fetch, engine download or capability admission is enabled here. + """ + + comparison_base: str | None = None + deadline_seconds: float = 30.0 + cancellation_allowance_seconds: float = 2.0 + cache_max_bytes: int = 2 * 1024 ** 3 + mode: Literal["advisory", "blocking"] = "advisory" + incomplete_coverage_policy: Literal["allow", "block"] = "allow" + + def __post_init__(self) -> None: + super().__post_init__() + if self.deadline_seconds <= 0 or self.cancellation_allowance_seconds <= 0 or self.cache_max_bytes <= 0: + raise ValueError("push deadlines, cancellation allowance and cache limit must be positive") + + @dataclass(frozen=True) class ResolvedConfig: models: ModelConfig = ModelConfig() @@ -181,6 +206,7 @@ class ResolvedConfig: variants: VariantsConfig = VariantsConfig() obligations: ObligationsConfig = ObligationsConfig() model: ModelLayerConfig = ModelLayerConfig() + push: PushConfig = PushConfig() runs_dir: str = ".openultrasast/runs" @@ -208,13 +234,19 @@ def load_config(config_path: Path | None = None) -> ResolvedConfig: variants=_load_variants(data.get("variants", {})), obligations=_load_obligations(data.get("obligations", {})), model=_load_model(data.get("model", {})), + push=PushConfig.from_payload(data.get("push", {})), runs_dir=os.environ.get("OPENULTRASAST_RUNS_DIR", ".openultrasast/runs"), ) +def config_payload(config: ResolvedConfig) -> dict[str, object]: + """Reproducible resolved settings, including the separate experimental push policy.""" + return asdict(config) + + def write_resolved_config(config: ResolvedConfig, path: Path) -> None: path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(json.dumps(asdict(config), indent=2, sort_keys=True) + "\n") + path.write_text(json.dumps(config_payload(config), indent=2, sort_keys=True) + "\n") def _section(value: object) -> dict[str, object]: diff --git a/src/openultrasast/contracts.py b/src/openultrasast/contracts.py new file mode 100644 index 0000000..6f17501 --- /dev/null +++ b/src/openultrasast/contracts.py @@ -0,0 +1,92 @@ +"""Strict immutable manifest values shared by engine and adapters (pre-push Req 7.3). + +These schemas validate structure, not source readability, Git object existence or alert +eligibility. Live resources deliberately have no manifest decoder. +""" +from __future__ import annotations + +import math +from dataclasses import fields +from types import UnionType +from typing import Any, Literal, Self, get_args, get_origin, get_type_hints + +_HINTS: dict[type, dict[str, Any]] = {} + + +def _hints(cls: type) -> dict[str, Any]: + if cls not in _HINTS: + _HINTS[cls] = get_type_hints(cls) + return _HINTS[cls] + + +def _value(value: object, kind: Any, *, decode: bool) -> object: + origin, args = get_origin(kind), get_args(kind) + if origin is UnionType: + for candidate in args: + try: + return _value(value, candidate, decode=decode) + except ValueError: + pass + raise ValueError(f"invalid union value: {value!r}") + if origin is Literal: + if any(type(value) is type(option) and value == option for option in args): + return value + elif origin is tuple: + if type(value) is tuple or (decode and type(value) is list): + items = tuple(value) # type: ignore[arg-type] + if len(args) == 2 and args[1] is Ellipsis: + return tuple(_value(item, args[0], decode=decode) for item in items) + if len(items) == len(args): + return tuple(_value(item, item_type, decode=decode) for item, item_type in zip(items, args, strict=True)) + elif isinstance(kind, type) and issubclass(kind, Contract): + if decode: + return kind.from_payload(value) + if isinstance(value, kind): + return value + elif kind is float: + if type(value) in (int, float): + try: + if math.isfinite(value): # type: ignore[arg-type] + return value + except OverflowError: + pass + elif kind is str: + if type(value) is str and value.strip(): + return value + elif type(value) is kind: + return value + raise ValueError(f"invalid {kind}: {value!r}") + + +def _payload(value: object) -> object: + if isinstance(value, Contract): + return value.to_payload() + if isinstance(value, tuple): + return [_payload(item) for item in value] + return value + + +class Contract: + """Base for frozen dataclasses with strict construction and JSON round trips.""" + + def __post_init__(self) -> None: + for name, kind in _hints(type(self)).items(): + try: + _value(getattr(self, name), kind, decode=False) + except ValueError as error: + raise ValueError(f"{type(self).__name__}.{name}: {error}") from error + + def to_payload(self) -> dict[str, object]: + return {item.name: _payload(getattr(self, item.name)) for item in fields(self)} # type: ignore[arg-type] + + @classmethod + def from_payload(cls, payload: object) -> Self: + if not isinstance(payload, dict) or not all(isinstance(key, str) for key in payload): + raise ValueError(f"{cls.__name__} requires an object") + hints = _hints(cls) + if unknown := payload.keys() - hints.keys(): + raise ValueError(f"unknown {cls.__name__} fields: {sorted(unknown)}") + try: + return cls(**{name: _value(value, hints[name], decode=True) for name, value in payload.items()}) + except TypeError as error: + raise ValueError(f"invalid {cls.__name__}: {error}") from error diff --git a/src/openultrasast/cpg/artifact.py b/src/openultrasast/cpg/artifact.py new file mode 100644 index 0000000..1a6d3d9 --- /dev/null +++ b/src/openultrasast/cpg/artifact.py @@ -0,0 +1,82 @@ +"""Backend-owned graph provenance and live resource boundary (pre-push Req 4.3, 7.3). + +Structural validity is necessary but insufficient for reuse. Task 6.1 will validate +actual source bytes, graph digest/census and engine compatibility before issuing leases. +""" +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import Literal, Protocol + +from openultrasast.contracts import Contract + + +@dataclass(frozen=True) +class GraphIdentity(Contract): + source_digest: str + declarations_digest: str + exclusions_digest: str + unit: str + language: str + engine_version: str + frontend: str + frontend_version: str + overlay_semantics: tuple[str, ...] + overlay_options: tuple[tuple[str, str], ...] + frontend_options: tuple[tuple[str, str], ...] = () + + def __post_init__(self) -> None: + super().__post_init__() + for options in (self.overlay_options, self.frontend_options): + if len({key for key, _ in options}) != len(options): + raise ValueError("graph option names must be unique") + + +@dataclass(frozen=True) +class GraphCensus(Contract): + files: int + methods: int + calls: int + + def __post_init__(self) -> None: + super().__post_init__() + if min(self.files, self.methods, self.calls) < 0: + raise ValueError("graph census counts must be nonnegative") + + +@dataclass(frozen=True) +class GraphArtifact(Contract): + identity: GraphIdentity + graph_digest: str + source_bytes: int + census: GraphCensus + completeness: Literal["complete", "partial"] + diagnostics: tuple[str, ...] + + def __post_init__(self) -> None: + super().__post_init__() + if self.source_bytes < 0: + raise ValueError("source byte count must be nonnegative") + if self.completeness == "complete" and (not self.source_bytes or not self.census.files or self.diagnostics): + raise ValueError("complete graph requires readable source, a file census and no incomplete diagnostics") + if self.completeness == "partial" and not self.diagnostics: + raise ValueError("partial graph requires a recorded reason") + + +class GraphLease(Protocol): + """A backend-issued live lease; never reconstructed from a JSON manifest. + + The backend validates the artifact before issuing the lease and isolates loading + when it may mutate graph bytes. The consumer owns releasing it exactly once; the + backend's release is idempotent and bounded by the supplied monotonic deadline. + A cached artifact survives release. Implementations belong to task 6.1. + """ + + @property + def artifact(self) -> GraphArtifact: ... + + @property + def path(self) -> Path: ... + + def close(self, *, deadline_monotonic: float) -> None: ... diff --git a/src/openultrasast/model/audit.py b/src/openultrasast/model/audit.py index 46ed0df..b569e3e 100644 --- a/src/openultrasast/model/audit.py +++ b/src/openultrasast/model/audit.py @@ -28,6 +28,11 @@ ROOTS = ("gate", "map_gate", "pair_gate", "cli", "__main__") # Modules that stand alone: their own CLI subcommand or external protocol, deliberately off the scan path. SPEC_OWNED = { + "contracts": "pre-push-safety-net Req 7.3: strict shared manifest serialization, independent of push", + "model.contracts": "pre-push-safety-net Req 4.1/7.3: generic deadline, change and exact scope contracts; integration pending", + "cpg.artifact": "pre-push-safety-net Req 4.3/7.3: backend graph identity and live lease contract; validation integration pending", + "push": "pre-push-safety-net Req 5.5/7.3: experimental push contract package; no hook installed", + "push.contracts": "pre-push-safety-net Req 5.5/7.3: immutable comparisons and independent result statuses; runner pending", "model.specs": "model-grounded-detection Req 7.2-7.4: the security vocabularies the CPG queries are parameterised by", "model.endpoint": "model-grounded-detection Req 8: the judge client, wired by model/judge.py in group 3", "model.audit": "model-grounded-detection Req 3: this classifier; run by the maintainer, not by the scan", diff --git a/src/openultrasast/model/contracts.py b/src/openultrasast/model/contracts.py new file mode 100644 index 0000000..7eb0077 --- /dev/null +++ b/src/openultrasast/model/contracts.py @@ -0,0 +1,115 @@ +"""Language-agnostic change evidence and exact ranker scope (pre-push Req 7.3). + +These are integration contracts; constructing them neither selects nor executes work. +Revision identities are opaque to the engine. The snapshot adapter resolves Git objects. +""" +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from typing import Literal + +from openultrasast.contracts import Contract + + +@dataclass(frozen=True) +class ExecutionBudget(Contract): + """One absolute monotonic deadline, with a distinct bounded cleanup allowance. + + Process-local: do not carry the absolute clock value across process restarts. + Manifests should record elapsed stage costs and configured durations instead. + """ + + deadline_monotonic: float + cancellation_allowance_seconds: float + + def __post_init__(self) -> None: + super().__post_init__() + if self.deadline_monotonic < 0 or self.cancellation_allowance_seconds <= 0: + raise ValueError("deadline must be nonnegative and cancellation allowance positive") + + +@dataclass(frozen=True) +class QuestionIdentity(Contract): + unit: str + language: str + path: str + function: str | None + family: str + + @property + def question_id(self) -> str: + encoded = json.dumps(self.to_payload(), sort_keys=True, separators=(",", ":")).encode() + return hashlib.sha256(encoded).hexdigest() + + +@dataclass(frozen=True) +class ChangedSpan(Contract): + path: str + start_line: int + end_line: int + side: Literal["base", "head"] + + def __post_init__(self) -> None: + super().__post_init__() + if self.start_line < 1 or self.end_line < self.start_line: + raise ValueError("changed spans require positive inclusive ordered lines") + + +@dataclass(frozen=True) +class PathRename(Contract): + base_path: str + head_path: str + + +@dataclass(frozen=True) +class AffectedRelationship(Contract): + source: QuestionIdentity + target: QuestionIdentity + kind: str + evidence: tuple[str, ...] + + +@dataclass(frozen=True) +class ChangeContext(Contract): + base_revision: str | None + head_revision: str + changed_paths: tuple[str, ...] + deleted_paths: tuple[str, ...] + spans: tuple[ChangedSpan, ...] + renames: tuple[PathRename, ...] + declaration_paths: tuple[str, ...] + relationships: tuple[AffectedRelationship, ...] + unresolved_boundaries: tuple[str, ...] + + +@dataclass(frozen=True) +class RankedQuestion(Contract): + identity: QuestionIdentity + priority: float + evidence: tuple[str, ...] + + +@dataclass(frozen=True) +class DeferredQuestion(Contract): + identity: QuestionIdentity + reason: str + evidence: tuple[str, ...] + + +@dataclass(frozen=True) +class ScopeDecision(Contract): + ranker_version: str + ranking_mode: str + # Explicit census assertion: an empty list by itself is never proof of no work. + population_complete: bool + selected: tuple[RankedQuestion, ...] + deferred: tuple[DeferredQuestion, ...] + unresolved_boundaries: tuple[str, ...] + + def __post_init__(self) -> None: + super().__post_init__() + identities = [item.identity for item in self.selected] + [item.identity for item in self.deferred] + if len(set(identities)) != len(identities): + raise ValueError("question identities must be unique across selected and deferred scope") diff --git a/src/openultrasast/push/__init__.py b/src/openultrasast/push/__init__.py new file mode 100644 index 0000000..b5d6c1f --- /dev/null +++ b/src/openultrasast/push/__init__.py @@ -0,0 +1 @@ +"""Experimental pre-push contracts; no hook or admitted capability is enabled here.""" diff --git a/src/openultrasast/push/contracts.py b/src/openultrasast/push/contracts.py new file mode 100644 index 0000000..841757c --- /dev/null +++ b/src/openultrasast/push/contracts.py @@ -0,0 +1,86 @@ +"""Push-owned immutable identities and independent result statuses (Req 5.5, 7.3). + +Git resolution, admission policy and execution are separate later implementations. +No default result status can turn an unexecuted contract into a complete negative. +""" +from __future__ import annotations + +from dataclasses import dataclass +from typing import Literal + +from openultrasast.contracts import Contract +from openultrasast.model.contracts import QuestionIdentity, ScopeDecision + + +@dataclass(frozen=True) +class PushUpdate(Contract): + local_ref: str + local_oid: str + remote_ref: str + remote_oid: str + + +@dataclass(frozen=True) +class PushComparison(Contract): + head_oid: str + base_oid: str | None + base_reason: str + refs: tuple[str, ...] + + def __post_init__(self) -> None: + super().__post_init__() + if not self.refs or len(set(self.refs)) != len(self.refs): + raise ValueError("comparison must retain distinct associated refs") + + +@dataclass(frozen=True) +class ComparisonAnalysis(Contract): + """Scope and completion belong to this exact base/head comparison. + + A question completed for one comparison never completes it for another base, + even when the target graph is shared. This record does not assert admission. + """ + + comparison: PushComparison + scopes: tuple[ScopeDecision, ...] + completed_questions: tuple[QuestionIdentity, ...] + coverage_status: Literal["complete_within_scope", "incomplete", "unavailable"] + + def __post_init__(self) -> None: + super().__post_init__() + selected_list = [item.identity for scope in self.scopes for item in scope.selected] + selected = set(selected_list) + completed = set(self.completed_questions) + if len(selected) != len(selected_list): + raise ValueError("a comparison cannot select the same question across multiple scopes") + if len(completed) != len(self.completed_questions) or not completed <= selected: + raise ValueError("completed questions must be unique members of this comparison's selected scope") + if self.coverage_status == "complete_within_scope": + if self.comparison.base_oid is None or not self.scopes: + raise ValueError("complete coverage requires compared revisions and an explicit scope census") + if completed != selected or any( + not scope.population_complete or scope.deferred or scope.unresolved_boundaries for scope in self.scopes + ): + raise ValueError("deferred, unexecuted or unresolved work cannot be complete coverage") + + +@dataclass(frozen=True) +class PushResult(Contract): + analyses: tuple[ComparisonAnalysis, ...] + finding_status: Literal["actionable", "none"] + coverage_status: Literal["complete_within_scope", "incomplete", "unavailable", "not_applicable"] + push_disposition: Literal["allow", "block"] + actionable_defect_ids: tuple[str, ...] + + def __post_init__(self) -> None: + super().__post_init__() + if (self.finding_status == "actionable") != bool(self.actionable_defect_ids): + raise ValueError("finding status must match admitted defect identities") + if len(set(self.actionable_defect_ids)) != len(self.actionable_defect_ids): + raise ValueError("admitted defect identities must be unique") + if self.coverage_status == "complete_within_scope" and ( + not self.analyses or any(item.coverage_status != "complete_within_scope" for item in self.analyses) + ): + raise ValueError("aggregate complete coverage requires completed comparison analyses") + if self.coverage_status == "not_applicable" and (self.analyses or self.actionable_defect_ids): + raise ValueError("not applicable cannot carry analyzed scope or findings") diff --git a/tests/test_push_contracts.py b/tests/test_push_contracts.py new file mode 100644 index 0000000..81ef9aa --- /dev/null +++ b/tests/test_push_contracts.py @@ -0,0 +1,237 @@ +"""Approved task 1.4 contracts: persist identities without inventing completed analysis.""" +import ast +import json +from dataclasses import FrozenInstanceError, replace +from pathlib import Path + +import pytest + +from openultrasast.config import PushConfig, config_payload, load_config +from openultrasast.contracts import Contract +from openultrasast.cpg.artifact import GraphArtifact, GraphCensus, GraphIdentity, GraphLease +from openultrasast.model.contracts import ( + ChangeContext, + ChangedSpan, + DeferredQuestion, + QuestionIdentity, + RankedQuestion, + ScopeDecision, +) +from openultrasast.push.contracts import ComparisonAnalysis, PushComparison, PushResult, PushUpdate + + +def question(language: str = "javascript", path: str = "src/api.js") -> QuestionIdentity: + return QuestionIdentity("api", language, path, "handler", "injection") + + +def scope() -> ScopeDecision: + return ScopeDecision("core-v1", "evidence", True, (RankedQuestion(question(), 2.5, ("source→operation",)),), (), ()) + + +def comparison() -> PushComparison: + return PushComparison("a" * 64, "b" * 64, "supplied_remote_tip", ("refs/heads/main",)) + + +def test_config_round_trip_and_ordinary_scan_compatibility(tmp_path: Path) -> None: + baseline = load_config() + path = tmp_path / "settings.toml" + path.write_text('[push]\ncomparison_base = "refs/heads/main"\ndeadline_seconds = 11.5\n' + 'cancellation_allowance_seconds = 1\ncache_max_bytes = 4096\nmode = "blocking"\n' + 'incomplete_coverage_policy = "block"\n') + configured = load_config(path) + assert configured.push == PushConfig("refs/heads/main", 11.5, 1, 4096, "blocking", "block") + assert PushConfig.from_payload(json.loads(json.dumps(config_payload(configured)["push"]))) == configured.push + before, after = config_payload(baseline), config_payload(configured) + before.pop("push") + after.pop("push") + assert before == after + assert baseline.push == PushConfig() + assert baseline.push.deadline_seconds == 30 + assert baseline.push.cancellation_allowance_seconds == 2 + assert baseline.push.cache_max_bytes == 2 * 1024 ** 3 + assert baseline.push.mode == "advisory" + assert baseline.push.incomplete_coverage_policy == "allow" + + +@pytest.mark.parametrize("setting", [ + 'push = "bad"', 'push = []', '[push]\ndeadline_seconds = true', '[push]\ndeadline_seconds = "30"', + '[push]\ndeadline_seconds = nan', '[push]\ndeadline_seconds = inf', '[push]\ndeadline_seconds = 0', + '[push]\ncancellation_allowance_seconds = -1', '[push]\ncache_max_bytes = 0', + '[push]\ncache_max_bytes = 1.5', '[push]\ncache_max_bytes = true', '[push]\nmode = "nag"', + '[push]\nincomplete_coverage_policy = "ignore"', '[push]\ncomparison_base = ""', + '[push]\ncomparison_base = 1', '[push]\ndeadline_second = 60', +]) +def test_push_configuration_rejects_unsafe_coercion(tmp_path: Path, setting: str) -> None: + path = tmp_path / "settings.toml" + path.write_text(setting) + with pytest.raises(ValueError): + load_config(path) + + +def test_contract_round_trips_preserve_partition_ref_and_comparison_identity() -> None: + update = PushUpdate("refs/heads/topic", "a" * 64, "refs/heads/main", "b" * 64) + context = ChangeContext("b" * 64, "a" * 64, ("src/api.js",), ("src/guard.js",), + (ChangedSpan("src/guard.js", 3, 8, "base"),), (), ("package.json",), (), ("dynamic dispatch",)) + for value in (update, comparison(), context, scope()): + assert type(value).from_payload(json.loads(json.dumps(value.to_payload()))) == value + assert question() != question("php", "src/api.js") + assert question().question_id != question("php", "src/api.js").question_id + other_base = replace(comparison(), base_oid="c" * 64) + assert other_base != comparison() + with pytest.raises(FrozenInstanceError): + update.local_oid = "other" # type: ignore[misc] + + +@pytest.mark.parametrize("finding,coverage,disposition", [ + ("none", "incomplete", "allow"), ("none", "unavailable", "block"), + ("actionable", "incomplete", "allow"), ("actionable", "complete_within_scope", "block"), +]) +def test_result_statuses_are_independent(finding: str, coverage: str, disposition: str) -> None: + payload = { + "analyses": [ComparisonAnalysis(comparison(), (scope(),), (question(),), coverage).to_payload()], + "finding_status": finding, + "coverage_status": coverage, "push_disposition": disposition, + "actionable_defect_ids": ["defect-1"] if finding == "actionable" else [], + } + result = PushResult.from_payload(payload) + assert PushResult.from_payload(json.loads(json.dumps(result.to_payload()))) == result + assert result.finding_status == finding + assert result.coverage_status == coverage + assert result.push_disposition == disposition + + +def test_incomplete_scope_cannot_be_serialized_as_complete_negative() -> None: + result = ComparisonAnalysis(comparison(), (scope(),), (question(),), "complete_within_scope") + assert ComparisonAnalysis.from_payload(result.to_payload()) == result + deferred = DeferredQuestion(question("php", "index.php"), "deadline", ("unanswered",)) + for changed in (replace(scope(), deferred=(deferred,)), replace(scope(), unresolved_boundaries=("vendor call",)), + replace(scope(), population_complete=False)): + with pytest.raises(ValueError): + replace(result, scopes=(changed,)) + with pytest.raises(ValueError): + replace(result, completed_questions=()) + with pytest.raises(ValueError): + replace(result, scopes=()) + with pytest.raises(ValueError): + replace(result, comparison=replace(comparison(), base_oid=None, base_reason="missing")) + with pytest.raises(ValueError): + replace(scope(), deferred=(DeferredQuestion(question(), "deadline", ()),)) + + +@pytest.mark.parametrize("value", [ + {"local_ref": "x", "local_oid": "a", "remote_ref": "y", "remote_oid": "b", "extra": 1}, + {"local_ref": "x", "local_oid": 1, "remote_ref": "y", "remote_oid": "b"}, + {"local_ref": "x", "local_oid": "a", "remote_ref": "y"}, +]) +def test_malformed_identity_payloads_fail(value: dict[str, object]) -> None: + with pytest.raises(ValueError): + PushUpdate.from_payload(value) + + +def test_graph_provenance_round_trip_and_live_lease_separation() -> None: + identity = GraphIdentity("source-digest", "declarations", "exclusions", "api", "javascript", + "4.0.625", "jssrc2cpg", "4.0.625", ("dataflowOss-v1",), (("option", "value"),)) + artifact = GraphArtifact(identity, "graph-sha256", 128, GraphCensus(1, 2, 3), "complete", ()) + assert GraphArtifact.from_payload(json.loads(json.dumps(artifact.to_payload()))) == artifact + assert replace(identity, exclusions_digest="changed") != identity + assert not issubclass(GraphLease, Contract) # a manifest cannot restore a live resource + with pytest.raises(ValueError): + replace(artifact, source_bytes=0) + with pytest.raises(ValueError): + replace(artifact, diagnostics=("file unreadable",)) + with pytest.raises(ValueError): + replace(artifact, census=GraphCensus(0, 0, 0)) + with pytest.raises(ValueError): + replace(identity, overlay_options=(("x", "1"), ("x", "2"))) + + +def test_strict_runtime_construction_not_only_decoding() -> None: + with pytest.raises(ValueError): + RankedQuestion(question(), float("nan"), ()) + with pytest.raises(ValueError): + ChangedSpan("x", True, 3, "head") + with pytest.raises(ValueError): + ChangedSpan("x", 4, 3, "head") + with pytest.raises(ValueError): + PushConfig(deadline_seconds=True) + with pytest.raises(ValueError): + PushConfig.from_payload({"mode": "blocking", "incomplete_coverage_policy": []}) + + +def test_generic_contracts_and_engine_do_not_depend_on_push() -> None: + root = Path("src/openultrasast") + paths = [root / "contracts.py", *(root / "model").glob("*.py"), *(root / "cpg").glob("*.py")] + for path in paths: + for node in ast.walk(ast.parse(path.read_text())): + if isinstance(node, ast.ImportFrom): + assert "push" not in (node.module or "").split("."), path + elif isinstance(node, ast.Import): + assert all("push" not in alias.name.split(".") for alias in node.names), path + + +def test_shared_head_does_not_merge_different_base_completion() -> None: + completed = ComparisonAnalysis(comparison(), (scope(),), (question(),), "complete_within_scope") + pending = ComparisonAnalysis(replace(comparison(), base_oid="c" * 64), (scope(),), (), "incomplete") + result = PushResult((completed, pending), "none", "incomplete", "allow", ()) + restored = PushResult.from_payload(json.loads(json.dumps(result.to_payload()))) + assert restored == result + assert restored.analyses[1].completed_questions == () + with pytest.raises(ValueError): + replace(result, coverage_status="complete_within_scope") + with pytest.raises(ValueError): + replace(pending, coverage_status="complete_within_scope") + + +def test_deletion_only_result_is_explicitly_not_applicable() -> None: + result = PushResult((), "none", "not_applicable", "allow", ()) + assert PushResult.from_payload(result.to_payload()) == result + with pytest.raises(ValueError): + replace(result, coverage_status="complete_within_scope") + + +@pytest.mark.parametrize("field,value", [ + ("finding_status", "clean"), ("coverage_status", "secure"), ("push_disposition", "warn"), + ("finding_status", True), ("coverage_status", None), +]) +def test_result_rejects_unknown_status_vocabulary(field: str, value: object) -> None: + payload = PushResult((), "none", "not_applicable", "allow", ()).to_payload() + payload[field] = value + with pytest.raises(ValueError): + PushResult.from_payload(payload) + + +def test_graph_partial_provenance_stays_partial_and_options_are_preserved() -> None: + identity = GraphIdentity("sources", "decls", "excludes", "core", "c", "4.0.625", "c2cpg", "4.0.625", + ("dataflowOss-v1",), (("depth", "2"),), (("include", "src"),)) + artifact = GraphArtifact(identity, "digest", 0, GraphCensus(0, 0, 0), "partial", ("source unreadable",)) + restored = GraphArtifact.from_payload(json.loads(json.dumps(artifact.to_payload()))) + assert restored == artifact + assert restored.identity.frontend_options == (("include", "src"),) + with pytest.raises(ValueError): + replace(restored, completeness="complete") + + +def test_execution_budget_requires_finite_clock_and_distinct_positive_allowance() -> None: + from openultrasast.model.contracts import ExecutionBudget + + budget = ExecutionBudget(123.0, 2.0) + assert budget.deadline_monotonic == 123 + assert budget.cancellation_allowance_seconds == 2 + for deadline, allowance in ((float("inf"), 2), (1, float("nan")), (True, 2), (1, 0), (-1, 2)): + with pytest.raises(ValueError): + ExecutionBudget(deadline, allowance) + + +def test_empty_population_needs_an_explicit_census_assertion() -> None: + empty = ScopeDecision("core-v1", "evidence", True, (), (), ()) + analyzed = ComparisonAnalysis(comparison(), (empty,), (), "complete_within_scope") + assert analyzed.completed_questions == () + with pytest.raises(ValueError): + replace(analyzed, scopes=(replace(empty, population_complete=False),)) + + +def test_direct_mutable_collection_is_not_an_immutable_contract() -> None: + with pytest.raises(ValueError): + PushComparison("head", "base", "supplied", ["refs/heads/main"]) # type: ignore[arg-type] + with pytest.raises(ValueError): + PushResult.from_payload({"finding_status": "none"})