feat(pre-push-safety-net): replay immutable revisions through the full transaction

This commit is contained in:
Marius Ciepluch
2026-09-13 14:50:25 +02:00
parent 6b12b73411
commit 31e5ec7430
14 changed files with 4589 additions and 29 deletions
@@ -0,0 +1,66 @@
# Implementation verification: group 5
Group 5 was authorized by the maintainer's “approved” after group 4 completion.
Task 5.1 is verified complete (15/27 total); task 5.2 is next.
Real capability eligibility and rollout readiness remain unproven.
## Execution and review mode
Fresh implementer dispatch for 5.1 returned `agent thread limit reached`.
The [Kiro implementation skill](../../../.opencode/skills/kiro-impl/SKILL.md) explicitly
states: “If multi-agent is not available, fall back to manual mode execution for all tasks.”
Its manual review rule permits review in the main context when a fresh reviewer is unavailable.
This run therefore uses manual implementation and adversarial review, not independent review.
## Task 5.1 brief
Wire explicit immutable base/head replay through the existing mapper, ranker, driver,
base comparison and admission/report contracts. Preserve the shared elapsed deadline,
source/index/ref state, completed evidence on failure, and separate coverage/exit policy.
Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3. Design sections are named:
Snapshot Adapter, Ranker Scope Contract, Delta and Actionability Policy, Result and Hook
Adapter, Cache and Engine Boundary. No new detector, ranking weights, framework facts,
cache or hook installation belongs to this task.
### Integration findings
- Snapshot resolution previously used fresh per-command timeouts. An optional shared
execution budget now covers constructor probes and explicit revision resolution.
- Discovery and provenance are bounded in a Linux child process. The parent drains output
incrementally, checks the deadline, kills a stalled worker and requires a valid receipt;
exit zero with no result is failure. No project code or engine runs in this child.
- The first real controlled PHP replay found the exact request-to-eval witness but novelty
was unknown: comparison policy treated another question's context gap as global. The
owning task 4.1 policy was repaired here with a failing regression first. Only recognized
boundaries attached to a different recorded question can be ignored for that candidate;
aggregate coverage retains every gap. Own-question and global boundaries still prevent
supported novelty. This is policy ownership repair, not a new upstream graph abstraction.
- Artifacts must resolve outside the analyzed repository. The runner refuses a destination
that could overwrite live source/index/ref state and preserves the real enforcement result.
### Evidence in progress
Initial OFF-state tests: `/tmp/ousast-replay-red.log`, six failures before implementation.
Policy ownership regression: `/tmp/ousast-replay-policy-red.log`, one failure before repair.
Live-source overwrite regression: `/tmp/ousast-replay-review-red.log`, reproduced before repair.
Final validation: 1,236 tests passed, nine skipped; Ruff/format, mypy (112 source files), compileall and all three regression gates passed. Final packaged CLI boot and the controlled positive replay passed. See `benchmarks/measurements/2026-09-13-replay-smoke.json`.
Additional integration repairs retain snapshot omissions in engine change context and distinguish
base questions proven tier zero from unfinished queries. The exact required base question still
must have completed, valid evidence; budget-deferred work still prevents a complete comparison.
RED evidence: `/tmp/ousast-replay-tierzero-red.log` and `/tmp/ousast-replay-snapshot-red.log`.
## Task 5.1 review verdict
- VERDICT: APPROVED
- TASK: 5.1
- MECHANICAL_RESULTS: Full suite exit 0 (1,236 passed, 9 skipped); Ruff/format/mypy/compileall and detection/map/pair gates PASS; no new placeholder markers or secrets; boundary WITHIN; behavioral RED VERIFIED.
- FINDINGS: Integration defects above repaired with regressions; no outstanding task-blocking finding.
- SUMMARY: Manual adversarial review approves bounded local replay; this is not independent review or rollout admission.
## Task 5.1 verification
- STATUS: VERIFIED
- CLAIM_TYPE: TASK
- CLAIM: Explicit local replay traverses the production mapper/ranker/arbiter/comparison/report path and retains uncertainty and shared deadlines.
- EVIDENCE: Final packaged controlled security change yields new/source_connection_absent_from_comparable_base. Fixed twin emits no modeled head finding. The packaged deadline control returns incomplete in 30.785 seconds, within 30 + 2 seconds. Exact final package policy/core/facts/query digests match the workspace. Complete commands and captured raw answers live in the measurement artifact.
- GAPS: Aggregate coverage remains incomplete, real capabilities are disabled, and lab timings do not satisfy the latency gate. Source-mounted fixed and earlier packaged deadline provenance is explicitly separated in the artifact. PMPro/Node transfer is task 5.2, not inferred from a small PHP control.
+13 -6
View File
@@ -1,7 +1,7 @@
{ {
"feature_name": "pre-push-safety-net", "feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z", "created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-13T11:21:07.667199+00:00", "updated_at": "2026-09-13T12:50:25.249104+00:00",
"language": "en", "language": "en",
"phase": "implementation", "phase": "implementation",
"approvals": { "approvals": {
@@ -19,7 +19,7 @@
} }
}, },
"ready_for_implementation": true, "ready_for_implementation": true,
"implementation_approval_note": "Groups 1\u20134 verified complete. Tasks 4.1/4.2 independently approved; 4.3 used the documented Kiro manual review fallback after agent thread limit. Remaining tasks require review and fresh verification.", "implementation_approval_note": "Group 5 authorized. Task5.1 verified through manual Kiro fallback review; 5.2 next. Real capabilities remain disabled.",
"depends_on": [ "depends_on": [
"contributor-scan" "contributor-scan"
], ],
@@ -60,6 +60,11 @@
"at": "2026-09-13T10:26:47.307154+00:00", "at": "2026-09-13T10:26:47.307154+00:00",
"source": "move on to group 4 as a whole", "source": "move on to group 4 as a whole",
"action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending." "action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending."
},
{
"at": "2026-09-13T12:40:57.961577+00:00",
"source": "approved",
"action": "Authorized group 5 after the group 4 completion summary: explicit transaction replay, then the first PMPro/frozen-core Node experiment. Review and fresh evidence remain required."
} }
], ],
"task_review": { "task_review": {
@@ -86,15 +91,17 @@
"3.4", "3.4",
"4.1", "4.1",
"4.2", "4.2",
"4.3" "4.3",
"5.1"
], ],
"remaining_tasks": 13, "remaining_tasks": 12,
"verification_record": "implementation-group-4.md", "verification_record": "implementation-group-5.md",
"verification_records": [ "verification_records": [
"implementation-group-1.md", "implementation-group-1.md",
"implementation-group-2.md", "implementation-group-2.md",
"implementation-group-3.md", "implementation-group-3.md",
"implementation-group-4.md" "implementation-group-4.md",
"implementation-group-5.md"
], ],
"next_group": 5, "next_group": 5,
"blocked_tasks": [] "blocked_tasks": []
+3 -2
View File
@@ -3,7 +3,7 @@
Status: task plan approved by the maintainer on 2026-09-12. Groups 1–4 (tasks 1.1–4.3) Status: task plan approved by the maintainer on 2026-09-12. Groups 1–4 (tasks 1.1–4.3)
are complete after task reviews and fresh verification (14/27 subtasks). Task 4.3 used are complete after task reviews and fresh verification (14/27 subtasks). Task 4.3 used
the documented manual review fallback after agent dispatch became unavailable. the documented manual review fallback after agent dispatch became unavailable.
Groups 5–8 remain pending; no production capability or hook latency is admitted. Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pending; no production capability or hook latency is admitted.
## Execution contract ## Execution contract
@@ -140,10 +140,11 @@ Groups 5–8 remain pending; no production capability or hook latency is admitte
- _Requirements: 3.1, 3.2, 3.3, 5.1, 5.2, 5.5, 7.3_ - _Requirements: 3.1, 3.2, 3.3, 5.1, 5.2, 5.5, 7.3_
- [ ] 5. Prove the first end-to-end experiment before cache optimization - [ ] 5. Prove the first end-to-end experiment before cache optimization
- [ ] 5.1 Wire explicit base/head replay through the full transaction - [x] 5.1 Wire explicit base/head replay through the full transaction
- Connect immutable snapshots, ranker decisions, existing arbiter, targeted comparison, admission diagnostics and compact artifacts through an explicit local replay command. - Connect immutable snapshots, ranker decisions, existing arbiter, targeted comparison, admission diagnostics and compact artifacts through an explicit local replay command.
- Apply the shared budget to preparation and both revisions; no model/service or installed hook is required. All runtime errors preserve coverage state. - Apply the shared budget to preparation and both revisions; no model/service or installed hook is required. All runtime errors preserve coverage state.
- Done when a controlled security change and fixed twin traverse the production runner through the existing ranker and produce replayable results with truthful deadlines. - Done when a controlled security change and fixed twin traverse the production runner through the existing ranker and produce replayable results with truthful deadlines.
- _Verified 2026-09-13:_ manual Kiro fallback review APPROVED; 1,236 tests passed, nine skipped; final packaged security replay and fixed/deadline controls verified. See `implementation-group-5.md`.
- _Boundary: Push runner, CLI, Snapshot Adapter, engine and policy integration_ - _Boundary: Push runner, CLI, Snapshot Adapter, engine and policy integration_
- _Depends: 1.3, 3.1, 3.4, 4.3_ - _Depends: 1.3, 3.1, 3.4, 4.3_
- _Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3_ - _Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3_
@@ -1,10 +1,10 @@
{ {
"counts": { "counts": {
"load_bearing": 101, "load_bearing": 102,
"standalone_capability": 9 "standalone_capability": 9
}, },
"modules": 110, "modules": 111,
"note": "Refreshed after pre-push-safety-net task 4.3: compact reporting preserves full evidence and decisions; transaction replay remains task 5.1.", "note": "Refreshed for task 5.1: explicit local replay integrates the existing ranker and comparison policy.",
"orphaned": [], "orphaned": [],
"rows": [ "rows": [
{ {
@@ -104,6 +104,7 @@
"model.endpoint", "model.endpoint",
"pairs", "pairs",
"push.policy", "push.policy",
"push.runner",
"regress.candidate", "regress.candidate",
"regress.recipes", "regress.recipes",
"run" "run"
@@ -135,7 +136,8 @@
"model.pipeline", "model.pipeline",
"model.scan", "model.scan",
"model.taint", "model.taint",
"push.policy" "push.policy",
"push.runner"
], ],
"module": "cpg", "module": "cpg",
"reason": "model-grounded-detection Req 4: the CPG seam package" "reason": "model-grounded-detection Req 4: the CPG seam package"
@@ -157,7 +159,8 @@
"model.pipeline", "model.pipeline",
"model.scan", "model.scan",
"model.taint", "model.taint",
"push.policy" "push.policy",
"push.runner"
], ],
"module": "cpg.backend", "module": "cpg.backend",
"reason": "model-grounded-detection Req 4.1/4.5: the single Joern subprocess boundary" "reason": "model-grounded-detection Req 4.1/4.5: the single Joern subprocess boundary"
@@ -327,6 +330,7 @@
"model.scan", "model.scan",
"pairs", "pairs",
"preprocess", "preprocess",
"push.runner",
"stage_processors" "stage_processors"
], ],
"module": "mapping", "module": "mapping",
@@ -362,6 +366,7 @@
"push.contracts", "push.contracts",
"push.policy", "push.policy",
"push.report", "push.report",
"push.runner",
"push.snapshot" "push.snapshot"
], ],
"module": "model", "module": "model",
@@ -407,6 +412,7 @@
"push.contracts", "push.contracts",
"push.policy", "push.policy",
"push.report", "push.report",
"push.runner",
"push.snapshot" "push.snapshot"
], ],
"module": "model.contracts", "module": "model.contracts",
@@ -497,7 +503,8 @@
"cli", "cli",
"model.layout", "model.layout",
"model.scan", "model.scan",
"push.policy" "push.policy",
"push.runner"
], ],
"module": "model.regions", "module": "model.regions",
"reason": "contributor-scan Req 2: regions from entry points, with the families each admits" "reason": "contributor-scan Req 2: regions from entry points, with the families each admits"
@@ -515,7 +522,8 @@
"importers": [ "importers": [
"cli", "cli",
"push.policy", "push.policy",
"push.report" "push.report",
"push.runner"
], ],
"module": "model.scan", "module": "model.scan",
"reason": "contributor-scan Req 3: the repository driver -- one CPG, one budget, ranked spend" "reason": "contributor-scan Req 3: the repository driver -- one CPG, one budget, ranked spend"
@@ -523,7 +531,8 @@
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
"cli" "cli",
"push.runner"
], ],
"module": "model.shipped", "module": "model.shipped",
"reason": "reachable from gate, map_gate, pair_gate or the CLI" "reason": "reachable from gate, map_gate, pair_gate or the CLI"
@@ -630,6 +639,7 @@
"model.partitions", "model.partitions",
"model.scan", "model.scan",
"pairs", "pairs",
"push.runner",
"rank", "rank",
"regress.candidate", "regress.candidate",
"semantic.overlay", "semantic.overlay",
@@ -668,8 +678,10 @@
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
"cli",
"push.policy", "push.policy",
"push.report", "push.report",
"push.runner",
"push.snapshot" "push.snapshot"
], ],
"module": "push", "module": "push",
@@ -680,6 +692,7 @@
"importers": [ "importers": [
"push.policy", "push.policy",
"push.report", "push.report",
"push.runner",
"push.snapshot" "push.snapshot"
], ],
"module": "push.contracts", "module": "push.contracts",
@@ -688,20 +701,33 @@
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [ "importers": [
"push.report" "push.report",
"push.runner"
], ],
"module": "push.policy", "module": "push.policy",
"reason": "pre-push-safety-net Req 2.4/3.1/3.4/4.1: targeted same-semantics base comparison; replay integration pending" "reason": "pre-push-safety-net Req 2.4/3.1/3.4/4.1: targeted same-semantics base comparison; replay integration pending"
}, },
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [], "importers": [
"push.runner"
],
"module": "push.report", "module": "push.report",
"reason": "pre-push-safety-net Req 5.1/5.2/5.5/7.3: compact output and complete artifacts; runner integration pending" "reason": "pre-push-safety-net Req 5.1/5.2/5.5/7.3: compact output and complete artifacts; runner integration pending"
}, },
{ {
"classification": "load_bearing", "classification": "load_bearing",
"importers": [], "importers": [
"cli"
],
"module": "push.runner",
"reason": "reachable from gate, map_gate, pair_gate or the CLI"
},
{
"classification": "load_bearing",
"importers": [
"push.runner"
],
"module": "push.snapshot", "module": "push.snapshot",
"reason": "pre-push-safety-net Req 1.1/1.3/1.4/7.1: local immutable ref resolution; runner integration pending" "reason": "pre-push-safety-net Req 1.1/1.3/1.4/7.1: local immutable ref resolution; runner integration pending"
}, },
File diff suppressed because it is too large Load Diff
+24
View File
@@ -242,3 +242,27 @@ allowance; a stalled writer is killed. A hard cancellation can leave a private
`.partial-` file, which is never the published artifact and cannot establish a `.partial-` file, which is never the published artifact and cannot establish a
completed result. Serialization/write failure emits one notice without replacing completed result. Serialization/write failure emits one notice without replacing
the finding, coverage or push decision. the finding, coverage or push decision.
## Explicit local replay
Task 5.1 adds `ousast pre-push REPOSITORY --base BASE --head HEAD --artifact OUTSIDE_REPOSITORY.json`.
Both revisions must already exist locally. The command records their resolved commit identities,
materializes tracked object bytes in private scratch storage, discovers regions through the existing
mapper, and runs the existing evidence ranker, arbiter, targeted base comparison and admission policy.
It installs no hook and requires no model endpoint. All real capabilities remain experimental and
ineligible for normal alerts until task 8.3; diagnostic candidates remain in the JSON artifact.
`--deadline` defaults to 30 seconds, `--cancellation-allowance` to 2 seconds, and `--max-regions`
to 500. Preparation, both revisions and reporting share that elapsed budget. `--mode blocking`
and `--incomplete-coverage block` explicitly select enforcement; default advisory exit zero means
allow, never complete analysis. This initial replay interface requires the artifact outside the
analyzed repository, including resolved symlink destinations, to preserve the live source and index.
Artifact failure cannot change the actual allow/block decision.
`ops/smoke_replay.py --case security|fixed` exercises a real controlled PHP change through this
production runner, without supplying hand-selected regions. Its default 900-second budget is a
separately labeled runtime laboratory check, not hook latency. `--deadline 30` exercises cancellation;
it does not demand a positive after an interrupted query. Run with the network-disabled packaged
image, mounting the smoke script read-only as for the other smoke scripts. Snapshot manifests
record verified immutable source bytes; full scan records retain exact selected/deferred questions,
raw answers, per-question gaps, change context and candidate dispositions.
+89
View File
@@ -0,0 +1,89 @@
"""Real production replay on a controlled PHP security change and its fixed twin."""
from __future__ import annotations
import argparse
import json
import os
import subprocess
import tempfile
import time
from pathlib import Path
from openultrasast.config import PushConfig
from openultrasast.push.runner import replay
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--case", choices=("security", "fixed"), required=True)
parser.add_argument("--deadline", type=float, default=900)
args = parser.parse_args()
with tempfile.TemporaryDirectory(prefix="ousast-replay-smoke-") as temporary:
root = Path(temporary) / "repo"
root.mkdir()
def git(*parts: str) -> str:
return subprocess.run(["git", "-C", str(root), *parts], check=True, capture_output=True, timeout=10).stdout.decode().strip()
git("init")
git("config", "user.name", "Replay smoke")
git("config", "user.email", "[email protected]")
git("config", "core.hooksPath", os.devnull)
source = root / "api.php"
before = '<?php\nfunction handler() {\n $value = "fixed";\n eval($value);\n}\nhandler();\n'
after = before.replace('$value = "fixed";', '$value = $_GET["value"];')
source.write_text(before)
git("add", "api.php")
git("commit", "-m", "fixed")
fixed = git("rev-parse", "HEAD")
source.write_text(after)
git("add", "api.php")
git("commit", "-m", "security")
security = git("rev-parse", "HEAD")
base, head = (fixed, security) if args.case == "security" else (security, fixed)
# Verify readable immutable source before trusting any reported zeros.
readable = git("show", head + ":api.php")
assert len(readable.encode()) > 0
source.write_text("dirty live content")
index = (root / ".git/index").read_bytes()
status = git("status", "--porcelain")
artifact = Path(temporary) / "replay.json"
start = time.monotonic()
delivery = replay(root, base=base, head=head, artifact=artifact, config=PushConfig(deadline_seconds=args.deadline))
payload = json.loads(artifact.read_text()) if artifact.exists() else None
print(
json.dumps(
{
"case": args.case,
"source_bytes": len(readable.encode()),
"elapsed_seconds": time.monotonic() - start,
"exit_code": delivery.exit_code,
"report_error": delivery.error,
"reporting_seconds": delivery.reporting_seconds,
"text": delivery.text,
"artifact": payload,
},
indent=2,
),
flush=True,
)
assert source.read_text() == "dirty live content"
assert (root / ".git/index").read_bytes() == index
assert git("status", "--porcelain") == status
assert payload and payload["scans"], "production runner produced no scan evidence"
assert delivery.result.finding_status == "none" and delivery.exit_code == 0
if args.deadline >= 900:
head_scan = next(r["scan"] for r in payload["scans"] if r["side"] == "head")
assert head_scan["scope"]["ranking_mode"] == "evidence"
if args.case == "security":
assert any(d["candidate"]["delta"]["novelty"] in ("new", "worsened") for d in payload["admission"]["dispositions"]), (
"supported security control missing: inspect upstream evidence before claiming replay complete"
)
else:
assert not head_scan["findings"], "fixed twin retained a modeled finding"
print(json.dumps({"verified": True, "case": args.case}), flush=True)
if __name__ == "__main__":
main()
+29
View File
@@ -110,6 +110,17 @@ def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="ousast") parser = argparse.ArgumentParser(prog="ousast")
subparsers = parser.add_subparsers(dest="command", required=True) subparsers = parser.add_subparsers(dest="command", required=True)
replay_parser = subparsers.add_parser("pre-push", help="replay explicit local base/head revisions (experimental)")
replay_parser.add_argument("path", type=Path)
replay_parser.add_argument("--base", required=True)
replay_parser.add_argument("--head", required=True)
replay_parser.add_argument("--artifact", type=Path, required=True)
replay_parser.add_argument("--deadline", type=float, default=30.0)
replay_parser.add_argument("--cancellation-allowance", type=float, default=2.0)
replay_parser.add_argument("--max-regions", type=int, default=500)
replay_parser.add_argument("--mode", choices=("advisory", "blocking"), default="advisory")
replay_parser.add_argument("--incomplete-coverage", choices=("allow", "block"), default="allow")
scan = subparsers.add_parser("scan", help="scan a local repository") scan = subparsers.add_parser("scan", help="scan a local repository")
scan.add_argument("path", type=Path) scan.add_argument("path", type=Path)
scan.add_argument("--mode", choices=("quick", "standard", "deep"), default="quick") scan.add_argument("--mode", choices=("quick", "standard", "deep"), default="quick")
@@ -189,6 +200,24 @@ def main(argv: list[str] | None = None) -> int:
candidates.add_argument("--json", action="store_true") candidates.add_argument("--json", action="store_true")
args = parser.parse_args(argv) args = parser.parse_args(argv)
if args.command == "pre-push":
from .config import PushConfig
from .push.runner import replay
try:
settings = PushConfig(
deadline_seconds=args.deadline,
cancellation_allowance_seconds=args.cancellation_allowance,
mode=args.mode,
incomplete_coverage_policy=args.incomplete_coverage,
)
delivery = replay(
args.path, base=args.base, head=args.head, artifact=args.artifact, config=settings, max_regions=args.max_regions
)
except ValueError as error:
parser.error(str(error))
print(delivery.text, end="")
return delivery.exit_code
if args.command == "scan": if args.command == "scan":
return _scan(args.path, args.config, args.mode, args.fail_on) return _scan(args.path, args.config, args.mode, args.fail_on)
if args.command == "index": if args.command == "index":
+21 -5
View File
@@ -195,9 +195,25 @@ def _valid_answer(scan: ModelScanResult, question: QuestionIdentity, operations:
return len(rows) == sum(op.question == question for op in operations) return len(rows) == sum(op.question == question for op in operations)
def _context_boundaries(scan: ModelScanResult) -> tuple[str, ...]: def _context_boundaries(scan: ModelScanResult, question: QuestionIdentity | None = None) -> tuple[str, ...]:
# A missing scheduling vector is not missing arbiter evidence after its question completed. # Aggregate coverage keeps every gap. A candidate may ignore only a recognized
return tuple(b for b in scan.scope.unresolved_boundaries if not b.startswith("evidence_unknown:")) if scan.scope else ("scope_missing",) # question-owned boundary attached to a different, recorded question. Unknown
# ownership and graph/declaration gaps remain global.
if scan.scope is None:
return ("scope_missing",)
owned_prefixes = (
"context_projection_unavailable:contributor-scan:",
"context_scope_empty:contributor-scan:",
"dynamic_external_or_depth_context_unresolved:contributor-scan:",
)
others = {q.identity.question_id for q in scan.scope.selected if q.identity != question}
others.update(q.identity.question_id for q in scan.scope.deferred if q.identity != question)
return tuple(
boundary
for boundary in scan.scope.unresolved_boundaries
if not boundary.startswith("evidence_unknown:")
and not (question is not None and boundary.startswith(owned_prefixes) and boundary.rsplit(":", 1)[-1] in others)
)
def _counterpart(question: QuestionIdentity, context: ChangeContext) -> QuestionIdentity: def _counterpart(question: QuestionIdentity, context: ChangeContext) -> QuestionIdentity:
@@ -268,7 +284,7 @@ def compare_evidence(
base base
and base.scope and base.scope
and base.scope.population_complete and base.scope.population_complete
and not base.scope.deferred and all(question.reason == "tier_zero" for question in base.scope.deferred)
and not _context_boundaries(base) and not _context_boundaries(base)
and not base.degradations and not base.degradations
and _valid_answer(base, counterpart, base_ops) and _valid_answer(base, counterpart, base_ops)
@@ -285,7 +301,7 @@ def compare_evidence(
reason = "change_context_unresolved" reason = "change_context_unresolved"
elif ( elif (
not head.scope not head.scope
or _context_boundaries(head) or _context_boundaries(head, op.question)
or head.degradations or head.degradations
or not _valid_answer(head, op.question, head_ops) or not _valid_answer(head, op.question, head_ops)
or op.question not in {q.identity for q in head.scope.selected} or op.question not in {q.identity for q in head.scope.selected}
+9 -3
View File
@@ -15,9 +15,9 @@ from pathlib import Path
from types import MappingProxyType from types import MappingProxyType
from typing import Literal from typing import Literal
from openultrasast.model.contracts import ExecutionBudget from openultrasast.model.contracts import ChangeContext, ExecutionBudget
from openultrasast.model.scan import ModelScanResult from openultrasast.model.scan import ModelScanResult
from openultrasast.push.contracts import PushComparison, PushResolution, PushResult from openultrasast.push.contracts import PushComparison, PushResolution, PushResult, SnapshotManifest
from openultrasast.push.policy import AdmissionResult, _context_boundaries from openultrasast.push.policy import AdmissionResult, _context_boundaries
@@ -36,6 +36,8 @@ class PushReport:
provenance: Mapping[str, str] provenance: Mapping[str, str]
timings: Mapping[str, float] timings: Mapping[str, float]
resolution: PushResolution | None = None resolution: PushResolution | None = None
snapshots: tuple[SnapshotManifest, ...] = ()
change_context: ChangeContext | None = None
def __post_init__(self) -> None: def __post_init__(self) -> None:
object.__setattr__(self, "provenance", MappingProxyType(dict(self.provenance))) object.__setattr__(self, "provenance", MappingProxyType(dict(self.provenance)))
@@ -109,6 +111,8 @@ def _write_artifact(report: PushReport, target: Path, temporary: Path, connectio
with os.fdopen(fd, "w", encoding="utf-8") as stream: with os.fdopen(fd, "w", encoding="utf-8") as stream:
payload = { payload = {
"schema_version": 1, "schema_version": 1,
"snapshots": [item.to_payload() for item in report.snapshots],
"change_context": report.change_context.to_payload() if report.change_context else None,
"result": report.result.to_payload(), "result": report.result.to_payload(),
"admission": report.admission.to_payload(), "admission": report.admission.to_payload(),
"scans": [asdict(record) for record in report.scans], "scans": [asdict(record) for record in report.scans],
@@ -223,7 +227,9 @@ def render_report(report: PushReport, *, artifact: Path | None, error: str | Non
notices.append( notices.append(
"Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying." "Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
) )
if error: if error == "artifact_inside_repository":
notices.append("Details could not be saved. Choose an artifact path outside the analyzed repository.")
elif error:
notices.append("Details could not be saved. Retry with a writable artifact path and available reporting time.") notices.append("Details could not be saved. Retry with a writable artifact path and available reporting time.")
if notices: if notices:
lines.append("Notice: " + " ".join(notices)) lines.append("Notice: " + " ".join(notices))
+279
View File
@@ -0,0 +1,279 @@
"""Bounded, local base/head replay through the existing repository driver."""
from __future__ import annotations
import hashlib
import multiprocessing
import os
import pickle
import selectors
import time
from collections.abc import Callable
from dataclasses import asdict, replace
from pathlib import Path
from typing import Any, Literal, TypeVar, cast
from openultrasast.config import PushConfig
from openultrasast.cpg.backend import joern_version, resolve_cpg_backend
from openultrasast.mapping import analyze_entry_points
from openultrasast.model.contracts import ExecutionBudget
from openultrasast.model.regions import ScanRegion, regions_for
from openultrasast.model.scan import ScanBudget, scan_repository
from openultrasast.model.shipped import declared_sources
from openultrasast.preprocess import build_file_target, enumerate_source_files
from openultrasast.push.contracts import ComparisonAnalysis, SnapshotManifest
from openultrasast.push.policy import (
AdmissionCandidate,
AdmissionResult,
CapabilityKey,
admission_push_result,
admit_candidates,
compare_targeted_base,
semantics_digest,
)
from openultrasast.push.report import PushReport, ReportDelivery, ScanRecord, _completed_scan, deliver_report, render_report
from openultrasast.push.snapshot import SnapshotAdapter
_T = TypeVar("_T")
def _prepare(function: Callable[[], _T], budget: ExecutionBudget) -> _T:
"""Bound pure local discovery/metadata work, including serialization, on Linux.
Read incrementally while the child writes; no pipe-buffer deadlock or unbounded
blocking recv. The child never launches engines or project code.
"""
if time.monotonic() >= budget.deadline_monotonic:
raise TimeoutError("deadline_exhausted")
reader, writer = os.pipe()
def child() -> None:
os.close(reader)
try:
payload: tuple[bool, Any]
try:
payload = (True, function())
except Exception as error:
payload = (False, type(error).__name__)
with os.fdopen(writer, "wb") as stream:
pickle.dump(payload, stream, protocol=5)
finally:
os._exit(0)
process = multiprocessing.get_context("fork").Process(target=child)
data = bytearray()
try:
process.start()
os.close(writer)
writer = -1
with selectors.DefaultSelector() as selector:
selector.register(reader, selectors.EVENT_READ)
while selector.get_map():
remaining = budget.deadline_monotonic - time.monotonic()
if remaining <= 0:
raise TimeoutError("deadline_exhausted")
for key, _ in selector.select(min(0.05, remaining)):
chunk = os.read(key.fd, 65536)
if not chunk:
selector.unregister(key.fileobj)
else:
data.extend(chunk)
if len(data) > 32 * 1024**2:
raise ValueError("preparation_output_limit")
process.join(max(0, budget.deadline_monotonic - time.monotonic()))
if process.is_alive() or process.exitcode != 0 or not data:
raise RuntimeError("preparation_worker_failed")
success, result = pickle.loads(data) # trusted child, never repository-provided pickle
if not success:
raise RuntimeError("preparation_failed:" + result)
return cast(_T, result)
finally:
os.close(reader)
if writer >= 0:
os.close(writer)
if process.pid is not None:
if process.is_alive():
process.kill()
process.join(max(0, budget.deadline_monotonic + budget.cancellation_allowance_seconds - time.monotonic()))
if not process.is_alive():
process.close()
def _discover(root: Path) -> tuple[tuple[ScanRegion, ...], tuple[bytes, ...]]:
# Same proposer and declaration-based shipping rules as ordinary scans. Do not
# call preprocess's live-HEAD probe on a materialized, intentionally Git-free tree.
files = enumerate_source_files(root)
targets = [build_file_target(root, path) for path in files]
regions = regions_for(analyze_entry_points(root, targets), targets, shipped=declared_sources(root))
# Conservatively seed every tracked declaration/non-source file. This is change
# context only, never a graph allowlist or a second scope selector.
sources = set(files)
declarations = tuple(os.fsencode(str(p.relative_to(root))) for p in root.rglob("*") if p.is_file() and p not in sources)
return regions, declarations
def _provenance(repository: Path, scan_budget: ScanBudget) -> dict[str, str]:
root = Path(__file__).resolve().parents[1]
def digest(paths: list[Path]) -> str:
hasher = hashlib.sha256()
for path in sorted(paths):
hasher.update(str(path.relative_to(root)).encode())
hasher.update(path.read_bytes())
return hasher.hexdigest()
version = joern_version()
engine = "joern-" + (".".join(map(str, version)) if version else "unavailable_or_unidentified")
return {
"repository": str(repository.absolute()),
"engine": engine,
"facts": digest(list((root / "ruleset").rglob("*.toml"))),
"queries": digest(list((root / "cpg/queries").glob("*.sc"))),
"policy": digest(list((root / "push").glob("*.py"))),
"core": digest(list((root / "model").glob("*.py")) + list((root / "cpg").glob("*.py"))),
"semantics": semantics_digest(engine_identity=engine, options=asdict(scan_budget)),
"ranking_mode": "evidence",
"model": "disabled",
"capabilities": "empty_registry_pending_task_8.3",
"cache": "cold_no_reuse",
}
def replay(
repository: Path,
*,
base: str,
head: str,
artifact: Path,
config: PushConfig | None = None,
backend: Any = None,
max_regions: int = 500,
) -> ReportDelivery:
"""Run a single explicit comparison; exit policy never stands for coverage."""
config = config or PushConfig()
if max_regions < 1:
raise ValueError("max_regions must be positive")
started = time.monotonic()
budget = ExecutionBudget(started + config.deadline_seconds, config.cancellation_allowance_seconds)
scan_budget = ScanBudget(max_model_calls=0, max_regions=max_regions, order_by_evidence=True)
records: list[ScanRecord] = []
manifests: list[SnapshotManifest] = []
reasons: list[str] = []
admission = AdmissionResult((), (), ())
comparison = None
context = None
timings: dict[str, float] = {
"deadline_seconds": config.deadline_seconds,
"cancellation_allowance_seconds": config.cancellation_allowance_seconds,
}
provenance = {key: "unavailable_preparation_incomplete" for key in ("engine", "facts", "queries", "policy")}
provenance.update(repository=str(repository.absolute()), model="disabled", requested_base=base, requested_head=head)
stage = "resolution"
stage_started = started
try:
adapter = SnapshotAdapter(repository, execution_budget=budget)
comparison = adapter.resolve_replay(base, head)
timings[stage + "_seconds"] = time.monotonic() - stage_started
stage, stage_started = "provenance", time.monotonic()
provenance.update(_prepare(lambda: _provenance(repository, scan_budget), budget))
timings[stage + "_seconds"] = time.monotonic() - stage_started
stage, stage_started = "preparation", time.monotonic()
with adapter.materialize(comparison.head_oid, budget=budget) as tip:
manifests.append(tip.manifest)
assert comparison.base_oid is not None
with adapter.materialize(comparison.base_oid, budget=budget) as old:
manifests.append(old.manifest)
for manifest in manifests:
reasons.extend("snapshot:" + boundary.reason for boundary in manifest.boundaries)
head_regions, head_declarations = _prepare(lambda: _discover(tip.root), budget)
base_regions, base_declarations = _prepare(lambda: _discover(old.root), budget)
context = adapter.compare(comparison, declaration_paths=tuple(set(head_declarations + base_declarations)), budget=budget)
context = replace(context, unresolved_boundaries=tuple(dict.fromkeys((*context.unresolved_boundaries, *reasons))))
reasons.extend(context.unresolved_boundaries)
timings[stage + "_seconds"] = time.monotonic() - stage_started
stage, stage_started = "head", time.monotonic()
head_scan = scan_repository(
tip.root,
head_regions,
backend=backend or resolve_cpg_backend(),
budget=scan_budget,
execution_budget=budget,
ranking_mode="evidence",
unit="repository",
population_complete=all(m.complete for m in manifests),
change_context=context,
)
records.append(ScanRecord(comparison, "head", head_scan))
timings[stage + "_seconds"] = time.monotonic() - stage_started
stage, stage_started = "base", time.monotonic()
delta = compare_targeted_base(
old.root,
head=head_scan,
head_regions=head_regions,
base_regions=base_regions,
context=context,
backend=backend or resolve_cpg_backend(),
execution_budget=budget,
scan_budget=scan_budget,
head_semantics=provenance["semantics"],
base_semantics=provenance["semantics"],
)
if delta.base_scan is not None:
records.append(ScanRecord(comparison, "base", delta.base_scan))
reasons.extend(delta.coverage_reasons)
timings[stage + "_seconds"] = time.monotonic() - stage_started
stage, stage_started = "admission", time.monotonic()
candidates = tuple(
AdmissionCandidate(
c,
CapabilityKey(
c.head_operation.question.language if c.head_operation else "unknown",
"unspecified",
"unspecified",
c.family,
c.head_operation.mechanism if c.head_operation else "taint",
"unreviewed",
delta.semantics,
),
comparison,
tuple(reasons),
)
for c in delta.candidates
)
admission = _prepare(lambda: admit_candidates(candidates), budget)
timings[stage + "_seconds"] = time.monotonic() - stage_started
stage, stage_started = "cleanup", time.monotonic()
timings[stage + "_seconds"] = time.monotonic() - stage_started
except Exception as error:
reasons.append(stage + "_failed:" + type(error).__name__)
timings[stage + "_seconds"] = time.monotonic() - stage_started
if time.monotonic() >= budget.deadline_monotonic:
reasons.append("deadline_exhausted")
head_records = [r.scan for r in records if r.side == "head"]
for record in records:
if record.scan.scope is not None:
reasons.extend(record.scan.scope.unresolved_boundaries)
if not records or not all(_completed_scan(r.scan) for r in records):
reasons.append("analysis_incomplete")
if not any(r.side == "base" for r in records):
reasons.append("base_comparison_unavailable")
admission = replace(admission, coverage_reasons=tuple(dict.fromkeys((*admission.coverage_reasons, *reasons))))
analyses: tuple[ComparisonAnalysis, ...] = ()
if comparison is not None:
scopes = tuple(scan.scope for scan in head_records if scan.scope is not None)
completed = tuple(outcome.identity for scan in head_records for outcome in scan.question_outcomes if outcome.status == "completed")
coverage: Literal["unavailable", "incomplete", "complete_within_scope"] = (
"unavailable" if not records else "incomplete" if admission.coverage_reasons else "complete_within_scope"
)
analyses = (ComparisonAnalysis(comparison, scopes, completed, coverage),)
result = admission_push_result(admission, analyses, config=config)
timings["total_seconds"] = time.monotonic() - started
report = PushReport(result, admission, tuple(records), provenance, timings, snapshots=tuple(manifests), change_context=context)
try:
artifact.resolve().relative_to(repository.resolve())
except ValueError:
pass
else:
write_error = "artifact_inside_repository"
return ReportDelivery(result, render_report(report, artifact=None, error=write_error), None, write_error)
return deliver_report(report, artifact, execution_budget=budget)
+26 -2
View File
@@ -44,7 +44,8 @@ class SnapshotInputError(ValueError):
class SnapshotAdapter: class SnapshotAdapter:
def __init__(self, repository: Path, *, comparison_base: str | None = None) -> None: def __init__(self, repository: Path, *, comparison_base: str | None = None, execution_budget: ExecutionBudget | None = None) -> None:
self.execution_budget = execution_budget
self.repository = repository.resolve() self.repository = repository.resolve()
self.comparison_base = comparison_base self.comparison_base = comparison_base
# Hook-inherited Git variables may redirect objects or worktrees elsewhere. # Hook-inherited Git variables may redirect objects or worktrees elsewhere.
@@ -72,10 +73,22 @@ class SnapshotAdapter:
def _repository_path(self, *args: str) -> Path: def _repository_path(self, *args: str) -> Path:
# Binary framing also matters for the repository's own whitespace/non-UTF8 name. # Binary framing also matters for the repository's own whitespace/non-UTF8 name.
raw = self._bounded_git(("rev-parse", *args), 65536, ExecutionBudget(time.monotonic() + 10.0, 2.0), None, []) raw = self._bounded_git(
("rev-parse", *args), 65536, self.execution_budget or ExecutionBudget(time.monotonic() + 10.0, 2.0), None, []
)
return Path(os.fsdecode(raw.removesuffix(b"\n"))).resolve() return Path(os.fsdecode(raw.removesuffix(b"\n"))).resolve()
def _git(self, *args: str, input: str | None = None) -> subprocess.CompletedProcess[str]: def _git(self, *args: str, input: str | None = None) -> subprocess.CompletedProcess[str]:
if self.execution_budget is not None:
if input not in (None, ""):
raise SnapshotInputError("bounded_resolution_requires_empty_stdin")
try:
raw = self._bounded_git(tuple(args), 16 * 1024**2, self.execution_budget, None, [])
return subprocess.CompletedProcess(args, 0, raw.decode("utf-8", "surrogateescape"), "")
except SnapshotInputError as error:
if str(error).startswith("local_git_") and "_failed_exit_" in str(error):
return subprocess.CompletedProcess(args, 1, "", "")
raise
try: try:
return subprocess.run( return subprocess.run(
["git", "-c", "protocol.allow=never", "-c", "core.hooksPath=" + os.devnull, "-C", str(self.repository), *args], ["git", "-c", "protocol.allow=never", "-c", "core.hooksPath=" + os.devnull, "-C", str(self.repository), *args],
@@ -97,6 +110,17 @@ class SnapshotAdapter:
raise SnapshotInputError(f"local Git {args[0]} failed (exit {result.returncode})") raise SnapshotInputError(f"local Git {args[0]} failed (exit {result.returncode})")
return result.stdout.strip() return result.stdout.strip()
def resolve_replay(self, base: str, head: str) -> PushComparison:
"""Resolve explicitly supplied local revisions without inferred baselines or fetching."""
resolved = []
for revision in (base, head):
oid = self._required("rev-parse", "--verify", "--end-of-options", revision)
commit, kind = self._commit(oid)
if commit is None:
raise SnapshotInputError("replay_revision_" + kind)
resolved.append(commit)
return PushComparison(resolved[1], resolved[0], "explicit_local_replay", ("refs/heads/replay",))
def parse_updates(self, data: str) -> tuple[PushUpdate, ...]: def parse_updates(self, data: str) -> tuple[PushUpdate, ...]:
"""Validate every record before attempting resolution; malformed input fails closed.""" """Validate every record before attempting resolution; malformed input fails closed."""
updates = [] updates = []
+32
View File
@@ -206,3 +206,35 @@ def test_non_string_operation_cannot_prove_base_absence():
row = json.loads(scan(line=5).question_outcomes[0].raw_rows_json)[0] row = json.loads(scan(line=5).question_outcomes[0].raw_rows_json)[0]
row["sink"] = 42 row["sink"] = 42
assert delta(scan(), scan(line=5, rows=[row])).novelty == "unknown" assert delta(scan(), scan(line=5, rows=[row])).novelty == "unknown"
def test_unrelated_question_context_gap_does_not_poison_completed_comparison():
head = scan()
other = replace(head.scope.selected[0].identity, family="path")
gap = "dynamic_external_or_depth_context_unresolved:contributor-scan:" + other.question_id
head = replace(
head,
scope=replace(head.scope, selected=(*head.scope.selected, RankedQuestion(other, 0.0, ())), unresolved_boundaries=(gap,)),
question_outcomes=(*head.question_outcomes, QuestionOutcome(other, "unresolved", "change_context_incomplete", "[]")),
)
assert delta(head, scan(rows=[], findings=False)).novelty == "new"
own_gap = gap.rsplit(":", 1)[0] + ":" + head.scope.selected[0].identity.question_id
assert (
delta(replace(head, scope=replace(head.scope, unresolved_boundaries=(own_gap,))), scan(rows=[], findings=False)).novelty
== "unknown"
)
assert (
delta(replace(head, scope=replace(head.scope, unresolved_boundaries=("graph_incomplete",))), scan(rows=[], findings=False)).novelty
== "unknown"
)
def test_proven_tier_zero_other_families_do_not_invalidate_base_answer():
from openultrasast.model.contracts import DeferredQuestion
base = scan(rows=[], findings=False)
other = replace(base.scope.selected[0].identity, family="path")
base = replace(base, scope=replace(base.scope, deferred=(DeferredQuestion(other, "tier_zero", ()),)))
assert delta(scan(), base).novelty == "new"
base = replace(base, scope=replace(base.scope, deferred=(DeferredQuestion(other, "budget_exhausted", ()),)))
assert delta(scan(), base).novelty == "unknown"
+198
View File
@@ -0,0 +1,198 @@
from __future__ import annotations
import json
import os
import subprocess
import time
import pytest
from openultrasast.config import PushConfig
from openultrasast.cpg.backend import NullBackend
from openultrasast.push.runner import replay
def history(tmp_path):
root = tmp_path / "repo"
root.mkdir()
def git(*args):
return subprocess.run(["git", "-C", str(root), *args], check=True, capture_output=True).stdout.strip().decode()
git("init")
git("config", "user.name", "Replay test")
git("config", "user.email", "[email protected]")
git("config", "core.hooksPath", os.devnull)
source = root / "api.js"
source.write_text('function handler(req, res) {\n const value = "fixed";\n eval(value);\n}\napp.post("/run", handler);\n')
git("add", "api.js")
git("commit", "-m", "base")
base = git("rev-parse", "HEAD")
source.write_text(source.read_text().replace('"fixed"', "req.body.value"))
git("add", "api.js")
git("commit", "-m", "head")
head = git("rev-parse", "HEAD")
source.write_text("dirty source must never be scanned")
return root, base, head, git
def test_replay_unavailable_preserves_input_and_records_scope(tmp_path):
root, base, head, git = history(tmp_path)
before = (root / ".git/index").read_bytes(), git("status", "--porcelain")
artifact = tmp_path / "result.json"
delivery = replay(root, base=base, head=head, artifact=artifact, backend=NullBackend())
data = json.loads(artifact.read_text())
assert delivery.exit_code == 0
assert delivery.result.coverage_status in ("unavailable", "incomplete")
assert data["result"]["analyses"][0]["comparison"]["base_oid"] == base
assert data["result"]["analyses"][0]["comparison"]["head_oid"] == head
assert data["scans"][0]["scan"]["scope"]["ranking_mode"] == "evidence"
assert data["provenance"]["model"] == "disabled"
assert data["snapshots"][0]["bytes_read"] > 0
assert (root / "api.js").read_text() == "dirty source must never be scanned"
assert ((root / ".git/index").read_bytes(), git("status", "--porcelain")) == before
def test_missing_revision_is_unavailable_not_clean(tmp_path):
root, base, head, _ = history(tmp_path)
result = replay(root, base="missing-ref", head=head, artifact=tmp_path / "missing.json")
assert result.result.coverage_status == "unavailable"
assert result.exit_code == 0
assert "Notice:" in result.text
def test_expired_preparation_obeys_total_deadline(tmp_path):
root, base, head, _ = history(tmp_path)
start = time.monotonic()
result = replay(
root,
base=base,
head=head,
artifact=tmp_path / "expired.json",
config=PushConfig(deadline_seconds=0.001, cancellation_allowance_seconds=0.5),
)
assert time.monotonic() - start < 0.55
assert result.result.coverage_status != "complete_within_scope"
def test_strict_incomplete_policy_and_artifact_failure(tmp_path):
root, base, head, _ = history(tmp_path)
result = replay(
root,
base=base,
head=head,
artifact=tmp_path,
config=PushConfig(mode="blocking", incomplete_coverage_policy="block"),
backend=NullBackend(),
)
assert result.exit_code == 1 and result.error and result.artifact is None
def test_hanging_discovery_is_bounded(tmp_path, monkeypatch):
from openultrasast.push import runner
root, base, head, _ = history(tmp_path)
monkeypatch.setattr(runner, "_discover", lambda *args: time.sleep(5))
start = time.monotonic()
result = replay(
root,
base=base,
head=head,
artifact=tmp_path / "hung.json",
config=PushConfig(deadline_seconds=0.3, cancellation_allowance_seconds=0.5),
)
assert time.monotonic() - start < 0.85
assert result.result.coverage_status == "unavailable"
assert "deadline" in (tmp_path / "hung.json").read_text()
def test_cli_exposes_replay(tmp_path, capsys):
from openultrasast.cli import main
root, base, head, _ = history(tmp_path)
assert (
main(["pre-push", str(root), "--base", base, "--head", head, "--artifact", str(tmp_path / "cli.json"), "--deadline", "0.001"]) == 0
)
assert "Notice:" in capsys.readouterr().out
def test_replay_artifact_cannot_overwrite_live_repository(tmp_path):
root, base, head, _ = history(tmp_path)
source = root / "api.js"
before = source.read_bytes()
result = replay(root, base=base, head=head, artifact=source, backend=NullBackend())
assert result.artifact is None and result.error == "artifact_inside_repository"
assert source.read_bytes() == before
def test_snapshot_resolution_uses_shared_deadline(tmp_path, monkeypatch):
from openultrasast.model.contracts import ExecutionBudget
from openultrasast.push.snapshot import SnapshotAdapter, SnapshotInputError
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
launcher = bin_dir / "git"
launcher.write_text("#!/bin/sh\nexec sleep 5\n")
launcher.chmod(0o755)
monkeypatch.setenv("PATH", str(bin_dir) + os.pathsep + os.environ["PATH"])
started = time.monotonic()
with pytest.raises(SnapshotInputError, match="deadline"):
SnapshotAdapter(tmp_path, execution_budget=ExecutionBudget(started + 0.1, 0.5))
assert time.monotonic() - started < 0.65
def test_preparation_exit_zero_without_receipt_is_failure():
from openultrasast.model.contracts import ExecutionBudget
from openultrasast.push.runner import _prepare
with pytest.raises(RuntimeError, match="worker_failed"):
_prepare(lambda: os._exit(0), ExecutionBudget(time.monotonic() + 1, 0.5))
def test_base_failure_retains_completed_head_evidence(tmp_path, monkeypatch):
from test_push_policy import scan
from openultrasast.push import runner
root, base, head, _ = history(tmp_path)
seen = []
def head_scan(root, regions, **kwargs):
assert (root / "api.js").read_bytes() and regions
assert kwargs["ranking_mode"] == "evidence" and kwargs["change_context"].head_revision == head
seen.append(kwargs["execution_budget"])
return scan(path="api.js")
def failed_base(*args, **kwargs):
assert kwargs["execution_budget"] is seen[0]
raise RuntimeError("failed comparison")
monkeypatch.setattr(runner, "scan_repository", head_scan)
monkeypatch.setattr(runner, "compare_targeted_base", failed_base)
artifact = tmp_path / "partial.json"
result = replay(root, base=base, head=head, artifact=artifact)
data = json.loads(artifact.read_text())
assert result.result.coverage_status == "incomplete"
assert len(data["scans"][0]["scan"]["findings"]) == 1
assert len(data["result"]["analyses"][0]["completed_questions"]) == 1
assert "base_failed:RuntimeError" in data["admission"]["coverage_reasons"]
def test_preparation_large_result_does_not_deadlock():
from openultrasast.model.contracts import ExecutionBudget
from openultrasast.push.runner import _prepare
assert len(_prepare(lambda: b"x" * 1_000_000, ExecutionBudget(time.monotonic() + 2, 0.5))) == 1_000_000
def test_snapshot_boundaries_reach_engine_change_context(tmp_path):
root, base, head, git = history(tmp_path)
(root / "external.js").symlink_to("/unavailable/external.js")
git("add", "external.js")
git("commit", "-m", "external context")
head = git("rev-parse", "HEAD")
artifact = tmp_path / "boundary.json"
result = replay(root, base=base, head=head, artifact=artifact, backend=NullBackend())
data = json.loads(artifact.read_text())
assert result.result.coverage_status == "incomplete"
assert "snapshot:symlink_not_materialized" in data["change_context"]["unresolved_boundaries"]