feat(pre-push-safety-net): validate graph identity and isolated leases

This commit is contained in:
Marius Ciepluch
2026-09-13 16:05:51 +02:00
parent 78a7dfc8c4
commit 8c9c91be91
9 changed files with 438 additions and 5 deletions
+15
View File
@@ -265,6 +265,21 @@ results. A query/fact change invalidates its answers without automatically rebui
input/frontend/overlay semantics are unchanged. This supports iteration on the core as well as
repeated push checks; it does not authorize reusing stale evidence after a source change.
Preparation is also a reusable artifact boundary. Task 5.2 measured PMPro exhausting
30 seconds after materialization and before graph construction. Cache complete discovery
by immutable source tree identity plus discovery code, shipping/layout facts and configuration;
retain snapshot readability proofs and declaration/boundary records. Reuse must preserve
repository-relative locations and must never deserialize repository-controlled executable objects.
Measure preparation, graph and query hits separately. An identical-tip hit demonstrates reuse,
not representative changed-code latency. Dependency/configuration changes invalidate affected
preparation and graph/query context even when an individual source file is unchanged.
Rollout approval remains an evidence decision under Requirement 6.5 and evaluation.md.
Group 6 establishes correctness and bounded reuse; group 8 must still demonstrate the joint
changed-code latency, completion, precision and recall gates on declared populations. Missing
independent positive/fixed/benign populations or an empty capability registry cannot yield GO.
No threshold, population denominator or alert eligibility rule is relaxed by this amendment.
Only validated, complete artifacts can satisfy complete-result lookups. Partial artifacts may be
retained for diagnostics but never promoted by cache presence. Publish atomically from private
temporary paths under a per-key lock; another push may wait only within its deadline. Apply a
@@ -0,0 +1,39 @@
# Group 6 implementation and verification
The maintainer authorized 6.1–6.4 together on 2026-09-13. A fresh implementer dispatch
failed with `agent thread limit reached`; the explicit kiro-impl manual fallback applies.
Reviews below are main-context adversarial reviews, not independent reviewer evidence.
The design amendment makes measured preparation/discovery costs explicit. Rollout gates
and the empty default capability registry are unchanged.
## Task 6.1 brief
Requirements 4.2, 4.3, 4.4 and 7.3; design: Cache and Engine Boundary.
Backend-owned identities bind readable source paths/bytes, declarations, exclusions,
frontend/engine versions, overlay implementation and options. Complete graphs require a
named source census, methods, calls and the dataflow overlay. Loading checks identity and
streamed graph digest, then issues a disposable copy through the existing CpgResult driver
contract. Old structural manifests without source paths cannot qualify. Sharded or partial
graphs cannot qualify. Query locations normalize to repository-relative paths.
## Status Report
- STATUS: READY_FOR_REVIEW
- TASK: 6.1
- RED_PHASE_OUTPUT: `/tmp/group6-1-red.log` and `/tmp/group6-1-flag-off.log`: four failures before implementation/with the temporary switch OFF. ON passed; switch removed and five final tests passed.
- TESTS_RUN: 105 focused tests passed; full suite 1246 passed, 9 skipped (105.31s), followed by the final additional driver integration test and focused rerun. Ruff and mypy passed.
- EVIDENCE: source-mounted nonroot Docker Joern 4.0.625, network disabled, 3 GiB limit: read 45 JavaScript bytes, census 1 file/4 methods/3 calls, sealed graph loaded through a disposable lease, original digest unchanged after query and cleanup. This is a correctness control, not a latency benchmark.
- CONCERNS: persistent publication and runner selection belong to 6.2–6.4; this task provides the validated backend lease contract. Main-context review fallback.
## Review Verdict
- VERDICT: APPROVED
- TASK: 6.1
- MECHANICAL_RESULTS: focused tests, full regression suite, ruff, mypy and diff whitespace checks passed; no introduced placeholders or secrets.
- FINDINGS: review repaired unbounded full-graph buffering to streamed hashing/copy; unreadable source walks fail explicitly; single PHP graph census passes through unchanged while sharded graphs cannot seal. Existing driver accepts and releases the leased CpgResult.
- SUMMARY: Backend identity and isolated leases satisfy task 6.1 within the approved boundary.
## Verification Result
- STATUS: VERIFIED
- CLAIM_TYPE: TASK
- CLAIM: Task 6.1 identity and lease behavior, not persistent runner reuse or rollout readiness.
- EVIDENCE: tests/test_graph_reuse.py, 105 focused passing tests, full suite and real Joern control above; measurement JSON retained in benchmarks/measurements/2026-09-13-graph-lease-control.json.
- GAPS: Warm push feasibility and independent capability eligibility remain unproven.
@@ -65,6 +65,11 @@
"at": "2026-09-13T12:40:57.961577+00:00",
"source": "approved",
"action": "Authorized group 5 after the group 4 completion summary: explicit transaction replay, then the first PMPro/frozen-core Node experiment. Review and fresh evidence remain required."
},
{
"at": "2026-09-13T13:58:52.336506+00:00",
"source": "continue with group 6 as a whole. do we need to amend the spec to get a go for the rollout",
"action": "Authorized group 6 (6.1\u20136.4). Clarified complete preparation/discovery reuse after measured group 5 bottleneck; rollout gates remain unchanged. Manual Kiro review fallback remains necessary after a fresh agent dispatch hit the thread limit."
}
],
"task_review": {
+2 -1
View File
@@ -161,7 +161,7 @@ hook latency is admitted.
- _Requirements: 6.2, 6.3, 6.4, 6.5, 8.1, 8.2, 8.3, 8.4_
- [ ] 6. Reuse valid graphs and results without stale conclusions
- [ ] 6.1 Validate graph artifact identity and leases
- [x] 6.1 Validate graph artifact identity and leases
- Have the backend own source/declaration/exclusion digests, frontend/engine/overlay options, graph census and complete/partial identity.
- Permit validated graph leases in the generic driver while preserving the fresh-build path; isolate potentially mutating loads and retain bounded cleanup.
- Done when identical valid inputs can reuse a graph while any changed graph input or failed census rejects the lease; original cached artifacts survive query loading unchanged.
@@ -181,6 +181,7 @@ hook latency is admitted.
- _Boundary: Push cache, policy integration_
- _Requirements: 3.4, 4.3, 4.4, 6.4, 7.2, 7.3_
- [ ] 6.4 Integrate cache reuse into the shared push deadline
- Reuse complete preparation/discovery by immutable tree and discovery/fact/configuration identity, retaining readable snapshot and declaration/boundary records; measure its costs and hits separately from graph/query reuse.
- Reuse identical snapshots/unchanged declared units across refs without merging different comparisons; changed units rebuild and stale graph relationships remain hints only.
- Account for lookup, lock wait, lease, build/query and result persistence in the same deadline; preserve completed outcomes on cancellation.
- Done when cached and uncached replay produce equal evidence/admission for the same completed questions under compatible semantics, with full equivalence under a sufficiently large controlled budget. Under the hook deadline, verify truthful extra completion/deferred scope from cache reuse rather than requiring identical scope; no stale answer survives a dependency/configuration edit.
@@ -0,0 +1,59 @@
{
"source_bytes": 45,
"artifact": {
"identity": {
"source_digest": "aab4a6878883db96c5eb1d0dd98e160e30d16fa2b215735bdac6ff5a41f9783d",
"declarations_digest": "none",
"exclusions_digest": "none",
"unit": "repository",
"language": "javascript",
"engine_version": "4.0.625",
"frontend": "jssrc2cpg",
"frontend_version": "4.0.625",
"overlay_semantics": [
"dataflowOss"
],
"overlay_options": [
[
"implementation",
"96d0a93a3edb7e4a999a380b5823cd6d593ae2a49cf2a503f923483eda43989a"
]
],
"frontend_options": [
[
"include_tests",
"True"
],
[
"java_options",
"default"
]
]
},
"graph_digest": "a02bef8f7d4d1f2c420cb9bb4c5cc9df2fe86d86ad4e429021c95ad76a996873",
"source_bytes": 45,
"census": {
"files": 1,
"methods": 4,
"calls": 3
},
"completeness": "complete",
"diagnostics": [],
"source_root": "/tmp/tmpb4ff84v8",
"source_paths": [
"a.js"
]
},
"lease_census": {
"files": "1",
"methods": "4",
"calls": "3",
"file_names": [
"a.js"
],
"overlays": "base,controlflow,typerel,callgraph,dataflowOss",
"maxHeapMB": "2048"
},
"original_unchanged": true,
"execution": "source-mounted openultrasast:dev image 7f71174daa9e; nonroot, network none, memory 3g; correctness control, not latency"
}
+110 -3
View File
@@ -1,7 +1,7 @@
"""Backend-owned graph provenance and live resource boundary (pre-push Req 4.3, 7.3).
Structural validity is necessary but insufficient for reuse. Task 6.1 will validate
actual source bytes, graph digest/census and engine compatibility before issuing leases.
Structural manifests carry provenance; the backend reads source bytes and seals a named
graph census, then verifies graph bytes and compatibility before issuing isolated leases.
"""
from __future__ import annotations
@@ -54,6 +54,8 @@ class GraphArtifact(Contract):
census: GraphCensus
completeness: Literal["complete", "partial"]
diagnostics: tuple[str, ...]
source_root: str = "relative"
source_paths: tuple[str, ...] = ()
def __post_init__(self) -> None:
super().__post_init__()
@@ -71,7 +73,7 @@ class GraphLease(Protocol):
The backend validates the artifact before issuing the lease and isolates loading
when it may mutate graph bytes. The consumer owns releasing it exactly once; the
backend's release is idempotent and bounded by the supplied monotonic deadline.
A cached artifact survives release. Implementations belong to task 6.1.
A cached artifact survives release.
"""
@property
@@ -81,3 +83,108 @@ class GraphLease(Protocol):
def path(self) -> Path: ...
def close(self, *, deadline_monotonic: float) -> None: ...
def digest_value(value: object) -> str:
import hashlib
import json
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode()).hexdigest()
def read_bytes(path: Path, deadline: float) -> bytes:
"""Read a regular local file without following a substituted final symlink."""
import os
import stat
import time
chunks: list[bytes] = []
fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
with os.fdopen(fd, "rb") as stream:
if not stat.S_ISREG(os.fstat(stream.fileno()).st_mode):
raise ValueError("artifact_not_regular")
while True:
if time.monotonic() >= deadline:
raise TimeoutError("deadline_exhausted")
chunk = stream.read(1024 * 1024)
if not chunk:
return b"".join(chunks)
chunks.append(chunk)
def source_inventory(root: Path, deadline: float) -> tuple[str, int, tuple[str, ...]]:
import hashlib
import os
import time
rows = []
size = 0
if root.is_symlink():
raise ValueError("symlink_source_root")
def unreadable(error: OSError) -> None:
raise error
for directory, dirs, files in os.walk(root, followlinks=False, onerror=unreadable):
if time.monotonic() >= deadline:
raise TimeoutError("deadline_exhausted")
if any((Path(directory) / name).is_symlink() for name in dirs):
raise ValueError("symlink_source")
for name in sorted(files):
path = Path(directory) / name
data = read_bytes(path, deadline)
size += len(data)
rows.append((path.relative_to(root).as_posix(), hashlib.sha256(data).hexdigest()))
if not rows or not size:
raise ValueError("source_unreadable_or_empty")
rows.sort()
return digest_value(rows), size, tuple(path for path, _ in rows)
@dataclass
class IsolatedGraphLease:
artifact: GraphArtifact
path: Path
_closed: bool = False
def close(self, *, deadline_monotonic: float) -> None:
from .backend import _DeadlineExpired, _remove_owned_tree
if self._closed:
return
try:
_remove_owned_tree(self.path.parent, deadline_monotonic)
except (OSError, _DeadlineExpired):
return
self._closed = True
def graph_bytes(path: Path, deadline: float, destination: Path | None = None) -> str:
"""Hash and optionally copy a graph in bounded chunks; never buffer the full graph."""
import hashlib
import os
import stat
import time
from contextlib import ExitStack
digest = hashlib.sha256()
count = 0
with ExitStack() as stack:
fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
source = stack.enter_context(os.fdopen(fd, "rb"))
if not stat.S_ISREG(os.fstat(fd).st_mode):
raise ValueError("graph_not_regular")
output = stack.enter_context(destination.open("xb")) if destination is not None else None
while True:
if time.monotonic() >= deadline:
raise TimeoutError("deadline_exhausted")
chunk = source.read(1024 * 1024)
if not chunk:
break
count += len(chunk)
digest.update(chunk)
if output is not None:
output.write(chunk)
if not count:
raise ValueError("graph_empty")
return digest.hexdigest()
+116 -1
View File
@@ -32,9 +32,10 @@ import time
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass, field, replace
from pathlib import Path
from typing import Protocol
from typing import Any, Protocol
from ..model.contracts import ExecutionBudget
from .artifact import GraphArtifact, GraphCensus, GraphIdentity
logger = logging.getLogger(__name__)
@@ -311,6 +312,118 @@ class JoernBackend:
self.last_failure = session.last_failure
self._diagnostics = list(session._diagnostics)
def graph_identity(
self,
root: Path,
*,
language: str,
declarations_digest: str,
exclusions_digest: str,
unit: str = "repository",
execution_budget: ExecutionBudget | None = None,
) -> GraphIdentity:
from .artifact import digest_value, read_bytes, source_inventory
deadline = execution_budget.deadline_monotonic if execution_budget else float("inf")
version = joern_version()
if version is None:
raise ValueError("engine_identity_unknown")
engine = ".".join(map(str, version))
source, _, _ = source_inventory(root, deadline)
code = digest_value([read_bytes(Path(__file__), deadline).hex(), read_bytes(self.queries_dir / "overlay.sc", deadline).hex()])
return GraphIdentity(
source,
declarations_digest,
exclusions_digest,
unit,
language,
engine,
_FRONTENDS.get(language, "joern-parse"),
engine,
(DATAFLOW_OVERLAY,),
(("implementation", code),),
(("include_tests", str(self.include_tests)), ("java_options", os.environ.get("JAVA_TOOL_OPTIONS", "default") or "default")),
)
def describe_graph(
self, result: CpgResult, root: Path, identity: GraphIdentity, *, execution_budget: ExecutionBudget | None = None
) -> GraphArtifact | None:
"""Seal only a named, complete, single graph census bound to bytes after loading."""
from .artifact import graph_bytes, source_inventory
budget = execution_budget or self.execution_budget
deadline = budget.deadline_monotonic if budget else float("inf")
try:
digest, size, paths = source_inventory(root, deadline)
if digest != identity.source_digest or result.unparsed:
return None
census = result.run("census", {})
if not isinstance(census, Mapping) or DATAFLOW_OVERLAY not in str(census.get("overlays", "")).split(","):
return None
names = census.get("file_names")
if not isinstance(names, list) or not all(isinstance(p, str) for p in names):
return None
normalized = {p.removeprefix(str(root) + "/").removeprefix("./") for p in names}
if not set(paths) <= normalized or int(str(census.get("shards", 1))) != 1:
return None
counts = GraphCensus(*(int(str(census[k])) for k in ("files", "methods", "calls")))
if counts.files < len(paths) or counts.methods < 1:
return None
return GraphArtifact(identity, graph_bytes(result.cpg_path, deadline), size, counts, "complete", (), str(root), paths)
except (OSError, ValueError, KeyError, TimeoutError):
return None
def load_graph(
self,
path: Path,
artifact: GraphArtifact,
expected: GraphIdentity,
*,
execution_budget: ExecutionBudget | None = None,
) -> CpgResult | None:
"""Validate sealed bytes, then query only a disposable private copy."""
from .artifact import IsolatedGraphLease, graph_bytes
budget = execution_budget or self.execution_budget
deadline = budget.deadline_monotonic if budget else float("inf")
cleanup_deadline = deadline + budget.cancellation_allowance_seconds if budget else float("inf")
scratch = None
try:
if artifact.identity != expected or artifact.completeness != "complete" or not artifact.source_paths:
return None
if artifact.census.files < len(artifact.source_paths) or artifact.census.methods < 1:
return None
scratch = Path(tempfile.mkdtemp(prefix="ousast-lease-"))
working = scratch / "cpg.bin"
if graph_bytes(path, deadline, working) != artifact.graph_digest:
raise ValueError("graph_digest_mismatch")
session = replace(self, execution_budget=budget)
lease = IsolatedGraphLease(artifact, working)
def normalize(value: object) -> Any:
if isinstance(value, str):
return value.replace(artifact.source_root + "/", "")
if isinstance(value, list):
return [normalize(item) for item in value]
if isinstance(value, dict):
return {key: normalize(item) for key, item in value.items()}
return value
return CpgResult(
working,
lambda kind, params: normalize(session.query(working, kind, params)),
lambda kind, requests: normalize(session.query_batch(working, kind, requests)),
cleanup=lambda: lease.close(deadline_monotonic=cleanup_deadline),
execution_diagnostics=lambda: tuple(session._diagnostics),
)
except (OSError, ValueError, TimeoutError):
if scratch is not None:
try:
_remove_owned_tree(scratch, cleanup_deadline)
except (OSError, _DeadlineExpired):
self._note("scratch_cleanup_incomplete")
return None
def _build_impl(self, root: Path, *, language: str = "", exclude: Sequence[str] = ()) -> CpgResult | None:
"""Build a CPG for ``root``. ``None`` on any failure — the caller degrades, never guesses.
@@ -671,6 +784,8 @@ class JoernBackend:
def query_across(self, shards: Sequence[Path], query: str, params: Mapping[str, object]) -> object | None:
"""One question over every shard, answers concatenated. ``None`` only when no shard could answer."""
if query == "census":
return self.query(shards[0], query, params) if len(shards) == 1 else None
merged: list[object] = []
answered = 0
for shard in shards:
+2
View File
@@ -23,6 +23,8 @@
ujson.Obj(
"files" -> cpg.file.size.toString,
"methods" -> cpg.method.size.toString,
"calls" -> cpg.call.size.toString,
"file_names" -> ujson.Arr.from(cpg.file.name.l),
"overlays" -> cpg.metaData.overlays.l.mkString(","),
"maxHeapMB" -> (Runtime.getRuntime.maxMemory / 1048576).toString
)
+90
View File
@@ -0,0 +1,90 @@
"""Real byte identity and isolated graph lease boundaries, without launching Joern."""
import time
from dataclasses import replace
import pytest
from openultrasast.cpg.backend import CpgResult, JoernBackend
from openultrasast.model.contracts import ExecutionBudget
def prepared(tmp_path, monkeypatch):
import openultrasast.cpg.backend as module
monkeypatch.setattr(module, "joern_version", lambda: (4, 0, 625))
root = tmp_path / "source"
root.mkdir()
(root / "a.js").write_text("function f(x) { return x; }")
graph = tmp_path / "cpg.bin"
graph.write_bytes(b"graph original")
backend = JoernBackend()
census = {"files": "1", "methods": "1", "calls": "0", "file_names": [str(root / "a.js")], "overlays": "dataflowOss"}
monkeypatch.setattr(JoernBackend, "query", lambda *args: census)
identity = backend.graph_identity(root, language="javascript", declarations_digest="declarations", exclusions_digest="exclusions")
result = CpgResult(graph, lambda *_: census)
artifact = backend.describe_graph(result, root, identity)
return backend, root, graph, identity, artifact
def test_identity_reads_content_and_rejects_changed_inputs(tmp_path, monkeypatch):
backend, root, graph, identity, artifact = prepared(tmp_path, monkeypatch)
assert artifact.source_bytes == (root / "a.js").stat().st_size > 0
lease = backend.load_graph(graph, artifact, identity)
assert lease is not None
lease.cleanup()
(root / "a.js").write_text("changed")
changed = backend.graph_identity(root, language="javascript", declarations_digest="declarations", exclusions_digest="exclusions")
assert changed != identity
assert backend.load_graph(graph, artifact, changed) is None
for field in ("declarations_digest", "exclusions_digest", "engine_version", "frontend_version"):
assert backend.load_graph(graph, artifact, replace(identity, **{field: "changed"})) is None
def test_lease_isolates_loads_and_cleanup_preserves_original(tmp_path, monkeypatch):
backend, root, graph, identity, artifact = prepared(tmp_path, monkeypatch)
leased = backend.load_graph(graph, artifact, identity)
assert leased is not None and leased.cpg_path != graph
leased.cpg_path.write_bytes(b"query mutated working copy")
assert graph.read_bytes() == b"graph original"
leased.cleanup()
leased.cleanup()
assert not leased.cpg_path.exists() and graph.exists()
def test_corruption_partial_census_and_deadline_are_misses(tmp_path, monkeypatch):
backend, root, graph, identity, artifact = prepared(tmp_path, monkeypatch)
graph.write_bytes(b"corrupted")
assert backend.load_graph(graph, artifact, identity) is None
graph.write_bytes(b"graph original")
assert backend.load_graph(graph, replace(artifact, completeness="partial", diagnostics=("dropped_file",)), identity) is None
budget = ExecutionBudget(time.monotonic() - 1, 0.1)
assert backend.load_graph(graph, artifact, identity, execution_budget=budget) is None
bad = CpgResult(graph, lambda *_: {"files": "1", "methods": "1", "calls": "0", "file_names": [], "overlays": "dataflowOss"})
assert backend.describe_graph(bad, root, identity) is None
def test_graph_identity_rejects_unreadable_or_symlink_sources(tmp_path, monkeypatch):
backend, root, *_ = prepared(tmp_path, monkeypatch)
(root / "link.js").symlink_to(root / "a.js")
with pytest.raises((ValueError, OSError)):
backend.graph_identity(root, language="javascript", declarations_digest="d", exclusions_digest="e")
def test_generic_driver_accepts_backend_validated_lease(tmp_path, monkeypatch):
from openultrasast.model.scan import _build, _dispose
backend, root, graph, identity, artifact = prepared(tmp_path, monkeypatch)
budget = ExecutionBudget(time.monotonic() + 5, 0.2)
class ReusingBackend:
def build(self, source, *, language, exclude, execution_budget):
assert source == root and language == "javascript"
assert execution_budget is budget
return backend.load_graph(graph, artifact, identity, execution_budget=execution_budget)
result = _build(ReusingBackend(), root, "javascript", execution_budget=budget)
assert result is not None
assert result.run("census", {})["file_names"] == ["a.js"]
_dispose(result)
assert graph.read_bytes() == b"graph original"