feat(pre-push-safety-net): measure frozen PMPro and Node replay feasibility

.kiro/specs/pre-push-safety-net/brief.md

@@ -1,6 +1,6 @@
 # Brief: pre-push-safety-net
 
-Status: requirements, design and tasks approved; groups 1–4 verified complete (14/27 subtasks). Groups 5–8 pending; real capability eligibility and hook latency remain unproven. Language: English.
+Status: requirements, design and tasks approved; groups 1–5 implemented and verified (16/27 subtasks). The first PMPro/Node experiment is measured NO-GO: six cold timeouts, no completed target checks. Groups 6–8 pending; normal capabilities remain disabled and warm hook latency unproven. Language: English.
 
 ## Problem
 

.kiro/specs/pre-push-safety-net/implementation-group-5.md

@@ -1,7 +1,7 @@
 # Implementation verification: group 5
 
 Group 5 was authorized by the maintainer's “approved” after group 4 completion.
-Task 5.1 is verified complete (15/27 total); task 5.2 is next.
+Tasks 5.1–5.2 are implemented and verified (16/27 total); the measured feasibility decision is NO-GO.
 Real capability eligibility and rollout readiness remain unproven.
 
 ## Execution and review mode
@@ -105,3 +105,80 @@ facts or detector semantics changed. The additional regression failed before rep
 - CLAIM: The reproduced Node cancellation path retains diagnostics and returns within 30 + 2 seconds.
 - EVIDENCE: Final packaged real Node replay returned in 30.1255 seconds with one complete diagnostic artifact and incomplete coverage. Final policy/core/facts/query hashes match the workspace. See `benchmarks/measurements/2026-09-13-replay-deadline-repair.json` for initial/partial-repair traces and final raw artifact.
 - GAPS: This proves the reproduced cancellation path, not representative warm performance or security coverage. Normal capabilities remain disabled.
+
+## Task 5.2 brief and measurement method
+
+Requirements 6.2–6.5 and 8.1–8.4: execute the prepared PMPro and Node security/fixed/benign
+changes through full replay, freeze the core before transfer, retain unresolved populations,
+and make an explicit feasibility decision. This task owns the evaluation harness, not new
+engine abstractions, framework facts, ranking weights or normal alert eligibility.
+
+The harness freezes the full input population and runtime profile before invoking the CLI.
+Authored variants are reproducible commits in a private Git object store, with byte/hash
+checks and the rest of each tree retained. The original cache is mounted read-only. All six
+cases run under 30 seconds plus a 2-second cancellation/report allowance, with complete CLI
+elapsed time measured. Exact artifacts/process exits remain separate from the scorer's
+witness association. Empty completed rows cannot manufacture a witnessed negative.
+
+Initial OFF-state evidence: `/tmp/ousast-experiment-red.log`, three failed acceptance tests.
+After implementation and flag removal, the same three tests pass, including whole-tree
+retention, unchanged source/index, reproducible authored identity, bad-hash refusal,
+timeout accounting and absence of fabricated negative evidence.
+
+The first v1 trial is retained as diagnostic history. Its Node post-deadline failures
+triggered the predecessor repair above; PMPro measurements overlapped regression tests.
+The v2 profile is frozen anew after repair; final timed runs have no competing test/build
+workload from this session. Both profiles and their distinct core/policy hashes are retained.
+The reserved Ghost workload remains untouched. No framework/fact adaptation or ranking
+retune is made from these Node results.
+
+## Final experiment: measured NO-GO
+
+[Joint scorecard](../../../benchmarks/measurements/2026-09-13-first-replay-experiment.json)
+records the final v2 profile, all metrics, hardware, exact case identities, initial trial
+and final-trial bundle hashes. The two gzip bundles are JSON mappings from each original
+filename to its exact UTF-8 contents; decompression was checked against every original file.
+They preserve all detailed CLI artifacts, stdout/stderr, pins, profiles and scorer inputs.
+
+| Workload | Cases | Whole CLI elapsed | Reached analysis | Result |
+|---|---:|---|---|---|
+| PMPro | 3 | 30.61–30.64 s | No; both trees materialized, discovery/preparation expired before change context | Timeout, no completed target check |
+| NodeGoat | 3 | 30.55–30.58 s | Driver reached after 7.78–7.96 s preparation; deadline consumed in head analysis/planning | Timeout, 436 deferred questions per case |
+
+All six processes exited 0 (advisory allow), saved artifacts and explicitly reported
+incomplete analysis. The harness exited **1 with a completed NO-GO decision**, distinct
+from an instrument crash. Both the initial and post-PMPro freeze match each final runtime
+and the current workspace's core/facts/query/policy hashes. CPU: Intel Core i7-8700,
+four exposed CPUs, 3 GiB container memory limit, network disabled. Tests/builds were not
+running during these final measurements. Filesystem caches were not flushed.
+
+The scorer retains all 11 unresolved cases; six were executed and the regression,
+unsupported-envelope and reserved populations were not scanned. No security change was
+recovered within budget, no witnessed fixed-side negative was established, and no normal
+alert was admitted. Precision has denominator zero and is undefined. The two benign
+pushes emitted no alert, but an empty registry cannot establish product usefulness.
+There are no warm-changed or identical-tip samples. Cold sample p50 is 30.58 s and p95
+30.64 s for six one-shot deadline-limited runs; these are not warm latency acceptance evidence.
+
+The negative result points first to preparation/discovery cost for PMPro, before graph
+persistence can help that path. Group6 can implement valid reuse, including compatible
+preparation work, but must not claim scale from identical-tip hits or bypass the ranker
+with another slicer. No new abstract capability or incremental graph mutation is justified
+by this measurement. Normal capability qualification remains task8.3.
+
+## Task 5.2 review verdict
+- VERDICT: APPROVED
+- TASK: 5.2
+- MECHANICAL_RESULTS: Full suite 1,242 passed, nine skipped, exit 0; Ruff/format (243 files), mypy (112 source files), compileall and detection/map/pair gates PASS; three behavioral OFF-state failures reproduced before implementation; no new placeholder markers/secrets; evaluation boundary respected.
+- FINDINGS: Initial instrument failures repaired in their owning driver/replay integration before the final freeze. All six final runtime provenance records match the frozen profile; scorer replay and complete compressed-artifact round trips reproduce exactly. No outstanding implementation blocker.
+- SUMMARY: Manual review approves the executed diagnostic experiment and truthful NO-GO assessment; it does not approve usefulness, warm latency or capability admission.
+
+## Group 5 verification and integration assessment
+- STATUS: VERIFIED
+- CLAIM_TYPE: TASK
+- CLAIM: Tasks5.1–5.2 implement full local replay and execute the first frozen PMPro/Node experiment with truthful evidence, failures, deadline accounting and an explicit feasibility decision.
+- EVIDENCE: Controlled security/fixed replay, packaged deadline repair, 1,242 passing tests, fresh static/regression checks, six final real CLI replays, exact frozen runtime hashes and reproducible 11-case scorecard. Task5.2 acceptance requires publishing the actual decision; this decision is NO-GO.
+- GAPS: No completed target checks in the 30-second cold workload, no warm measurements, no independent eligibility, and remaining groups6–8. The first usefulness/quality/latency milestone has not passed; the feature is not complete.
+
+Implementation and measurement integration: GO for the scoped work in group5.
+Product feasibility and rollout: **NO-GO**. Kiro state is 16/27 tasks verified; group6 next.

.kiro/specs/pre-push-safety-net/spec.json

@@ -1,7 +1,7 @@
 {
   "feature_name": "pre-push-safety-net",
   "created_at": "2026-09-12T15:35:00Z",
-  "updated_at": "2026-09-13T13:11:42.850127+00:00",
+  "updated_at": "2026-09-13T13:22:55.634332+00:00",
   "language": "en",
   "phase": "implementation",
   "approvals": {
@@ -19,7 +19,7 @@
     }
   },
   "ready_for_implementation": true,
-  "implementation_approval_note": "Group5 authorized. Task5.1 and its observed deadline regression repaired and verified with manual Kiro fallback; task5.2 resumes under a newly frozen core identity.",
+  "implementation_approval_note": "Groups1\u20135 implemented and verified (16/27). Group5 used manual Kiro fallback review, repaired observed predecessor deadline bugs, and executed a newly frozen PMPro/Node experiment. Measured feasibility NO-GO; real capabilities remain disabled. Groups6\u20138 pending.",
   "depends_on": [
     "contributor-scan"
   ],
@@ -92,9 +92,10 @@
       "4.1",
       "4.2",
       "4.3",
-      "5.1"
+      "5.1",
+      "5.2"
     ],
-    "remaining_tasks": 12,
+    "remaining_tasks": 11,
     "verification_record": "implementation-group-5.md",
     "verification_records": [
       "implementation-group-1.md",
@@ -103,7 +104,8 @@
       "implementation-group-4.md",
       "implementation-group-5.md"
     ],
-    "next_group": 5,
+    "next_group": 6,
-    "blocked_tasks": []
+    "blocked_tasks": [],
+    "feasibility": "NO-GO: six cold replay timeouts; no completed supported target checks; warm performance and eligibility unproven"
   }
 }

.kiro/specs/pre-push-safety-net/tasks.md

@@ -1,9 +1,11 @@
 # Implementation Plan: pre-push-safety-net
 
-Status: task plan approved by the maintainer on 2026-09-12. Groups 1–4 (tasks 1.1–4.3)
+Status: task plan approved by the maintainer on 2026-09-12. Groups 1–5 (tasks 1.1–5.2)
-are complete after task reviews and fresh verification (14/27 subtasks). Task 4.3 used
+are implemented, reviewed and verified (16/27 subtasks). Tasks 4.3 and group 5 used the
-the documented manual review fallback after agent dispatch became unavailable.
+documented Kiro manual review fallback after agent dispatch reached its thread limit.
-Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pending; no production capability or hook latency is admitted.
+The first experiment is measured NO-GO: all six cold PMPro/Node cases timed out without
+completed target checks. Groups 6–8 remain pending; no production capability or warm
+hook latency is admitted.
 
 ## Execution contract
 
@@ -139,7 +141,7 @@ Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pendin
   - _Boundary: Result and Hook Adapter_
   - _Requirements: 3.1, 3.2, 3.3, 5.1, 5.2, 5.5, 7.3_
 
-- [ ] 5. Prove the first end-to-end experiment before cache optimization
+- [x] 5. Prove the first end-to-end experiment before cache optimization
 - [x] 5.1 Wire explicit base/head replay through the full transaction
   - Connect immutable snapshots, ranker decisions, existing arbiter, targeted comparison, admission diagnostics and compact artifacts through an explicit local replay command.
   - Apply the shared budget to preparation and both revisions; no model/service or installed hook is required. All runtime errors preserve coverage state.
@@ -148,11 +150,12 @@ Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pendin
   - _Boundary: Push runner, CLI, Snapshot Adapter, engine and policy integration_
   - _Depends: 1.3, 3.1, 3.4, 4.3_
   - _Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3_
-- [ ] 5.2 Measure PMPro and frozen-core Node transfer
+- [x] 5.2 Measure PMPro and frozen-core Node transfer
   - _Prerequisite repaired 2026-09-13:_ 3.1/5.1 deadline regression verified in the packaged Node replay; re-freeze the experiment after this core identity change.
   - Replay each prepared security/fixed/benign case, first using the declared hook deadline; record timeouts and stage costs. A separately named extended lab run may obtain witnesses but cannot satisfy the hook latency gate.
   - Freeze the core after PMPro, run Node with existing facts, and classify instrument, mapping, fact, scope and domain gaps without tuning on an untouched holdout.
   - Done when one joint artifact reports actual evidence and outcomes for both workloads, with unresolved cases retained and an explicit feasibility decision; no normal capability is enabled here.
+  - _Verified 2026-09-13:_ manual Kiro review APPROVED; six final CLI cases measured under unchanged core, all timed out in 30.55–30.64 seconds with artifacts retained. Full 11-case population retained; feasibility NO-GO, no eligibility enabled. 1,242 tests passed, nine skipped. See `implementation-group-5.md`.
   - _Boundary: Evaluation harness, full-run integration_
   - _Depends: 1.2, 1.3, 5.1_
   - _Requirements: 6.2, 6.3, 6.4, 6.5, 8.1, 8.2, 8.3, 8.4_
@@ -234,3 +237,9 @@ Task 5.1 is also verified (15/27 total); task 5.2 and groups 6–8 remain pendin
   - Done when a reproducible validation artifact states GO only with all mandatory integration and joint quality/latency gates satisfied; otherwise it names the remaining NO-GO conditions without marking the feature complete.
   - _Boundary: Packaged runtime, full feature validation integration_
   - _Requirements: 3.1, 3.4, 5.1, 5.2, 5.3, 5.4, 5.5, 6.5, 7.1, 7.3, 7.4, 8.3, 8.4_
+
+
+## Implementation Notes
+
+- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
+- Cancellation must cover semantic planning and reporting, not only engine subprocesses: `_collect` and `_scope_work` now consume the shared deadline, and replay instantiates the budget-aware backend once. Preserve the 400-question failure-census and real Node artifact-retention regressions.

.kiro/steering/roadmap.md

@@ -46,28 +46,33 @@ engine are out of scope.
 - [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
   reuse, total deadline, change attribution, actionability admission and compact hook behavior.
   Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
-  (8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–4 are complete (14/27 tasks), including the resolved frontend-retention prerequisite.
+  (8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–5 are implemented and verified (16/27 tasks), including the resolved frontend-retention prerequisite.
 
 ### Next concrete step
 
-Groups 1–4 are verified: runtime, inputs, scoring, snapshots, shared deadline,
+Groups 1–5 now connect immutable replay to the existing ranker, detector, comparison,
-ranker scope, change context, vendor-free partitions, novelty comparison, actionability
+admission and compact artifacts. The first frozen PMPro/Node experiment is **NO-GO**:
-admission and compact reporting. Tasks 4.1/4.2 passed independent review; 4.3 used the
+all six cold security/fixed/benign cases timed out without completed target checks.
-Kiro manual review fallback after the agent service reached its thread limit.
+After repairing post-deadline semantic planning and a repeated availability probe,
-Group 5 is next: wire explicit full transaction replay, then run the first bounded
+every final CLI run saved its artifact and returned in 30.55–30.64 seconds. PMPro spent
-experiment. Real capabilities remain disabled pending independent evaluation.
+its deadline in preparation/discovery after materializing both trees; Node reached the
-Automatic dominance-only change-context projection remains an upstream limit to
+driver but did not complete checks. The exact 11-case population remains unresolved.
-retain during replay integration. See `pre-push-safety-net/implementation-group-4.md`
+No normal capability is enabled and these six cold samples establish no warm p95 claim.
-for verification and scope limits.
+
-The first complete experiment is 5.2, before persistent caching and hook
+Group 6 is next for valid graph/result identity, leases and reuse. Account for snapshot
-installation work. Keep PMPro as the primary core-optimization example, then freeze the core and
+and discovery costs before treating graph persistence as the sole optimization; record
-prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through
+compatible preparation reuse alongside graph reuse. Representative changed-code warm
-the same ranker, with a real deadline and vendor-free graph. Include benign pushes and retain Python
+latency and coverage must still be measured later, without substituting identical-tip hits.
-regressions. `pre-push-safety-net/evaluation.md` proposes NodeGoat for integration, a separate untouched
+Do not change ranking weights, add a second source-slicer or assume incremental Joern
-Node API repository for generalization, and libpng for C envelope/abstraction-limit evidence. Shared
+mutation from this negative performance evidence.
-abstract evidence and decisions are mandatory; framework facts/adaptations are reported separately.
+
-Measure realistic changed-code cache behavior before more ranker tuning. Broader runtime and
+Tasks 4.3 and group 5 used the documented Kiro manual review fallback after the agent
-capability admission follow that result; an LLM ranker is optional.
+thread limit; this is not independent review. See `pre-push-safety-net/implementation-group-5.md`
+and `benchmarks/measurements/2026-09-13-first-replay-experiment.json` for provenance,
+initial instrument failures, their repairs and the newly frozen final trial. The NodeGoat
+teaching workload did not cause a framework/fact adaptation or ranking retune. Ghost remains
+untouched; independent PHP/Python eligibility populations and C/C++ property support remain
+separate gaps. Automatic dominance-only context projection is still an upstream limit.
 
 ## Historical roadmap narrative — September 9–11
 

.kiro/steering/safety-net.md

@@ -79,11 +79,16 @@ updates as a shortcut to an unmeasured latency promise.
 The product direction above is user-requested. The amended requirements, design and initial
 evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12
 ("ok, next kiro phase"). The maintainer then approved the task plan and starting group 1
-("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–4 are now verified
+("approve.") and then group 2 ("ok, next kiro phase, approved"). Groups 1–5 are now
-complete (14/27 total), including the approved frontend-retention prerequisite.
+implemented, reviewed and verified (16/27), including the frontend-retention prerequisite
-Group 4 adds bound novelty, operation-specific admission and compact artifacts;
+and the deadline regression exposed by full Node replay. Tasks 4.3 and group 5 used the
-4.1/4.2 passed independent review and 4.3 used the documented Kiro manual fallback
+explicit Kiro manual review fallback after the agent thread limit.
-after agent dispatch became unavailable. Group 5 is next for full replay and the
+
-first bounded experiment. Task completion still requires review and fresh evidence.
+The first newly frozen PMPro/Node experiment is measured **NO-GO**: all six cold cases
-Real capability eligibility, automatic dominance context projection and hook
+timed out without completed supported target checks, while retaining all 11 declared cases.
-performance remain unproven; the default capability registry is empty.
+The repaired CLI saved all artifacts and returned in 30.55–30.64 seconds. PMPro exhausted
+the deadline in preparation/discovery before graph analysis, so compatible preparation
+reuse matters alongside graph persistence. Group 6 is next; warm changed-code performance
+and useful coverage remain mandatory later gates. No ranking/fact adaptation or holdout
+inspection occurred in the final transfer run. Real capability eligibility and automatic
+dominance context projection remain unproven; the default capability registry is empty.

benchmarks/push/README.md

@@ -89,3 +89,51 @@ results, per-capability metrics, coverage and cold/warm latency. Missing benign
 have unknown interruption rate. VAmPI's readable regression may be measured while
 independent admission prerequisites remain missing. Unsupported/unreviewed populations
 stay visible. All results remain experimental pending the later joint admission gate.
+
+## First full replay experiment (task 5.2)
+
+`measure.py` runs the six prepared PMPro/NodeGoat security, fixed and benign cases
+through the production `ousast pre-push` CLI. It does not supply a hand-selected
+source slice or regions. The complete 11-case population remains in the scorer;
+Ghost is not scanned, and the separate VAmPI/libpng populations are retained as
+unexecuted regression/envelope evidence rather than being silently dropped.
+
+Build the local image first, then run with a writable output parent and the existing
+repository-object cache mounted read-only:
+
+```sh
+docker run --rm --network none --memory 3g --entrypoint python \
+  -v "$PWD:/context:ro" \
+  -v "$HOME/.cache/openultrasast/repos:/cache:ro" \
+  -v /path/to/output-parent:/results \
+  openultrasast:dev /context/benchmarks/push/measure.py \
+  --manifest /context/benchmarks/push/inputs.json --cache /cache --out /results/new-run
+```
+
+The output directory must not already exist. Input validation opens and verifies
+pinned source/license blobs before any measurement. A private bare Git repository
+reads the existing objects and creates reproducible authored commits using only the
+specified edits. Every other tracked blob remains in the tree; the cache's files,
+index and refs are unchanged. No project code, package installation or network is used.
+
+`profile.json` is written before the first replay. PMPro runs first, followed by an
+explicit `freeze-after-pmpro.json` checkpoint and Node with identical installed core,
+facts, queries and policy. Both analysis and cancellation allowances stay 30 and 2
+seconds. Whole CLI elapsed time, separate available stage costs, exact input pins,
+source byte counts, selected/deferred identities, raw artifacts, process exits and
+terminal output remain available. `rank` denotes an observed selected-question position;
+an unlocated/unselected target retains null. A completed empty query cannot earn a
+witnessed negative or fixed-side silence. No transitive path is fabricated from rank.
+
+`scorecard.json` uses the existing frozen-population scorer. `decision.json` explicitly
+returns NO-GO because this initial cold diagnostic does not qualify independent
+capabilities or prove joint quality and warm latency. Exit **1** with a completed
+`decision.json` means measured NO-GO; a traceback without that completion record is a
+failed instrument. Zero CLI exit means advisory allow, never complete analysis.
+Missing artifacts and unanswered questions remain errors/unresolved, not clean results.
+
+Cold here means no persisted graph/query/comparison reuse; filesystem caches are not
+flushed. Six one-shot runs are diagnostic samples, not representative warm p95 evidence.
+Normal capabilities remain disabled, so zero alerts do not establish precision or useful
+recall. A separately named longer lab run may obtain witnesses, but cannot replace the
+30-second measurements or satisfy the hook's latency gate.

benchmarks/push/measure.py

@@ -0,0 +1,313 @@
+"""First frozen-core PMPro/Node diagnostic experiment, using local Git objects only."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import platform
+import subprocess
+import tempfile
+import time
+from dataclasses import replace
+from pathlib import Path
+
+from openultrasast.model.contracts import QuestionIdentity
+from openultrasast.model.scan import ScanBudget
+from openultrasast.push.runner import _provenance
+from openultrasast.push_inputs import validate_manifest
+from openultrasast.push_scoring import freeze_profile, score
+from openultrasast.repos import checkout_path, load_repo_recipe
+
+
+def git(root: Path, *args: str, data: bytes | None = None) -> bytes:
+    env = {key: value for key, value in os.environ.items() if not key.startswith("GIT_")}
+    env.update(
+        GIT_CONFIG_GLOBAL=os.devnull,
+        GIT_CONFIG_NOSYSTEM="1",
+        GIT_NO_REPLACE_OBJECTS="1",
+        GIT_NO_LAZY_FETCH="1",
+        GIT_ALLOW_PROTOCOL="",
+        GIT_GRAFT_FILE=os.devnull,
+        GIT_AUTHOR_NAME="Authored evaluation",
+        GIT_AUTHOR_EMAIL="[email protected]",
+        GIT_COMMITTER_NAME="Authored evaluation",
+        GIT_COMMITTER_EMAIL="[email protected]",
+        GIT_AUTHOR_DATE="2026-09-12T00:00:00+00:00",
+        GIT_COMMITTER_DATE="2026-09-12T00:00:00+00:00",
+    )
+    return subprocess.run(
+        ["git", "-c", "core.hooksPath=" + os.devnull, "-C", str(root), *args],
+        input=data,
+        capture_output=True,
+        check=True,
+        timeout=30,
+        env=env,
+    ).stdout
+
+
+def pin_snapshot(bare: Path, commit: str, snapshot: dict) -> str:
+    """Apply declared edits to a private Git index; retain every other tracked blob."""
+    git(bare, "read-tree", commit)
+    edited = False
+    for entry in snapshot["files"]:
+        content = git(bare, "cat-file", "blob", commit + ":" + entry["path"])
+        if not content:
+            raise ValueError("empty input bytes")
+        if edit := entry.get("edit"):
+            before, after = edit["before"].encode(), edit["after"].encode()
+            if not edit["provenance"].startswith("authored") or not before or content.count(before) != 1:
+                raise ValueError("authored edit must match exactly once")
+            if hashlib.sha256(content).hexdigest() != entry["original_sha256"]:
+                raise ValueError("original hash mismatch")
+            content = content.replace(before, after, 1)
+            edited = True
+        if len(content) != entry["size"] or hashlib.sha256(content).hexdigest() != entry["sha256"]:
+            raise ValueError("resulting input identity mismatch")
+        if entry.get("edit"):
+            blob = git(bare, "hash-object", "-w", "--stdin", data=content).decode().strip()
+            mode = git(bare, "ls-tree", commit, "--", entry["path"]).split(b" ", 1)[0].decode()
+            git(bare, "update-index", "--add", "--cacheinfo", mode, blob, entry["path"])
+    if not edited:
+        return commit
+    tree = git(bare, "write-tree").decode().strip()
+    return (
+        git(bare, "commit-tree", tree, "-p", commit, data=("Authored evaluation snapshot: " + snapshot["id"] + "\n").encode())
+        .decode()
+        .strip()
+    )
+
+
+def execution_record(case_id: str, artifact: dict, *, elapsed: float, target: dict | None = None) -> dict:
+    """Associate existing answers; empty rows cannot manufacture a witnessed negative."""
+    selected, deferred, queries = [], [], []
+    rank = None
+    modes = set()
+    for record in artifact["scans"]:
+        if record["side"] != "head":
+            continue
+        scan = record["scan"]
+        if scope := scan.get("scope"):
+            modes.add(scope["ranking_mode"])
+            if target is not None:
+                rank = next(
+                    (
+                        index
+                        for index, q in enumerate(scope["selected"], 1)
+                        if all(q["identity"][key] == target[key] for key in ("path", "function", "family"))
+                    ),
+                    None,
+                )
+            selected.extend(QuestionIdentity.from_payload(q["identity"]).question_id for q in scope["selected"])
+            deferred.extend(QuestionIdentity.from_payload(q["identity"]).question_id for q in scope["deferred"])
+        for outcome in scan["question_outcomes"]:
+            identity = QuestionIdentity.from_payload(outcome["identity"])
+            query = {
+                "id": identity.question_id,
+                "status": "answered" if outcome["status"] == "completed" else "unanswered",
+                "outcome": "unknown",
+            }
+            # Keep the exact question/answer association. A raw finding reached from
+            # another function is not a proved transitive path for this scorer.
+            findings = [
+                f
+                for f in scan["findings"]
+                if f["family"] == identity.family
+                and f["site"].startswith(identity.path + ":")
+                and f["site"].rsplit(":", 1)[-1] == identity.function
+            ]
+            if outcome["status"] == "completed" and findings:
+                target_sites = {f"{target['path']}:{line}:{target['function']}" for line in target["lines"]} if target else set()
+                finding = next((f for f in findings if f["site"] in target_sites), findings[0])
+                query.update(outcome="positive", finding={**finding, "site": finding["site"].rsplit(":", 1)[0]})
+            queries.append(query)
+    if modes and modes != {"evidence"}:
+        raise ValueError("runtime ranker differs from frozen evidence mode")
+    deltas = [d["candidate"]["delta"]["novelty"] for d in artifact["admission"]["dispositions"]]
+    novelty = next((d for d in ("new", "worsened", "unknown", "unchanged") if d in deltas), "unknown")
+    reasons = artifact["admission"]["coverage_reasons"]
+    coverage = (
+        "timeout"
+        if any("deadline" in reason for reason in reasons)
+        else ("complete" if artifact["result"]["coverage_status"] == "complete_within_scope" else "incomplete")
+    )
+    if artifact["admission"]["defects"]:
+        raise ValueError("unqualified experiment unexpectedly emitted an admitted alert")
+    return {
+        "case_id": case_id,
+        "ranking_mode": "evidence",
+        "rank": rank,
+        "rank_kind": "selected_question_position",
+        "selected": selected,
+        "deferred": deferred,
+        "queries": queries,
+        "alerts": [],
+        "delta": novelty,
+        "coverage": coverage,
+        "cache_state": "cold",
+        "timings": {
+            "total": elapsed,
+            "build": sum(r["scan"].get("build_seconds", 0) for r in artifact["scans"]),
+            "query": sum(r["scan"].get("query_seconds", 0) for r in artifact["scans"]),
+        },
+    }
+
+
+def write(path: Path, data: dict) -> None:
+    path.write_text(json.dumps(data, indent=2, allow_nan=False) + "\n")
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--manifest", type=Path, required=True)
+    parser.add_argument("--cache", type=Path, required=True)
+    parser.add_argument("--out", type=Path, required=True)
+    args = parser.parse_args()
+    args.out.mkdir(parents=True, exist_ok=False)
+    manifest = json.loads(args.manifest.read_bytes())
+    validation = validate_manifest(args.manifest, cache_root=args.cache, fetch=False)
+    write(args.out / "input-validation.json", validation)
+    settings = ScanBudget(max_model_calls=0, max_regions=500, order_by_evidence=True)
+    initial = _provenance(args.manifest.parent, settings)
+    keys = ("ranking_mode", "engine", "facts", "queries", "policy", "core")
+    config = {key: initial[key] for key in keys}
+    config.update(
+        version="first-replay-v2-2026-09-13",
+        scope="full tracked tree; declared first-party graph/target exclusions; evidence ranker",
+        hardware=platform.platform(),
+        budget_seconds=30,
+        targets=json.loads((args.manifest.parent / "diagnostic-config-v1.json").read_text())["targets"],
+    )
+    profile = freeze_profile(args.manifest.read_bytes(), config)
+    write(args.out / "profile.json", profile)  # Freeze on disk BEFORE any measurement.
+    run = {"profile_sha256": profile["profile_sha256"], "context": dict(config), "records": []}
+    evidence = {"cases": [], "core_changes": [], "adapter_changes": [], "freeze_after_pmpro": None}
+    ready = {c["id"]: c["status"] == "ready" for c in validation["cases"]}
+    snapshots = {s["id"]: s for s in manifest["snapshots"]}
+    # Fixed declared order; reserved, regression and unsupported populations stay
+    # in the joint scorer without scanning or tuning on the holdout.
+    cases = [c for prefix in ("pmpro-", "nodegoat-") for c in manifest["cases"] if c["id"].startswith(prefix)]
+    with tempfile.TemporaryDirectory(prefix="ousast-experiment-") as temporary:
+        bare = Path(temporary) / "objects.git"
+        bare.mkdir()
+        git(bare, "init", "--bare")
+        sources = {}
+        for case in cases:
+            for sid in (case["base"], case["tip"]):
+                if sid in sources or not ready[case["id"]]:
+                    continue
+                snapshot = snapshots[sid]
+                recipe = load_repo_recipe(args.manifest.parent / snapshot["recipe"])
+                recipe = replace(recipe, commit=snapshot.get("commit", recipe.commit))
+                source = checkout_path(recipe, args.cache)
+                objects = git(source, "rev-parse", "--path-format=absolute", "--git-path", "objects").decode().strip()
+                sources[sid] = (recipe.commit, objects)
+        (bare / "objects/info/alternates").write_text("\n".join(sorted({value[1] for value in sources.values()})) + "\n")
+        pins = {sid: pin_snapshot(bare, commit, snapshots[sid]) for sid, (commit, _) in sources.items()}
+        write(
+            args.out / "replay-pins.json",
+            {"snapshots": pins, "provenance": "Full upstream trees; declared authored edits in a private bare Git repository."},
+        )
+        for case in cases:
+            if not ready[case["id"]]:
+                evidence["cases"].append({"case_id": case["id"], "status": "missing_input_prerequisite"})
+                continue
+            current = _provenance(args.manifest.parent, settings)
+            if any(current[key] != initial[key] for key in keys):
+                raise ValueError("core/facts/query/policy changed after freezing")
+            if case["id"].startswith("nodegoat-") and evidence["freeze_after_pmpro"] is None:
+                evidence["freeze_after_pmpro"] = {key: current[key] for key in keys}
+                write(args.out / "freeze-after-pmpro.json", evidence["freeze_after_pmpro"])
+            artifact_path = args.out / (case["id"] + ".json")
+            command = [
+                "ousast",
+                "pre-push",
+                str(bare),
+                "--base",
+                pins[case["base"]],
+                "--head",
+                pins[case["tip"]],
+                "--artifact",
+                str(artifact_path),
+                "--deadline",
+                "30",
+                "--cancellation-allowance",
+                "2",
+                "--max-regions",
+                "500",
+            ]
+            start = time.monotonic()
+            process = subprocess.run(command, capture_output=True, text=True, timeout=62)
+            elapsed = time.monotonic() - start
+            (args.out / (case["id"] + ".stdout")).write_text(process.stdout)
+            (args.out / (case["id"] + ".stderr")).write_text(process.stderr)
+            entry = {
+                "case_id": case["id"],
+                "command": command,
+                "process_exit_code": process.returncode,
+                "elapsed_seconds": elapsed,
+                "artifact": artifact_path.name if artifact_path.exists() else None,
+                "terminal": process.stdout,
+                "core_unchanged": True,
+                "input_pins": {"base": pins[case["base"]], "head": pins[case["tip"]]},
+            }
+            if artifact_path.exists():
+                artifact = json.loads(artifact_path.read_text())
+                # Unmeasured provenance caused by preparation timeout is explicit;
+                # never replace it with a fictitious runtime measurement.
+                entry["runtime_provenance_available"] = all(artifact["provenance"].get(key) == initial[key] for key in keys)
+                record = execution_record(case["id"], artifact, elapsed=elapsed, target=config["targets"][case["id"]])
+                entry["coverage_reasons"] = artifact["admission"]["coverage_reasons"]
+                entry["stage_timings"] = artifact["timings"]
+                entry["snapshot_bytes_read"] = sum(s["bytes_read"] for s in artifact["snapshots"])
+                entry["classification"] = "scope/deadline" if record["coverage"] == "timeout" else "recorded context/capability limits"
+            else:
+                record = {
+                    "case_id": case["id"],
+                    "ranking_mode": "evidence",
+                    "rank": None,
+                    "selected": [],
+                    "deferred": [],
+                    "queries": [],
+                    "alerts": [],
+                    "delta": "unknown",
+                    "coverage": "error",
+                    "cache_state": "cold",
+                    "timings": {"total": elapsed, "build": 0, "query": 0},
+                }
+                entry["classification"] = "instrument/artifact unavailable; zero stage costs mean unmeasured"
+            if process.returncode != 0:
+                entry["classification"] = "instrument/CLI failure"
+                record["coverage"] = "error"
+            run["records"].append(record)
+            evidence["cases"].append(entry)
+            write(args.out / "run.json", run)
+            write(args.out / "execution-evidence.json", evidence)
+            print(json.dumps({"case": case["id"], "coverage": record["coverage"], "elapsed_seconds": elapsed}), flush=True)
+    scored = score(profile, validation, run)
+    write(args.out / "scorecard.json", scored)
+    decision = {
+        "decision": "NO-GO",
+        "capabilities_enabled": False,
+        "scorecard": "scorecard.json",
+        "reason": "First cold diagnostic experiment; independent eligibility and joint quality/warm-latency gates remain unmet.",
+        "recorded_cases": len(run["records"]),
+        "declared_population": len(manifest["cases"]),
+        "core_unchanged": True,
+        "adapter_changes": [],
+        "limits": [
+            "No warm-cache measurements",
+            "Untouched Ghost population not scanned",
+            "PHP/Python admission populations missing",
+            "libpng retained as unsupported envelope; no C/C++ detection claim",
+            "Zero normal alerts from an empty registry cannot establish precision or useful recall",
+        ],
+    }
+    write(args.out / "decision.json", decision)
+    print(json.dumps(decision), flush=True)
+    return 1  # Measurement completed; the feasibility/admission decision is explicitly NO-GO.
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

tests/test_push_experiment.py

@@ -0,0 +1,91 @@
+from __future__ import annotations
+
+import hashlib
+import importlib.util
+import subprocess
+from pathlib import Path
+
+import pytest
+
+spec = importlib.util.spec_from_file_location("push_measure", Path("benchmarks/push/measure.py"))
+measure = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(measure)
+
+
+def test_authored_snapshot_keeps_whole_tree_and_original_repository(tmp_path):
+    from test_push_runner import history
+
+    root, base, head, git = history(tmp_path)
+    (root / "unchanged.txt").write_text("first-party context remains in the whole tree")
+    git("add", "unchanged.txt")
+    git("commit", "-m", "unchanged context")
+    head = git("rev-parse", "HEAD")
+    # The helper consumes objects, never the deliberately dirty live source.
+    before = git("show", head + ":api.js") + "\n"
+    after = before.replace("req.body.value", '"fixed"')
+    entry = {
+        "id": "authored",
+        "files": [
+            {
+                "path": "api.js",
+                "size": len(after.encode()),
+                "sha256": hashlib.sha256(after.encode()).hexdigest(),
+                "original_sha256": hashlib.sha256(before.encode()).hexdigest(),
+                "edit": {"before": "req.body.value", "after": '"fixed"', "provenance": "authored repair"},
+            }
+        ],
+    }
+    bare = tmp_path / "objects.git"
+    subprocess.run(["git", "init", "--bare", str(bare)], check=True, capture_output=True)
+    (bare / "objects/info/alternates").write_text(str(root / ".git/objects") + "\n")
+    original = (root / ".git/index").read_bytes(), (root / "api.js").read_bytes()
+    oid = measure.pin_snapshot(bare, head, entry)
+    result = subprocess.run(["git", "-C", str(bare), "show", oid + ":api.js"], check=True, capture_output=True)
+    assert result.stdout.decode() == after
+    retained = subprocess.run(["git", "-C", str(bare), "show", oid + ":unchanged.txt"], check=True, capture_output=True)
+    assert retained.stdout == b"first-party context remains in the whole tree"
+
+    assert original == ((root / ".git/index").read_bytes(), (root / "api.js").read_bytes())
+    assert oid == measure.pin_snapshot(bare, head, entry), "authored object identities must be reproducible"
+    entry["files"][0]["sha256"] = "0" * 64
+    with pytest.raises(ValueError, match="hash|identity"):
+        measure.pin_snapshot(bare, head, entry)
+
+
+def payload():
+    return {
+        "result": {"coverage_status": "incomplete"},
+        "admission": {"coverage_reasons": ["deadline_exhausted"], "dispositions": [], "defects": []},
+        "scans": [],
+        "timings": {"total_seconds": 30},
+    }
+
+
+def test_timeout_stays_unresolved_and_uses_whole_cli_elapsed_time():
+    record = measure.execution_record("case", payload(), elapsed=30.8)
+    assert record["coverage"] == "timeout" and record["delta"] == "unknown"
+    assert record["rank"] is None and record["queries"] == [] and record["alerts"] == []
+    assert record["timings"]["total"] == 30.8 and record["cache_state"] == "cold"
+
+
+def test_zero_cli_exit_cannot_earn_negative_without_witness():
+    data = payload()
+    data["result"]["coverage_status"] = "complete_within_scope"
+    data["admission"]["coverage_reasons"] = []
+    identity = {"unit": "repository", "language": "php", "path": "api.php", "function": "handler", "family": "injection"}
+    data["scans"] = [
+        {
+            "side": "head",
+            "scan": {
+                "scope": {"ranking_mode": "evidence", "selected": [{"identity": identity}], "deferred": []},
+                "question_outcomes": [{"identity": identity, "status": "completed", "raw_rows_json": "[]"}],
+                "findings": [],
+                "build_seconds": 1,
+                "query_seconds": 2,
+            },
+        }
+    ]
+    record = measure.execution_record("case", data, elapsed=4)
+    assert record["queries"][0]["outcome"] == "unknown"
+    assert "witness" not in record["queries"][0]
+    assert record["alerts"] == []