unseen-eval: sourcing memory -- compact journal records, bounded caches, streamed history (4.6 GB -> 107 MB RSS)

The sourcing run was killed four times (exit 143/137) after growing to 4.6 GB
RSS on the 7.7 GB VM. Live after the fix: flat 107 MB through 3,029
advisories. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

.github/workflows/docs.yml

@@ -0,0 +1,43 @@
+name: Docs
+
+# Builds the MkDocs site and publishes it to GitHub Pages on every push to main that touches it.
+on:
+  push:
+    branches: [main]
+    paths: ["docs/**", "mkdocs.yml", "README.md", ".github/workflows/docs.yml"]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+      - uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+      - uses: actions/setup-python@v6
+        with:
+          python-version-file: .python-version
+      - run: uv sync --extra docs
+      - run: uv run mkdocs build --strict
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: site
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - id: deployment
+        uses: actions/deploy-pages@v4

.github/workflows/engine-image.yml

@@ -0,0 +1,69 @@
+name: Engine image
+on:
+  workflow_dispatch:
+  push:
+    branches: [main]
+    paths:
+      - 'src/**'
+      - 'benchmarks/learn/engine_trace*.py'
+      - 'benchmarks/learn/engine_task_entry.py'
+      - 'benchmarks/unseen/replay_entry.py'
+      - 'benchmarks/learn/engine_worker_service.py'
+      - 'Dockerfile'
+      - 'plane/Dockerfile.engine-task'
+      - 'ops/frontend-retention/install.py'
+      - 'pyproject.toml'
+      - '.github/workflows/engine-image.yml'
+permissions:
+  contents: read
+  packages: write
+concurrency:
+  group: engine-image-${{ github.ref }}
+  cancel-in-progress: false
+jobs:
+  image:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - package: openultrasast
+            dockerfile: Dockerfile
+          - package: ousast-engine-task
+            dockerfile: plane/Dockerfile.engine-task
+            target: engine
+          - package: ousast-replay-task
+            dockerfile: plane/Dockerfile.engine-task
+            target: replay
+    steps:
+      - uses: actions/checkout@v5
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - id: names
+        env:
+          OWNER: ${{ github.repository_owner }}
+          PACKAGE: ${{ matrix.package }}
+        run: |
+          echo "image=ghcr.io/${OWNER,,}/$PACKAGE" >> "$GITHUB_OUTPUT"
+          echo "sha=sha-${GITHUB_SHA:0:7}" >> "$GITHUB_OUTPUT"
+      - id: build
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: ${{ matrix.dockerfile }}
+          target: ${{ matrix.target }}
+          platforms: linux/amd64
+          push: true
+          tags: |
+            ${{ steps.names.outputs.image }}:${{ steps.names.outputs.sha }}
+            ${{ steps.names.outputs.image }}:main
+      - env:
+          IMAGE: ${{ steps.names.outputs.image }}
+          DIGEST: ${{ steps.build.outputs.digest }}
+        run: |
+          echo "$IMAGE@$DIGEST"
+          echo "$IMAGE@$DIGEST" >> "$GITHUB_STEP_SUMMARY"

.gitignore

@@ -36,3 +36,6 @@ workspace/
 
 # MkDocs build output (mkdocs build)
 site/
+
+# Unseen evaluation pointers for non-permissive repositories.
+benchmarks/unseen/private/

.kiro/specs/learned-decision-engine/fix-mechanisms-plan.md

@@ -0,0 +1,80 @@
+# Plan: learn fix mechanisms into the engine (draft for review, 2026-10-04)
+
+Status: approved 2026-10-04 by the maintainer. Increment 1 = discharge facts only; coverage additions (Req 7) come later.
+Decisions: minimum 2 proposing repositories before a candidate is tried; admission per language (not pooled by callee name).
+
+## Why
+
+The Joern trace run over exp-005's 501 units shows the engine cannot tell a vulnerable function from its
+fix. In the 105 pairs where both sides were analysed, a taint path appears on both sides in 12, on the
+vulnerable side only in 2, on the fixed side only in 0, and on neither in 91
+(`benchmarks/measurements/2026-10-04-joern-slice-leak-audit/record.json`).
+
+So the fix is invisible to the engine. Either the flow survives and the engine does not know the fix's
+mechanism (an `allowed_classes` argument, a path normalisation check, an escaping call), or the engine's
+sources and sinks miss the function on both sides. A classifier fed this evidence cannot improve on it.
+
+The corpus now holds 710 complete fix pairs (`2026-10-04-decision-engine-harvest-3/record.json`). Each fix
+diff shows what was added. That is the material the engine lacks.
+
+## Goal
+
+Grow the engine's discharge knowledge (sanitizers, guards, safe-argument forms) and, second, its source and
+sink coverage, from the fix diffs. Admit a fact only when it improves held-out separation. Spend no model
+money.
+
+## Requirements (proposed)
+
+1. **Mining.** From each fix pair, extract what the fixed function adds on the vulnerable flow: new calls,
+   new conditions guarding the sink, new or changed sink arguments. Record each as a candidate fact with its
+   language, family, form (sanitizer call, guard predicate, safe argument, sink restriction) and the pairs
+   that propose it. Mining is deterministic (tree-sitter diff of the two function bodies). No model calls.
+2. **Generalisation is explicit.** A candidate is a pattern, not a repository name: a callee name, an
+   argument key, a predicate shape. Candidates naming project-specific identifiers are rejected (Req 8: no
+   application-specific trees).
+3. **Admission by held-out separation.** Folds are by repository, as in the decision engine. A candidate is
+   admitted for a fold only if, on that fold's held-out pairs, adding it to the fact tables:
+   - increases pairs where the vulnerable side keeps a path and the fixed side loses it or becomes
+     sanitized;
+   - and does not remove any held-out vulnerable-side path (no recall loss).
+   A candidate proposed by fewer than 2 distinct repositories in the training folds is not tried.
+4. **No train-on-test.** Mining and admission never read held-out pairs; the closed-loop leak recorded in
+   memory (`closed-loop-train-on-test-leak`) is the failure this rule exists to prevent. Population v3 is
+   never read.
+5. **Measurement by the trace runner.** Each admission round re-runs the engine on the affected pins only
+   (the runner's resume and selection already support this; on k3s the worker Deployment runs them in
+   parallel). The outcome is the pair-asymmetry table per family, before and after, on held-out folds.
+6. **Gate before production.** An admitted fact enters `src/openultrasast/ruleset/semantic/*.toml` only after
+   the detection gates (`gate`, `map_gate`, `pair_gate`) show no regression, and only as a registered
+   experiment whose decision rule was fixed before the run.
+7. **Coverage second.** After discharge facts, the same loop proposes source and sink additions from pairs
+   where neither side has a path (91 of 105 today), admitted under the same held-out rule.
+
+## How it relates to what exists
+
+- `corpus-seeded-mechanisms` already has a closed loop that admits and retracts mechanisms by leave-one-out.
+  This plan reuses its lever and store, and points it at discharge facts mined from real fix diffs instead
+  of variant search.
+- `benchmarks/pairs/mechanisms.toml` (23 mechanisms) labels what kind of bug a pair is. The new candidates
+  describe what a fix does. They live in the fact tables, not in the mechanism vocabulary.
+
+## Measures of success
+
+- Pair asymmetry on held-out folds: vulnerable-only paths rise from 2 of 105 toward a substantial share,
+  with zero fixed-only paths.
+- No held-out recall loss on vulnerable sides.
+- Then the decision engine is re-run with the new engine evidence (exp-005 design), and the pooled block gate
+  (exp-004 design) is re-measured.
+
+## Cost and time
+
+- Model cost: $0 (mining and admission are deterministic; the classifier re-run afterwards is a separate,
+  approved spend).
+- Engine time: one pass over affected pins per admission round; about 2 hours per full pass on k3s plus the
+  VM, about 6 hours on the VM alone.
+
+## Open questions for the maintainer
+
+1. Should coverage additions (Req 7) be in the same increment or a later one?
+2. Minimum proposing repositories: 2 is the proposal. Higher is safer, lower finds more.
+3. Admission per family and language, or pooled across languages for the same callee name?

.kiro/specs/learned-decision-engine/requirements.md

@@ -119,6 +119,19 @@ rule hits.
 2. Without a trained model available the tools fall back to today's behaviour and say so in the report.
 3. Hand-written rules remain as signal producers; changing their enabled/shadow status no longer changes what is
    reported unless the engine's decision changes.
+4. *(Amended 2026-10-03, maintainer decision; refines the BLOCK rule of 7.3 and 7.4. Tested by exp-004.)* BLOCK
+   precision is demonstrated pooled across families, not per family:
+   - **Normalisation**, declared before any run: each family's raw score is put on one scale by a per-family Platt
+     map (`learn.calibrate.fit_map` with its floors) fitted on training folds only. A family the training folds
+     cannot map does not block on that fold.
+   - **t_block**: one threshold on the normalised score for all families, chosen on training folds as the lowest
+     one whose Wilson 95% lower bound on precision, over the training flags of all families together, is >= 0.95.
+     It is reported by the Wilson 95% lower bound over all held-out flags of all families, before the floor.
+   - **Family floor**: a family may block only if its own held-out flags have zero errors and number at least F
+     (F = 10, reasoned in the exp-004 registration); otherwise that family stays ADVISORY.
+   - **t_quiet**: per family, the highest threshold keeping >= 90% of the paired positives at or above it (chosen
+     on training folds, its held-out recall reported); findings below it are silent.
+   - No held-out label moves a threshold or a normalisation map; candidates whose verdict is `unsure` never block.
 
 ### Requirement 7: Built for unseen repositories -- a repo safety net
 
@@ -136,7 +149,8 @@ rule hits.
    ADVISORY, trading precision for recall; both reported with out-of-repository intervals. The decision unit is a
    candidate in changed code (`pre-push` delta), with whole-repository scans as a secondary mode.
 4. Calibration is checked on held-out repositories; where it does not hold, only ADVISORY is offered for that
-   family.
+   family. *(Amended 2026-10-03: once exp-004 adopts the pooled gate of 6.4, BLOCK precision is shown by its held-out
+   bound and family floor, and calibration is reported, not required.)*
 5. Any adaptation to a user's own repository (facts reuse, their dismissals) is opt-in, separate from the
    generalisation model, and never contributes to its reported numbers.
 

.kiro/specs/learned-decision-engine/spec.json

@@ -1,7 +1,7 @@
 {
   "feature_name": "learned-decision-engine",
   "created_at": "2026-09-30T14:00:00Z",
-  "updated_at": "2026-10-01T10:20:00Z",
+  "updated_at": "2026-10-04T18:00:00Z",
   "language": "en",
   "phase": "implementation",
   "approvals": {

.kiro/specs/learned-decision-engine/tasks.md

@@ -8,6 +8,10 @@ memory, compiled DSPy-style in-house); tasks 1-5 are unaffected, tasks 6-10 are
 against the provider balance read just before it, and none starts with a ceiling above that balance. No test makes
 a model or embedding call.
 
+Direction 2026-10-04: `.kiro/steering/roadmap.md` (section of 2026-10-04, phases G0-G5) is the authority. Every
+change is judged on the unseen-repository pool (G0, spec `unseen-repo-evaluation`) by catch rate and false-alarm
+rate per push. Within-pair AUC stays a development signal. Tasks 11-14 below follow that roadmap.
+
 - [x] 1. Schema and features
   - `learn/schema.py` (closed, typed allow-list; no identities; missing != zero), `learn/features.py` (host
     builder), new memory `KINDS`, `plane/tasks/features.py` and the `remember` input; tests enforcing the
@@ -45,6 +49,11 @@ a model or embedding call.
   - _Requirements: 1.1_
 
 - [ ] 6. Program, local memory and compilation (design sections 4-5)
+  - Status 2026-10-04: 6.1-6.6 are done. 6.7 is partial: leave-one-source-out, the 0/25/50% memory-size points and
+    the source-count curve are not run. Around this task the experiment machinery grew: rule arms
+    (`precision_bound_block`, `pooled_block_gate` with a family floor) in `learn/experiments.py`, the response-cache
+    fill with a hard ceiling and a dry-run estimate in `learn/fill.py`, and the in-function slice in `learn/slice.py`.
+    These ran exp-003, exp-004 and exp-005 (7.5-7.7). Memory now holds 710 complete fix pairs (7.9).
   - [x] 6.1 Excerpts and the example store (no model)
     - `learn/excerpt.py` (function span at the label's pin, bounds 80/4,000 for candidates and 40/2,000 for
       examples, numbered lines, no path/repository/commit, advisory ids and security wording in comments
@@ -151,6 +160,28 @@ a model or embedding call.
     rows' null plane-only features (1f525ff), the 788 embedding vectors absent from RustFS (copied from the file
     store), the result row's column-type collision under S3 Select (780caaa). Further program variants (`k = 5` vs
     `k = 1`, the confidence source) as new manifests.
+  - [x] 7.5 exp-003 run and recorded 2026-10-03 ($0, replay only): **inconclusive**. Selective blocking (a per-family
+    raw-score threshold chosen on training folds by its Wilson lower bound) offers no BLOCK on held-out paired units
+    in any family; the incumbent stays. 211 of 353 frozen units evaluated (142 replay misses). Proving 0.95 needs at
+    least 73 clean flags. Result: `benchmarks/experiments/exp-003-precision-bound-blocking/result.json`.
+  - [x] 7.6 exp-004 run and recorded 2026-10-03: **inconclusive**. One pooled BLOCK threshold over all families with a
+    family floor (Req 6.4). No outer fold found a threshold, pooled or per family; B offers no BLOCK. 501 units,
+    replay at $0 after a cache fill of $1.24 (ceiling $2; the dry-run estimate over-priced 4.5x). Result and record:
+    `benchmarks/experiments/exp-004-pooled-block-gate/result.json`, `record.json`.
+  - [x] 7.7 exp-005 run and recorded 2026-10-03: **inconclusive**. The in-function def-use slice in the prompt of the
+    six slice-2 programs. Within-pair AUC 0.762 -> 0.746 (CI [-0.063, 0.032], spans 0); unsure rate 0.29 -> 0.24
+    (favours B); usd per candidate slightly higher. A $0 leak audit came first
+    (`benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json`). Result:
+    `benchmarks/experiments/exp-005-graph-slice/result.json`.
+  - [x] 7.8 Negatives harvest 2026-10-03 ($0.0014 embeddings, no chat-model call): 98 real fix pairs whose function
+    survives the fix, from reviewed advisories, as pointer-pair corpus `advisory-fixes-2`; memory 937 -> 1133
+    examples. Record: `benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest/record.json`.
+  - [x] 7.9 Harvest 3, 2026-10-04 ($0.003 embeddings, no chat-model call): 251 pairs as `advisory-fixes-3`; complete
+    pairs in memory 459 -> 710. Record: `benchmarks/measurements/2026-10-04-decision-engine-harvest-3/record.json`.
+  - [x] 7.10 Joern trace run over exp-005's 501 units and its leak audit, 2026-10-04 ($0): no leak flagged. In the 105
+    pairs where both sides were asked, a path appears on both sides in 12, on the vulnerable side only in 2, on the
+    fixed side only in 0, and on neither in 91. The engine cannot see what a fix changed. Record:
+    `benchmarks/measurements/2026-10-04-joern-slice-leak-audit/record.json`.
   - _Requirements: 3.6, 4.1-4.4_
 
 - [ ] 8. Integration
@@ -176,3 +207,50 @@ a model or embedding call.
     `result-v3-engine.json` is committed. Nothing between the two commits may touch the program, memory snapshot,
     features, priors or thresholds. This spec's code never opens v3 files.
   - _Requirements: 5.3_
+
+- [ ] 11. Fix mechanisms (plan approved 2026-10-04; see `fix-mechanisms-plan.md`)
+  - Roadmap G2 arm (`.kiro/steering/roadmap.md`, section of 2026-10-04): one candidate improvement, not the spine.
+    It is judged on the unseen-repository pool (G0, spec `unseen-repo-evaluation`) by catch rate and false-alarm
+    rate per push. **Paused after 11.1 until the G0 pool exists.**
+  - [ ] 11.1 Mine candidate discharge facts from the 710 fix diffs (sanitizer calls, guarding conditions, safe sink
+    arguments, safe replacements); deterministic, $0. In progress.
+  - [ ] 11.2 Admission by held-out separation per outer fold: a candidate turns "path on both sides" into "path on the
+    vulnerable side only" on folds it was not mined from, removes no held-out vulnerable path, has at least 2
+    proposing repositories, and is admitted per language.
+  - [ ] 11.3 Engine re-run on the affected pins (kube-ax lanes plus the VM lane).
+  - [ ] 11.4 Registered experiment, with the detection gates green, before `ruleset/semantic/*.toml` changes.
+  - [ ] 11.5 Coverage increment: source and sink additions for pairs where neither side has a path (later).
+  - [ ] 11.6 TypeScript admission in the engine (branch `engine-typescript`).
+  - _Requirements: 4.1-4.4, 7.1, 8.1_
+
+- [ ] 12. exp-006: Joern witness slice in the prompt **(budget go-ahead, about $2-4)**
+  - Planned, roadmap G1/G2. Against the current prompt, after a $0 leak audit, on re-frozen units. Judged on the
+    unseen-repository pool, not on within-pair AUC alone.
+  - _Requirements: 4.1-4.4, 7.1_
+
+- [ ] 13. exp-007: the combiner ($0 replay)
+  - Planned, roadmap G1/G2. Three or four weights (score, engine entailment, neighbour agreement) over cached
+    responses. Judged on the unseen-repository pool.
+  - _Requirements: 4.1-4.4, 7.1_
+
+- [ ] 14. Re-run the pooled block gate on re-frozen units
+  - Planned, roadmap G3. The exp-004 design with the family floor (Req 6.4), together with the pool's false-alarm
+    rate per family.
+  - _Requirements: 6.4, 7.3_
+
+- [ ] 15. Trained classifier arm (maintainer decision 2026-10-04: train a model, not hand-set profiles; judged on the
+  unseen-repo pool, roadmap G1)
+  - [ ] 15.1 Features: code embedding of the excerpt (stored vectors) + every allow-listed signal; no hand-picked
+    profile. Automated screening: a signal that separates pair sides by itself on training folds above the leak
+    threshold is dropped for that fold (the leak audit as a filter, fitted per fold).
+  - [ ] 15.2 Models: regularised logistic regression and gradient-boosted trees, fitted per repository-held-out outer
+    fold with inner-fold hyper-parameter search; per-family and pooled variants.
+  - [ ] 15.3 Scores on held-out folds: within-pair AUC, pooled AUC, top-k precision, Wilson bound at the pooled
+    threshold; compared with the current LLM programs on the same units.
+  - [ ] 15.4 Scored on the unseen-repo pool when G0 exists: catch rate at fixed false-alarm budgets.
+  - _Requirements: 3 (revised 2026-10-04 by maintainer decision), 6, 7_
+
+- [ ] 16. Jointly optimised LLM program arm (judged on the unseen-repo pool)
+  - Search input fields, retrieval parameters, k, instruction and demonstrations together (DSPy MIPRO-style) on
+    training folds; no hand-set input profile. Budget go-ahead before paid search.
+  - _Requirements: 3, 6, 7_

.kiro/specs/plane-on-kubernetes/design.md

@@ -1,5 +1,20 @@
 # Design Document
 
+> **Status 2026-10-04.** Groups 1-2 are done on kind (records `benchmarks/measurements/2026-10-02-k8s-01-profile/`
+> and `2026-10-02-k8s-02-store-delivery/`). Task 2.6 is paused. The production target changed. The operator runs a
+> kube-ax cluster (k3s, Agent Substrate v0.3.0, AX v0.3.1, 2 nodes). There we may only submit AX Tasks: no k8s
+> object creation, no secrets, no access to `ate-system`. The engine now runs there as AX Tasks through
+> `benchmarks/learn/engine_trace_ax.py`. The engine-task image (`plane/Dockerfile.engine-task`) is built FROM
+> `ghcr.io/norandom/ax-task-runner:v0.3.1`. Tasks read and write through presigned HTTPS links, are named
+> `ousast-engine-*`, and run a 1.7 GB heap with `ActiveProcessorCount=2` and one resident Joern session per pin. A
+> pin whose sandbox dies is retried once under a new name, then handed to the VM lane (2.5 GB heap). Egress is
+> operator-managed: HTTPS 443 to public hosts is open; LAN and cluster-internal addresses are blocked. Archive
+> names (`.tar`, `.gz`, `.zip`, `.bin`) failed through the store host's Cloudflare cache until the operator added a
+> cache-bypass rule; presigned objects are staged as `.dat` as a safeguard. Groups 3-8 are **to re-plan** against
+> that cluster under roadmap G4 (`.kiro/steering/roadmap.md`, section of 2026-10-04). Their text below is kept
+> as written. G4 (detection on push for any repository: `ousast plane scan --base --head`, engine pins on kube-ax,
+> model calls on the VM or CI) is the new target for group 7, sized by what G1-G3 keep.
+
 ## Overview
 
 The plane keeps its shape: `ousast plane run` is the reconciler, ax is the only executor, every task is one
@@ -254,6 +269,13 @@ containerd; `Dockerfile:1-7` already argues against it).
   a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a
   separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays
   `alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such.
+  **Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.**
+  With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor
+  requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS
+  1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so
+  group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether
+  Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the
+  k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path.
 - **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and
   `docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose
   `engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`.

.kiro/specs/plane-on-kubernetes/spec.json

@@ -1,7 +1,7 @@
 {
   "feature_name": "plane-on-kubernetes",
   "created_at": "2026-10-02T20:00:00Z",
-  "updated_at": "2026-10-02T22:00:00Z",
+  "updated_at": "2026-10-04T18:00:00Z",
   "language": "en",
   "phase": "implementation",
   "approvals": {

.kiro/specs/plane-on-kubernetes/tasks.md

@@ -1,5 +1,20 @@
 # Implementation Plan
 
+> **Status 2026-10-04.** Groups 1-2 are done on kind (records `benchmarks/measurements/2026-10-02-k8s-01-profile/`
+> and `2026-10-02-k8s-02-store-delivery/`). Task 2.6 is paused. The production target changed. The operator runs a
+> kube-ax cluster (k3s, Agent Substrate v0.3.0, AX v0.3.1, 2 nodes). There we may only submit AX Tasks: no k8s
+> object creation, no secrets, no access to `ate-system`. The engine now runs there as AX Tasks through
+> `benchmarks/learn/engine_trace_ax.py`. The engine-task image (`plane/Dockerfile.engine-task`) is built FROM
+> `ghcr.io/norandom/ax-task-runner:v0.3.1`. Tasks read and write through presigned HTTPS links, are named
+> `ousast-engine-*`, and run a 1.7 GB heap with `ActiveProcessorCount=2` and one resident Joern session per pin. A
+> pin whose sandbox dies is retried once under a new name, then handed to the VM lane (2.5 GB heap). Egress is
+> operator-managed: HTTPS 443 to public hosts is open; LAN and cluster-internal addresses are blocked. Archive
+> names (`.tar`, `.gz`, `.zip`, `.bin`) failed through the store host's Cloudflare cache until the operator added a
+> cache-bypass rule; presigned objects are staged as `.dat` as a safeguard. Groups 3-8 are **to re-plan** against
+> that cluster under roadmap G4 (`.kiro/steering/roadmap.md`, section of 2026-10-04). Their text below is kept
+> as written. G4 (detection on push for any repository: `ousast plane scan --base --head`, engine pins on kube-ax,
+> model calls on the VM or CI) is the new target for group 7, sized by what G1-G3 keep.
+
 One task group per increment of `design.md` section 7, in that order; a later group never starts before the
 earlier group's kind proof is recorded. Every task ships with its tests; a task is done when its tests pass in
 the host suite, ruff and mypy are clean, and nothing in `src/`, `plane/`, `ops/` (minus `ops/ax/up.sh` and
@@ -9,8 +24,8 @@ its kind proof and the record it writes under `benchmarks/measurements/<date>-k8
 (commit, images, the instrument checks, the numbers). Steps that call a model are marked **paid: needs a budget
 go-ahead** with a ceiling; steps that need a hand on a cluster that does not exist yet are marked **manual**.
 
-- [ ] 1. Profile and literals (design section 1; open-question checks 2, 3, 7)
-- [ ] 1.1 `PlaneProfile`
+- [x] 1. Profile and literals (design section 1; open-question checks 2, 3, 7)
+- [x] 1.1 `PlaneProfile`
   - `src/openultrasast/plane/profile.py`: frozen dataclass with the fields of design section 1; `load_profile()`
     reads `ops/k8s/profiles/<OUSAST_PLANE_PROFILE>.toml` (a name or a path; default `kind`), then applies
     `OUSAST_<FIELD>` overrides; rejects an unknown key, a secret-looking key (`*_KEY`, `*_SECRET`), an `exec`
@@ -19,7 +34,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
   - `tests/test_plane_profile.py`: defaults; file then env precedence; each rejection names the field; both
     committed profiles load; no field holds a value matching a credential pattern.
   - _Requirements: 1.1_
-- [ ] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
+- [x] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
   - `doctor(profile)`: context from `profile.kube_context`; the registry check resolves the `images` file's runner
     reference (`docker manifest inspect` for a remote registry, `/v2/_catalog` for a local one); `open_store`
     verifies `profile.memory`; prints every configured address (Req 1.2). `Egress(context=...)`,
@@ -29,46 +44,46 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
     profile's context, never a built one.
   - Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile.
   - _Requirements: 1.1, 1.2_
-- [ ] 1.3 Host-literal scan test
+- [x] 1.3 Host-literal scan test
   - `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does)
     over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`,
     `localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged
     `ax-tunnel`; fails listing `path:line`.
   - Evidence: the test passes on this commit and fails when a literal is reinserted (shown once in the record).
   - _Requirements: 1.3_
-- [ ] 1.4 `up.sh` by profile
+- [x] 1.4 `up.sh` by profile
   - `ops/ax/up.sh`: `KO_DOCKER_REPO` and the context from `kind.toml` (read with `python -m
     openultrasast.plane.profile --print registry|kube_context`); writes `kind-images.json` instead of
     `runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3).
   - Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes.
   - _Requirements: 1.1, 2.4_
-- [ ] 1.5 Placement probe (open question 2, decided before the engine work)
+- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any`
   - `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's
     engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records
     with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after.
   - Evidence: `record.json` says whether Substrate places by resource fit (`placement: fit|any`). `any` selects
     the design's fallback for group 5 and is written into `design.md` section 4 before group 5 starts.
   - _Requirements: 4.1_
-- [ ] 1.6 Reconciler bound (open question 7)
+- [x] 1.6 Reconciler bound (open question 7) -- 489 lines after 1.2; projected 514 during group 2: the bound moves to 550 in 2.4
   - Record `wc -l src/openultrasast/plane/reconciler.py` (477 at 17ac7ad) and the lines task 2.4 may add
     (receiver wiring removed: `-12`, `Delivery` calls: `+8` estimated). If the estimate crosses 500, the bound
     in `tests/test_plane_reconciler.py:803` moves to 550 in task 2.4 with the reason in the assertion message.
   - Evidence: the numbers in `record.json`.
   - _Requirements: 2.1_
-- [ ] 1.7 gvisor tarball reachability (open question 3) **manual**
+- [x] 1.7 gvisor tarball reachability (open question 3) **manual** -- documented as planned in `docs/deployment.md` B.1; the live check waits for k3s
   - `docs/deployment.md` (B.1, planned): the k3s node must fetch
     `gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd` (`substrate/manifests/ate-install/sandboxconfig-gvisor.yaml`)
     or a mirrored copy named in a copied SandboxConfig; the check is `kubectl -n ate-system logs` of the worker
     pod's atelet fetch line, run by hand on the day k3s exists. Labelled planned until then.
   - _Requirements: 7.2_
-- [ ] 1.8 Kind proof and record 01
+- [x] 1.8 Kind proof and record 01
   - `ousast plane doctor --profile kind` passes every line; `ousast plane doctor --profile k3s` fails only on the
     unreachable context (the expected line); `test_plane_literals` 0 offenders; the probe of 1.5; the bound of 1.6.
   - `benchmarks/measurements/<date>-k8s-01-profile/record.json`.
   - _Requirements: 1.1, 1.2, 1.3, 7.1_
 
 - [ ] 2. Store delivery, receiver-less egress (design section 3)
-- [ ] 2.1 Presigned URLs and the store run directory
+- [x] 2.1 Presigned URLs and the store run directory
   - `src/openultrasast/plane/delivery.py`: `Delivery(store, run)` mints presigned PUT for
     `runs/<run>/<task>/output.tar` and GET for `runs/<run>/<producer>/<artifact>` (boto3 `generate_presigned_url`
     on `S3Client.sdk`, expiry `OUSAST_TASK_TIMEOUT + 600`); `delivered(task)` HEADs the tar; `collect(task)`
@@ -78,7 +93,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
   - `tests/test_plane_delivery.py`: against a local S3-shaped fake (PUT/GET/HEAD by key); URLs never appear in
     logs; expiry; member rejection; declared-only re-put; state seeding.
   - _Requirements: 3.1, 2.1_
-- [ ] 2.2 Runner: `delivery` in the start body
+- [x] 2.2 Runner: `delivery` in the start body
   - `src/openultrasast/plane/runner.py`: `StartGate.offer` validates an optional `delivery {put, inputs}` (https,
     hostname, no echo); `_deliver_output` PUTs the tar to `delivery.put` (same retry and `DeliveryError`
     semantics); `fetch_inputs` GETs from `delivery.inputs[name]`; `OUSAST_ARTIFACT_URL` and `OUSAST_ARTIFACT_DIAL`
@@ -86,14 +101,14 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
   - `tests/test_plane_runner.py`: accepted body with `delivery`; 400 on an `http` or IP URL; PUT to a local
     server; inputs fetched from presigned-shaped URLs; the marker and retry-only boot unchanged.
   - _Requirements: 3.1_
-- [ ] 2.3 Egress without the receiver
+- [x] 2.3 Egress without the receiver
   - `src/openultrasast/plane/egress.py`: `policy_for(task, workspaces, model, store_host)` emits one
     `tlsPassthrough` rule on 443 over the sorted union of store host, Git hosts, Model hosts; no `http` rule; an
     IP store host raises naming `S3_ENDPOINT`.
   - `tests/test_plane_egress.py`: for every committed template under `plane/tasks/` the rendered policy is the
     expected set; the IP refusal.
   - _Requirements: 3.2_
-- [ ] 2.4 Reconciler on `Delivery`
+- [x] 2.4 Reconciler on `Delivery`
   - `src/openultrasast/plane/reconciler.py`: `run()` opens the store first (no fallback), builds `Delivery`,
     passes `delivery` to `router.starter`/`start_task` (body extended in `router.py`), `_await` on
     `Delivery.delivered`, outcome from the collected `summary.json`; the receiver thread is kept behind
@@ -101,13 +116,13 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
   - `tests/test_plane_reconciler.py`: the ax fake run completes through the delivery fake; a rerun from an empty
     results root resumes from the store's `state.json`.
   - _Requirements: 2.1, 3.1_
-- [ ] 2.5 Kind proof and record 02
+- [x] 2.5 Kind proof and record 02
   - `ops/ax/smoke-run.sh` with `OUSAST_DELIVERY=store`: `facts.json` and `summary.json` arrive through the store;
     the actor's policy read back has one TLS rule naming the store host; `ousast plane status ax-e2e-smoke`
     prints the table. Instrument: the tar's byte size from the store equals the extracted files' total.
   - `benchmarks/measurements/<date>-k8s-02-store-delivery/record.json`: bytes, PUT-to-poll seconds, rules.
   - _Requirements: 3.1, 3.2, 7.1_
-- [ ] 2.6 Remove the receiver (last; after 2.5 is recorded)
+- [ ] 2.6 Remove the receiver (last; after 2.5 is recorded) -- paused 2026-10-04
   - Delete `Receiver`, `_Handler`, `receiver_address`, `receiver_cluster_ip`, `RECEIVER_HOST`, `RECEIVER_PORT`
     from `egress.py`; `OUSAST_ARTIFACT_URL/DIAL/HOST/PORT` from `runner.py` and `reconciler.py`; the
     `OUSAST_DELIVERY` switch; `ops/ax/receiver-service.yaml.tmpl` and the `up.sh` receiver step; the EndpointSlice
@@ -117,7 +132,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
     `receiver_removed: <commit>`.
   - _Requirements: 3.1, 1.3_
 
-- [ ] 3. `--exec remote` from a CI-shaped principal (design section 2; open-question check 1)
+- [ ] 3. `--exec remote` from a CI-shaped principal (design section 2; open-question check 1) -- to re-plan (kube-ax, roadmap G4)
 - [ ] 3.1 Execution mode and RBAC manifests
   - `cli.py`: `ousast plane run --exec local|remote` (default `profile.exec`); `reconciler.Ax` passes the
     profile's context to `ax` (its context flag, read from `ax --help` and pinned in a test); `remote` refuses a
@@ -150,7 +165,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
     status, usd spent.
   - _Requirements: 2.1, 2.2, 2.3, 2.4, 7.1_
 
-- [ ] 4. Images on GHCR (design section 8)
+- [ ] 4. Images on GHCR (design section 8) -- to re-plan (kube-ax, roadmap G4)
 - [ ] 4.1 `images.yml`
   - `.github/workflows/images.yml` on tags `v*`: `docker/build-push-action` for `plane/Dockerfile.runner` ->
     `ghcr.io/norandom/ousast-runner:<tag>` and the root `Dockerfile` -> `ghcr.io/norandom/ousast-engine:<tag>`;
@@ -178,7 +193,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
   - `benchmarks/measurements/<date>-k8s-04-ghcr-images/record.json`: image sizes, pull seconds, visibility.
   - _Requirements: 1.1, 4.2, 7.1_
 
-- [ ] 5. The engine as a plane task (design section 4)
+- [ ] 5. The engine as a plane task (design section 4) -- to re-plan (kube-ax, roadmap G4)
 - [ ] 5.1 `cpg/dump.py` and `tasks/engine.py`
   - Lift `benchmarks/push/finding_dump.py`'s scan into `src/openultrasast/cpg/dump.py` (the script becomes a
     thin caller; `ops/README.md:78` command unchanged). `src/openultrasast/plane/tasks/engine.py`: env
@@ -208,7 +223,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
     placement, image size and pull seconds.
   - _Requirements: 4.1, 4.2, 7.1_
 
-- [ ] 6. Pools, autoscaling, node checklist (design section 5)
+- [ ] 6. Pools, autoscaling, node checklist (design section 5) -- to re-plan (kube-ax, roadmap G4)
 - [ ] 6.1 `plane/k8s.py` and the manifests
   - `src/openultrasast/plane/k8s.py`: `render(profile, out)` writes `workerpool-default.yaml`,
     `workerpool-engine.yaml`, `ax-server-snapshots.yaml` (patch with `profile.snapshots_bucket`), `rbac-ci.yaml`;
@@ -232,7 +247,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
   - `benchmarks/measurements/<date>-k8s-06-pools/record.json`: footprint table, securityContext, dry-run result.
   - _Requirements: 5.1, 5.2, 7.1_
 
-- [ ] 7. `ousast plane scan` (design section 6)
+- [ ] 7. `ousast plane scan` (design section 6) -- to re-plan (kube-ax, roadmap G4)
 - [ ] 7.1 Workspaces and candidates from URL + commits
   - `generate.scan_cases(url, base, head, cache, *, all=False)`: shallow clone into the case cache, two pinned
     Workspaces, head-side hunks via `fix_ranges(side="new")`, functions via `repo_facts.enclosing` over
@@ -265,7 +280,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
     seconds from submit to verdict, whether the known CVE site is among the agreed.
   - _Requirements: 6.1, 6.2, 6.3, 7.1_
 
-- [ ] 8. Runbook (design section 7, Req 7)
+- [ ] 8. Runbook (design section 7, Req 7) -- to re-plan (kube-ax, roadmap G4)
 - [ ] 8.1 `docs/deployment.md` as the runbook
   - Sections: prerequisites for k3s (Substrate, ax with the snapshots patch, GHCR and the pull secret, buckets,
     the CI SA and its grant from 3.2, the node checklist of 6.2), the profile switch, `--exec remote` from a laptop

.kiro/specs/pre-push-safety-net/requirements.md

@@ -147,6 +147,43 @@ with Express-style routes and middleware, including vulnerable, fixed and benign
 frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or
 unsupported-property results separately from demonstrated vulnerability detection.
 
+## Requirement 9: Operate as an advisory safety net on any repository
+
+Added 2026-10-02, maintainer-approved, from the operational-readiness audit. The maintainer's
+definition: a user installs the hook on ANY repository and gets honest, advisory-only results;
+nothing ever blocks a push unless blocking was explicitly chosen.
+
+9.1 (added 2026-10-02, maintainer-approved) When the installed hook cannot run the check, because the
+`ousast` executable is missing or an argument or setting is invalid, the hook shall allow the push and
+print one line saying the check did not run and how to fix it; only an explicit `--mode blocking`
+opt-in may turn an analysis result into a rejected push.
+
+9.2 (added 2026-10-02, maintainer-approved) When a check is skipped or incomplete, the terminal output
+shall name each distinct reason in plain words, one line per reason, with repository paths decoded and
+no opaque hash tokens; a language with no engine frontend and no quick rules shall be named with its
+file count; a first push of a new branch shall say there was no base to compare against and what was
+checked instead. The JSON artifact keeps the raw reason values.
+
+9.3 (added 2026-10-02, maintainer-approved) Before the engine, within the same deadline, the Safety Net
+shall run the shipped quick pattern rules over the head side of the changed lines and show their
+findings as advisory pattern matches, labeled separately from admitted alerts. This amends 3.2 for this
+tier only: quick-rule findings are shown, never admitted, and never block.
+
+9.4 (added 2026-10-02, maintainer-approved) When a finding's operation lies in a file that is new in the
+head revision, the comparison shall classify it as new rather than unknown; an engine finding whose
+novelty is established but whose capability is not yet qualified shall be shown as an advisory engine
+finding, not an admitted alert. Genuinely unknown comparisons keep their vetoes.
+
+9.5 (added 2026-10-02, maintainer-approved) When the engine cannot finish within the deadline, one line
+shall say so; on explicit request (`--engine background`) the Safety Net shall start the engine
+detached after the push, record its result in the artifact directory, and show it on the next run as
+the engine result for that head from the previous push.
+
+9.6 (added 2026-10-02, maintainer-approved) The hook installer and remover shall ship in the wheel
+(`ousast pre-push install|uninstall`), refuse to overwrite an existing hook unless forced, and a
+hook-callable container wrapper shall pass the push input and artifact path through; the documentation
+shall give a first-run guide, a glossary of skip reasons, a CI recipe and help text for every flag.
+
 ## Accepted experimental acceptance profile
 
 These values are the initial implementation/evaluation profile accepted for task generation, not

.kiro/specs/pre-push-safety-net/tasks.md

@@ -475,7 +475,7 @@ Preserve all completed task evidence below.
   - _Depends: 13.1_
   - _Requirements: 6.2, 6.3, 8.1, 8.2_
 
-- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
+- [x] 14. Ask every declared family whether it establishes anything (precondition for M4)
   - Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
 - [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
   - Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
@@ -575,7 +575,7 @@ Preserve all completed task evidence below.
   - _Boundary: Evaluation harness, full-run integration_
   - _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
   - _Requirements: 6.2, 6.3, 8.1, 8.2_
-- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
+- [x] 13.4 Let a question inside one partition complete, whatever else the repository holds
   - **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
   - The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
   - **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
@@ -587,6 +587,7 @@ Preserve all completed task evidence below.
   - _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
   - _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
   - _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
+  - _Checked 2026-10-02 per the operational-readiness audit:_ implemented and verified on PMPro (120 of 127, above). 13.3 stays open on its own blocker; this box was left unchecked only because it waits on it.
   - _Depends: 13.3_
   - _Requirements: 2.1, 2.2, 6.3, 8.1_
 
@@ -619,6 +620,55 @@ Preserve all completed task evidence below.
   - _Requirements: 8.1, 8.2_
 
 
+- [x] 17. Operational readiness (2026-10-02)
+  - **Named by the 2026-10-02 operational-readiness audit**, maintainer-approved as the first increment. The maintainer's definition: a user installs the hook on ANY repository and gets honest, advisory-only results; nothing ever blocks a push. The audit found the plumbing sound (snapshots, resolution, deadline accounting, advisory exit 0, the JSON artifact) and five things in the way: the hook can block a push, quick rules are not part of pre-push, the engine cannot finish a 26-file repository inside 30 s, the terminal hides every skip reason, and there is no supported install path without the source checkout. Before numbers for every item are the audit's live runs on records (Python), serve-static (JavaScript) and unrolled/secure (Go), each with one planted commit.
+  - This group is plumbing and honesty. It enables no capability, edits no rule for a planted sample, and leaves graph reuse (the engine speed work decided in 11.6) as a later task.
+  - Record: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (see 17.7).
+  - _Done 2026-10-02:_ all seven subtasks done. The engine still does not fit the 30 s hook deadline (graph reuse, the 11.6 direction, stays a later task), and plane verdicts after the push were not in this increment.
+  - _Requirements: 9.1–9.6_
+- [x] 17.1 Never block a push the user did not choose to block
+  - The installed hook exits 0 with one line when `ousast` is missing (the check did not run, and how to install). In hook (stdin) mode, an argument or usage error (an invalid `OUSAST_PUSH_DEADLINE`, a cache directory inside the repository) is advisory: one line and exit 0. `--mode blocking` stays the explicit opt-in that can reject a push, and keeps its usage errors. The prior-hook input-retention failure keeps exit 1, because it is the chained hook's verdict that cannot be obtained, not ours.
+  - Files: `ops/pre-push`, `src/openultrasast/cli.py`. Tests: `tests/test_push_hook.py` (a push with `ousast` absent from PATH succeeds and prints the line; a bad deadline in hook mode exits 0; blocking mode keeps exit 2).
+  - Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0.
+  - _Done 2026-10-02:_ the hook prints one line and exits 0 (chaining a prior hook if present) when `ousast` (or `OUSAST_COMMAND`) is missing; in hook mode without `--mode blocking`, argparse/usage errors and unexpected exceptions print one line and exit 0. Tests: `test_missing_ousast_never_blocks_the_push`, `test_hook_mode_usage_errors_are_advisory_unless_blocking` (blocking keeps exit 2), and `test_cli_refuses_the_flag_outside_explicit_replay` updated (the experimental flag is still refused in hook mode, now advisory). Gates: 1987 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.1, 5.3, 5.4_
+- [x] 17.2 Name every skipped check in plain words
+  - One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: <lang> (<n> files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`).
+  - Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`.
+  - Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch.
+  - _Done 2026-10-02:_ new `push/explain.py` turns every raw coverage reason into one plain `Skipped:` line per distinct kind (hex paths decoded, sha256 tokens dropped and counted, unknown reasons still printed); the existing one-line `Notice:` keeps its next action and gains an engine-missing variant. `language_not_covered:<lang>:<changed>:<total>` is recorded for a changed language with no engine frontend and no quick rules; `ousast scan` prints `notice: language not covered: go (N files)` and its report says so. `reports.py` renders the eight previously dropped degradations and prints any reason it has no sentence for. The hook passes `OUSAST_COMPARISON_BASE`. The new-branch line names the base it lacks; what was checked instead lands with the quick tier (17.3). Tests: `tests/test_push_explain.py` (6), two in `tests/test_reports.py`; module-audit manifest regenerated for the new module. Gates: 1995 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.2, 5.2, 8.2_
+- [x] 17.3 Run the quick rules as the fast first tier
+  - Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block.
+  - Files: `src/openultrasast/push/quick.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_quick.py` (new).
+  - Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
+  - _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.3_
+- [x] 17.4 Classify a file new in head as new, not unknown
+  - `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
+  - Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
+  - Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
+  - _Done 2026-10-02:_ `ChangeContext.added_paths` records Git status-A paths (rename targets excluded). In `_compare_evidence`, after every existing context/head/deadline check, an operation in an added file is `new`/`file_added_in_head` when the claim is entailed, undischarged and has change evidence, and no base answer holds the same operation (else `operation_moved_within_change`); an unpaired add/delete still blocks as `change_context_unresolved`. Admission accepts `(new, file_added_in_head)`; with the empty registry such a candidate's only veto is `capability_unavailable`, and the terminal now shows it under "Engine: N finding(s) this push introduced. Advisory ... not an alert." Any comparison, evidence or location veto keeps a candidate out of that line. Tests: `tests/test_push_new_file.py` (9). Live engine evidence depends on host memory (17.7). Gates: 2007 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.4, 3.4_
+- [x] 17.5 Say when the engine cannot finish, and offer it in the background
+  - One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
+  - Files: `src/openultrasast/push/background.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/cli.py`, `ops/pre-push`. Tests: `tests/test_push_background.py` (new).
+  - Evidence: a hook push with `OUSAST_PUSH_ENGINE=background` followed by a second push showing the previous result.
+  - _Done 2026-10-02:_ `--engine inline|background|off` (hook: `OUSAST_PUSH_ENGINE`) and `--background-deadline` (default 900 s). `off` and `background` run the quick tier and record `engine_off`/`engine_background` with a plain skip line; the deadline line names the deadline and the background option. New `push/background.py` starts one detached explicit replay (`start_new_session`, stdin closed, output to a log) per artifact directory and repository (lock file with a live-pid check: a second start reports busy), writes its artifact under `<artifact dir>/engine/<repo key>/<head>.json`, and the next run prints "Engine result for <head> from the previous push: ..." once, then marks it shown. Not usable through the Docker wrapper (the container exits with the hook); documented in 17.6. Tests: `tests/test_push_background.py` (4, one launching a real detached run). Gates: 2011 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.5, 4.1, 4.2_
+- [x] 17.6 Install, Docker and documentation for a first-time user
+  - `ousast pre-push install|uninstall [--force] [--docker] [REPO]` ships the hook as package data, honours `core.hooksPath`, refuses an existing hook unless `--force` (which chains it as `pre-push.before-ousast`), and works from a pip install. `ops/ousast-docker` is a hook-callable script (not a shell function) that passes stdin and the artifact path through. The `openultrasast:dev` image is rebuilt only if `df -h /` shows at least 4 GB free; otherwise the rebuild command is documented and the record says it was not rebuilt. `docs/scanning.md` gains a first-run guide, a glossary of skip reasons, a GitHub Actions recipe and full `--help` text; README tiers 1 and 2 updated, pulling `ghcr.io/norandom/openultrasast:2.0.1` where the wrapper allows.
+  - Files: `src/openultrasast/push/hook/pre-push`, `src/openultrasast/push/install.py` (new), `src/openultrasast/cli.py`, `ops/ousast-docker`, `docs/scanning.md`, `README.md`, `ops/README.md`. Tests: `tests/test_push_install.py` (new), `mkdocs build --strict`.
+  - Evidence: install from the package into a scratch clone, push through it, uninstall restores the prior hook.
+  - _Done 2026-10-02:_ `ousast pre-push install|uninstall [REPO] [--force] [--docker]` (`push/install.py`) installs the hook shipped as package data (`openultrasast/push/hook/pre-push`, byte-identical to `ops/pre-push`, verified in the built wheel), honours `core.hooksPath` without editing it, refuses an existing hook unless `--force` (ours is replaced; a foreign hook moves to `pre-push.before-ousast` and is chained), and uninstall removes only a hook it wrote and restores the chained one. `ops/ousast-docker` (also package data; `--docker` installs it beside the hook, which then uses it) is a script that mounts the repository read-only and the artifact directory at their own absolute paths, passes stdin with `-i`, runs as the user with no network and `--memory 3g`, defaults to `ghcr.io/norandom/openultrasast:2.0.1`, never pulls inside a hook, and exits 0 unless `OUSAST_PUSH_MODE=blocking`. The hook passes `--engine`/`--comparison-base` only when set, so an older ousast or image still runs. Every pre-push flag has help text; `docs/scanning.md` has the first-run guide, settings table, new-branch and background notes, the artifact keys, a skip-reason glossary (a test keeps it in sync with `push/explain.py`), a GitHub Actions recipe and the full `--help` text; README tiers 1 and 2 and `ops/README.md` updated (pull and tag the GHCR image instead of building). **Image not rebuilt:** `df -h /` showed 1.1 GB free (< 4 GB); rebuild with `docker build -t openultrasast:dev .`. Tests: `tests/test_push_install.py` (11) and the glossary test. Gates: 2022 passed, 42 skipped; ruff, format, mypy clean; `mkdocs build --strict` exit 0.
+  - _Requirements: 9.6, 7.4_
+- [x] 17.7 Prove it on the audit's three repositories
+  - Rerun records, serve-static and unrolled/secure (same planted commits) with the new build in quick mode and with the hook installed, plus one Docker engine run on serve-static if free memory >= 3 GB and free disk >= 1 GB. Prove each run read its input (files, bytes).
+  - Files: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json`. Evidence: before (the audit) and after: exit codes, wall time against the deadline, findings shown, every skip line printed. A planted sample still missed is a coverage gap, recorded, never a rule edit.
+  - _Done 2026-10-02:_ `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (before = the audit, after = this build installed from its wheel into a scratch venv). 10 hook pushes and 6 replays, all exit 0, 0.69-1.21 s against 30 s on the host; the quick tier read exactly the changed file and every head snapshot read the full tree (records 43,577 B, serve-static 61,525 B, secure 103,023 B). The planted JS `eval` in the new `debug.js` surfaces: quick tier on the host, and as two advisory engine findings classified `new`/`file_added_in_head` in one Docker engine run (190.9 s against a 300 s deadline; MemAvailable 3.28 GB, 1.16 GB disk). **The planted Python `%`-formatted SQL is still missed**: no quick rule matches it and the host has no engine. Recorded as a coverage gap, no rule edited. Go is named as not covered. The proof caught a wrong install hint (`pip install openultrasast`, not on PyPI), corrected in the hook in this commit.
+  - _Requirements: 9.1–9.6, 6.3_
+
+
 ## Implementation Notes
 
 - Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.

.kiro/specs/unseen-repo-evaluation/brief.md

@@ -0,0 +1,32 @@
+# Brief: does the safety net work on repositories it has never seen?
+
+Maintainer, 2026-10-04: "we want to spread the safety net across repos in general, unseen code. that was the purpose
+of the AB tests to see if our approach is generalizeable." And: "we should run our tests / runs in kube. now we can
+speed it up and design the workloads for ax directly. kind is only a fallback."
+
+## Why this spec exists
+
+The A/B experiments so far (exp-003, exp-004, exp-005) score the decision engine on repository-held-out folds of known
+fix pairs. That measures whether the vulnerable side of a pair scores above its fixed side. It does not measure what a
+safety net on an arbitrary repository lives on:
+
+- on an ordinary push, does it stay quiet? (false-alarm rate)
+- on a push that introduces a real vulnerability, does it say so? (catch rate)
+
+Populations v1 and v2 measured the engine scan on fresh cases once (1 of 11, 0 of 17) and are spent. Nothing since has
+measured push-level behaviour on unseen repositories.
+
+## What this spec delivers
+
+1. A frozen pool of repositories never used anywhere in development, with two kinds of change each: real
+   vulnerability-introducing commits and ordinary commits.
+2. A push-level metric: each change replayed through the hook exactly as a user would run it, and through other
+   arms (decision engine, engine evidence) where an experiment asks.
+3. Execution designed for the kube-ax cluster from the start: one push replay is one AX Task; kind and the VM are
+   fallbacks.
+4. A baseline for today's hook, then a table of arms scored on the same pool.
+
+## Not in scope
+
+Changing detection. This spec only builds the measurement. Improvements are separate, registered experiments judged
+by it.

.kiro/specs/unseen-repo-evaluation/design.md

@@ -0,0 +1,431 @@
+# Design Document
+
+## Overview
+
+This spec builds the instrument for roadmap G0: a frozen pool of repositories that no development step has seen,
+and a push-level score on it. The score is what a user of the hook sees. On a vulnerability-introducing push, did
+the hook say so (catch rate)? On an ordinary push, did it say anything (false-alarm rate)? Replays run as AX Tasks
+on kube-ax. kind and the VM run the same tasks as fallbacks.
+
+Four facts from the code shape the design:
+
+- **Populations already have a freeze-and-guard pattern.** `benchmarks/independent/freeze.py` verifies cases by
+  reading git only, `evaluate.py` scores with fixed rules and Wilson intervals, and
+  `tests/test_independent_population.py` fails when a reserved repository is named anywhere in `benchmarks/`,
+  `src/` or `tests/` outside its directory. Harvest batch 3 used that guard as a black box: candidates written,
+  guard run with output discarded, subsets bisected on red. That is how v3 stays unread.
+- **The hook can already replay a change.** `ousast pre-push PATH --base B --head H --artifact F` resolves two
+  local commits (`snapshot.py`, `resolve_replay`: no fetch, no merge-base), runs quick rules and the engine inside
+  one deadline (default 30 s, engine inline) and writes a JSON artifact. The terminal text has one `Skipped:` line
+  per skipped check. The artifact holds three finding streams: admitted defects, advisory engine findings
+  (new or worsened, vetoed only by capability) and quick-rule matches with their CWE.
+- **The AX pattern works but lives in one script.** `engine_trace_ax.py` (manifest, egress checks, presigned
+  objects, retry once under a new name, cleanup) and the lane scheduler in `engine_trace.py` (`run_plan`: fixed
+  lanes, fallback queue, `progress.json`, `STOP`, resume by input digest) are engine-specific.
+- **Placement on kube-ax is random.** One worker per node, shared by all actors. Two heavy tasks fit per worker,
+  but only two concurrent tasks in all guarantee at most two on one worker.
+
+### Goals
+
+- A pool frozen by code against every used source (Req 1), with two kinds of change per repository (Req 2).
+- A scorer whose numbers mean what the hook shows, with silent zeros visible (Req 3).
+- One replay is one AX Task via a dispatcher shared with the engine (Req 4); a baseline and fresh slices (Req 5).
+
+### Non-Goals
+
+Detection changes, population v3, model calls in the cluster, languages the hook does not analyse.
+
+## Architecture
+
+```mermaid
+flowchart LR
+    M[pool manifest slice] --> D[batch dispatcher on VM]
+    D --> P[presign spec GET and record PUT]
+    P --> T[AX Task ousast-engine-replay]
+    T --> G[shallow fetch base and head from GitHub]
+    G --> H[ousast pre-push --base --head]
+    H --> R[record tar: artifact, text, instrument]
+    R --> S[(store via presigned PUT)]
+    D -->|sandbox died twice or OOM| V[VM docker lane]
+    D -->|cluster down| K[kind lane, same tasks]
+    V --> S
+    K --> S
+    S --> C[scorer: catch, false alarm, coverage]
+    S --> A[model arms on VM, budgeted]
+    A --> C
+    C --> O[record: counts and digests]
+```
+
+## Components and Interfaces
+
+### 1. Pool sourcing and eligibility (Req 1)
+
+**Decision 1.1: sources and ecosystems.** Candidates come from GitHub reviewed advisories (`gh api /advisories`,
+the enumeration harvest-3 used) for pip, npm, maven and composer, published 2019 or later, whose CWE maps to one of
+the seven families through the project's family CWE table. Each needs a linked fix commit with one parent in one
+repository. OSV records for the same advisory give the first affected version. That version is a cross-check for
+the introducing commit (section 2). Reason: these four ecosystems are the five languages the hook analyses
+(Python, JavaScript, TypeScript, PHP, Java). GHSA links commits and CWEs; OSV adds version brackets. Rejected:
+drawing from every ecosystem (Go, Ruby, C#). That would mostly measure missing frontends, which is already known.
+Coverage still reports uncovered files inside the drawn repositories. Rejected: NVD and published fix datasets.
+NVD links commits weakly, and the datasets are mined corpora that development or model training has likely seen.
+
+**Decision 1.2: eligibility by code against every used source.** `benchmarks/unseen/eligibility.py` builds the
+used set before the draw. It never prints names; it records counts per source and a digest of the set.
+
+| Source (Req 1.1) | Where it is read |
+| --- | --- |
+| Pair corpora | every catalog, recipe and dataset file under `benchmarks/pairs/`, plus the local pointer-pair manifests |
+| Populations v1, v2 | `population-v1.toml`, `population-v2.toml`, read directly |
+| Population v3 | never opened; the existing guard test only (below) |
+| Harvest records | the batch catalogs under `benchmarks/pairs/advisory-fixes*` (the records hold counts only) |
+| Memory examples | `example` rows of the local FileStore and the S3 store (`repo` field) |
+| Experiment units | `plane/experiments/*.units.jsonl` and registered units files |
+| Measurement records | everything under `benchmarks/measurements/` and `benchmarks/experiments/` |
+| Anything else | a text sweep for `github.com/<owner>/<name>` over `benchmarks/`, `src/`, `tests/`, `plane/`, `.kiro/` |
+
+Names are normalised (lower case, no `.git`, no trailing slash). Each candidate is resolved through the GitHub API
+to its canonical `full_name`, its former names (redirects) and its fork network (`parent`, `source`). A candidate
+is out if any of these is in the used set. A fork shares code with what it was forked from.
+
+v3 is checked like harvest-3 did. The draft manifest is written into `benchmarks/unseen/`. Then
+`pytest -q tests/test_independent_population.py -k referenced` runs with its output discarded. The guard already
+sweeps `benchmarks/` outside the population directory. On red, the candidate list is bisected by rewriting the
+draft until green, and the dropped count is recorded. No v3 name is ever printed or read by a person.
+
+**Decision 1.3: post-cutoff preference.** An advisory published after 2026-06-01 (harvest-3's cutoff for the
+decision engine's model) is flagged `post_cutoff`. Within each ecosystem stratum post-cutoff advisories are drawn
+first. Results are reported for both groups. Reason: a post-cutoff advisory cannot be in the model's training
+data as a known vulnerability. Rejected: post-cutoff only. Harvest-3 kept 12 post-cutoff pairs out of 251, and
+three slices of 100 repositories cannot be filled from that window.
+
+**Decision 1.4: licences.** The licence is read at the pinned head commit (LICENSE file, then the API's SPDX id).
+Only permissive licences are published by name: MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, 0BSD, Unlicense,
+Zlib, PostgreSQL, BSL-1.0. Everything else (copyleft, source-available, none) goes to the gitignored
+`benchmarks/unseen/private/pool-p1.toml`. The committed manifest then holds an opaque id and the sha256 of
+(URL, commits) for that entry, so the freeze digest still covers it. Reason: Req 1.4 read literally. Rejected:
+"OSI-approved" as the bar, as population v2 used. It is wider than the requirement says (open question 1,
+resolved 2026-10-04: copyleft stays private).
+
+Also required: anonymous HTTPS, at most 500 MB, at least 40 eligible ordinary commits, one per fork network.
+
+### 2. Change extraction (Req 2.1, 2.2)
+
+Extraction runs on the VM over a full `--filter=blob:none` clone in a scratch cache. It uses git only, like
+`freeze.py`. No scanner, engine or model touches a pool repository before the freeze.
+
+**Decision 2.1: the vulnerability-introducing range.** Let F be the fix and P its parent. The vulnerable lines are
+the old-side lines of F's hunks in the vulnerable function. For a pure-addition fix (an absence bug such as a
+missing guard), they are the vulnerable function's own lines.
+
+1. **`introducing` (preferred).** `git blame -w -M -C --porcelain P -L <lines> -- <file>`. The candidate V is the
+   commit that last wrote the sink line (for absence bugs, the function's declaration). V is accepted if all hold:
+   at V's first parent the sink line's normalised text is absent from the file's vulnerable function; V is not a
+   merge or root commit; V changes at most 50 files and 2,000 lines; when OSV names a first affected version, V is
+   reachable from that tag and not from the previous release tag. If the text is present at V's parent, blame
+   walks back from there, at most five steps. The replay is `base = V^`, `head = V`.
+2. **`last_touch` (fallback).** When no V passes, `git log -L :<function>:<file> P` gives the newest commit L
+   before F that changed the vulnerable function. The replay is `base = L^`, `head = P`. The vulnerable code is in
+   head, and the range contains a change to the vulnerable function.
+
+Each change records its method, family, vulnerable function (resolved in head), head-side changed lines,
+`post_cutoff` and the advisory id.
+
+Bias of each, stated in the record. `introducing` inherits SZZ noise: blame finds the last writer, not the author
+of the flaw. A refactor, a move the heuristics miss or a reformat can stand in for the real introduction. The
+absence check, the walk-back and the version bracket cut this down, but do not remove it. The bias runs both
+ways: a wrong V may introduce nothing (a miss is not the hook's fault) or may move a flaw (the delta-based hook is
+right to call it existing). `last_touch` is pessimistic for a delta hook. The flaw is often already in base, so
+the hook's novelty rule correctly stays quiet and the change counts as a miss. Catch is therefore reported per
+method. The headline is `introducing`; `last_touch` is a lower bound. Rejected: fix-parent against a fixed old
+base (for example 50 commits back). The range often misses the vulnerable function entirely and grows without
+bound on busy repositories.
+
+Label check: every third `introducing` change is read by an agent before the freeze, with no model call, as
+harvest-3 reviewed diffs. It is marked `confirmed`, `refactor` or `unclear`. The confirmed share is the label
+noise estimate. The scorer also reports catch on the confirmed subset.
+
+**Decision 2.2: ordinary commits.** The candidates are non-merge commits on the first-parent line of the default
+branch that change at least one source file of an analysed language. Excluded:
+
+- explicit security signals in the message: `secur|cve|ghsa|vuln|inject|xss|csrf|ssrf|exploit|sanitiz|escap`;
+- reverts (`^Revert`, `This reverts commit`);
+- dependency bumps (bot authors such as dependabot and renovate, `bump|upgrade .* from`, or only manifest and lock
+  files changed);
+- commits whose source changes are all generated or vendored (`vendor/`, `node_modules/`, `dist/`, `*.min.js`,
+  lock files, `linguist-generated` in `.gitattributes`, "generated" headers);
+- any fix commit of any advisory of the repository, and any commit touching a known vulnerable function of it;
+- commits over 1,000 changed source lines (counted, reported).
+
+Twelve are drawn per repository with a seeded random draw. Strata are three time thirds of the repository's
+history crossed with four size buckets of changed source lines (1-10, 11-50, 51-200, 201-1,000). Allocation is
+proportional to the repository's own commits per cell, with at least one per time third. Each change records base
+(its first parent), head, changed files and line ranges. Reason: proportional allocation keeps the false-alarm
+rate a per-push rate as a user meets it. Rejected: `freeze.py`'s broad `SECURITY_WORDS`. It also drops commits
+saying "query", "token", "session" or "valid", which removes the ordinary commits where false alarms are most
+likely, and the rate would look better than it is. Rejected: docs-only commits. They are trivially quiet and
+would dilute the rate.
+
+Label noise: an ordinary commit may itself introduce an unknown vulnerability, so a flag counted as a false alarm
+may be true. The false-alarm rate stays as Req 3.2 defines it. Flagged ordinary changes are adjudicated beside it
+(all if 30 or fewer, else every third) against protocol-v2's standard, and the adjudicated share is reported.
+
+### 3. Sizing and slices (Req 2.3, 5.2)
+
+Wilson 95% interval, z = 1.96: centre (p + z²/2n)/(1 + z²/n), half width z·sqrt(p(1-p)/n + z²/4n²)/(1 + z²/n).
+Taking the larger distance from p as the width, the smallest n that meets the bar, computed exactly:
+
+| Rate | Assumed p | Bar | n needed |
+| --- | --- | --- | --- |
+| false alarm | 0.05 | +/- 3 points | 315 |
+| false alarm | 0.10 | +/- 3 points | 483 |
+| false alarm | 0.20 | +/- 3 points | 756 |
+| catch | 0.10 | +/- 10 points | 62 |
+| catch | 0.25 | +/- 10 points | 88 |
+| catch | 0.40 | +/- 10 points | 97 |
+
+Catch: 100 vulnerability-introducing changes per slice, one per repository. At p = 0.4, Wilson gives
+[0.309, 0.498], within 10 points. One per repository keeps them independent.
+
+False alarm: ordinary changes of one repository are correlated (noisy code flags often). Wilson assumes
+independence, so the count is inflated by a design effect 1 + (m - 1)·ICC. With m = 12 per repository and an
+assumed ICC of 0.05, that is 1.55. 100 repositories × 12 = 1,200 changes, effective n 774 ≥ 756. The plain Wilson
+interval at p = 0.2, n = 1,200 is about +/- 2.3 points. A repository-cluster bootstrap interval is reported beside
+it, and the ICC measured on slice 1 is recorded. If it exceeds 0.05, the next pool sizes up. A frozen slice is
+never changed.
+
+**Decision 3.1: slices.** Pool p1 has three slices of 100 repositories each (100 + 1,200 changes per slice; 300
+repositories and 3,900 changes in all). Repositories are assigned by a seeded shuffle, stratified by ecosystem and
+`post_cutoff`. Slice 1 serves the baseline and the G1 arms table. Slices 2 and 3 serve the first two adoption
+decisions. When they are spent, pool p2 is drawn by the same code, and it excludes p1. Reason: each slice alone
+meets the Req 2.3 widths, so any decision can stand on one slice. Rejected: one large pool scored whole. Every
+decision would then see every repository, which Req 5.2 forbids. Rejected: ten small slices. None would meet the
+widths. Per family, 100 changes split over seven families give intervals of about +/- 20 to 25 points. They are
+reported, not sized (open question 3).
+
+Use ledger: `benchmarks/unseen/usage-p1.jsonl` is append-only, one row per use (slice, decision or experiment id,
+date, result digest). A slice with a row whose purpose is `informed` cannot appear again as `qualifies` for that
+change. A test checks this.
+
+### 4. Scoring rules (Req 3.1, 3.2, 3.3)
+
+Each replay uses the hook's defaults: `--deadline 30`, `--engine inline`, `--max-regions 500`, `--mode advisory`.
+That is what a user's push gets. The text output is kept with every `Skipped:` line, and so is the artifact.
+
+**Decision 4.0: a registered long-deadline arm** (maintainer decision 2026-10-04, open question 5). Beside the
+default replay, slice 1 is replayed once more with `--deadline 300` and every other flag unchanged. The arm is
+registered (manifest committed, `unseen: {pool: p1, slice: 1}`) before its first replay. G1 then reports both
+what the user sees (30 s) and what the engine could catch given time (300 s), so a miss caused by the deadline is
+told apart from a miss by the engine. It is a measurement arm, not a proposed default: adopting a longer hook
+deadline would be a separate decision under 7.2.
+
+**Decision 4.1: catch, per family.** A finding counts if its family is the change's family. Engine streams carry a
+family; a quick-rule match maps its CWE through the family CWE table. Its head location must also be in the
+vulnerable function or within 10 lines of the change's head-side hunks (protocol v1's window).
+
+| Family | Location rule |
+| --- | --- |
+| injection, path, deserialization, output_encoding, untrusted_destination | sink line in the vulnerable function, or the changed lines +/- 10 |
+| access_control | the vulnerable function itself (the handler that lacks the check), or the changed lines +/- 10 |
+| config_secrets | the changed file and lines +/- 10 (file scope; function `<global>`) |
+
+All three streams count: admitted defects, advisory engine findings and quick-rule matches. Each catch also names
+the stream that made it. Reason: the hook shows all three to the user. Rejected: admitted defects only. That
+scores the blocking path, which G3 decides separately, and would hide what users read.
+
+**Decision 4.2: false alarm.** An ordinary change is a false alarm when any of the three streams holds at least one
+finding, of any family, anywhere. The alerts-only rate (admitted defects) is reported beside it.
+
+**Decision 4.3: coverage, so zeros cannot hide.** Every change gets exactly one state:
+
+- `instrument_failure`: no record, a failed fetch, the head commit not verified, or zero bytes of the changed files
+  read. It is re-run once, then reported. It never counts as quiet or missed.
+- `unanalysable`: no changed file is in a language the hook covers (the hook's `language_not_covered` notice).
+- `quick_only`: the quick tier read the changed files, but no engine question completed on them.
+- `engine_covered`: at least one engine question completed on a changed file.
+
+The task entry proves its input as AGENTS.md requires. It records files and bytes in the checkout, bytes of each
+changed file, the JVM's wall time and the hook's exit code. A replay faster than a JVM can start is flagged
+`suspect_fast` and goes to the failure path. Catch and false alarm are reported over all changes and over each
+coverage state.
+
+### 5. The AX workload (Req 4)
+
+**Decision 5.1: the image.** `plane/Dockerfile.engine-task` gains a second stage, `FROM engine AS replay`, built
+with `--target replay` and pinned by digest. It adds `git`, the package with its runtime dependencies and
+`benchmarks/unseen/replay_entry.py`. Joern, php-cli, Java 21 and the frontend-retention patch come from the engine
+stage, so both lanes run the same Joern. The import smoke runs `ousast pre-push --help`. The root image stays
+`ghcr.io/norandom/ax-task-runner:v0.3.1`. Rejected: a separate Dockerfile. It would duplicate the Joern install,
+and two Joern copies are two instruments that drift. Rejected: one stage for both. The engine image's isolated
+venv without yaml and boto3 is proven, and replay dependencies would change its digest on every bump.
+
+**Decision 5.2: the task entry.** `replay_entry.py` is standard library only, like `engine_task_entry.py`, and
+reuses its `transfer` (egress readiness wait, three attempts, no URL text in errors).
+
+1. GET `SPEC_URL` (presigned, `.json`): repository URL, base, head, change id, hook flags.
+2. `git init`, then `git fetch --depth 1 origin <head>` and `git fetch --depth 1 origin <base>` over HTTPS
+   (GitHub serves commits by SHA). Verify both with `git cat-file -e`. The fetch is bounded at 180 s; the 500 MB
+   repository cap bounds its size.
+3. Run `ousast pre-push <checkout> --base <base> --head <head> --artifact /tmp/out/push.json` with defaults, the
+   backend heap at 1700 MB and the cluster JVM flags of `engine_trace_worker.py`. Capture stdout.
+4. Pack `push.json`, `push.txt`, `engine.json` (the head and base scan records and witness rows from the artifact,
+   keyed by repository and revision, for engine-evidence arms) and `instrument.json` (section 4.3, image digest,
+   step timings). PUT to `RESULT_URL` as `.dat`. The PUT is completion.
+
+On any error it PUTs a failure record naming the step and HTTP status, as the engine entry does. Task names are
+`ousast-engine-replay-<slug>-<digest>`, the prefix the operator's egress watcher matches. No key or store
+credential enters the manifest. Only the two presigned URLs do, and the spec object keeps repository names out of
+AX status and cluster telemetry.
+
+**Decision 5.3: the generic dispatcher.** `benchmarks/ax/batch.py` takes over from `engine_trace_ax.AX` and
+`engine_trace.run_plan`:
+
+- `Item`: id, input objects (name to bytes), the result object name, command, extra env, input digest.
+- `Workload`: image digest, allowed URL env names, deadline, how to validate a received result.
+- Lanes: `ax` (manifest, apply, poll the result object and AX phase, delete, clean objects), `kind` (the same
+  manifest against the kind AX), `docker` (the VM image, one container at a time).
+- Scheduler: two AX lanes plus one VM lane by default. A sandbox death is retried once under a new name; a second
+  death or a JVM OOM goes to the VM fallback queue, which beats overflow on the VM lane. `progress.json` is written
+  only by the scheduler. A `STOP` file halts new launches. Resume skips items whose stored record has the same
+  input digest. Cleanup failure stops dispatching.
+
+The engine pass becomes a workload over it, with the same tests. Reason: the egress, secrecy and cleanup logic took
+several fixes and must exist once. Rejected: a replay-specific copy of `engine_trace_ax.py`. Rejected: putting it in
+`src/` now. G4's `ousast plane scan` may need it there, and it moves when G4 is planned.
+
+Concurrency is two AX tasks. Placement is random, and only two guarantees two per worker at most. Four would need
+the operator to add placement (open question 2, resolved 2026-10-04: stays at two, no placement change
+requested). Estimated wall time per slice: about 75 s per replay (fetch, 30 s hook, setup), 1,300 replays over
+three lanes, about 9 hours. The long-deadline arm (4.0) is bounded at about 345 s per replay; at that bound 1,300
+replays over three lanes take about 42 hours, less where the hook finishes early. It runs with `STOP` and resume
+across sessions.
+
+**Decision 5.4: model arms on the VM.** Arms that need DeepSeek (the decision engine, the combiner) read the stored
+records on the VM. They build candidates from the head side's changed functions, score them, and take the change
+score as the maximum. Each runs under its experiment manifest's `budget_usd` through the metered client, dry run
+first, with responses cached. Nothing in the cluster calls a model.
+
+**kind (Req 4.5)** runs the identical manifest and image through the `kind` lane. Every new task is proven there
+before kube-ax, as `plane-on-kubernetes` requires.
+
+### 6. Records and the guard (Req 1.2, 1.3, 5.1)
+
+`benchmarks/unseen/` holds `pool-p1.toml` (repositories, licences, slices, changes with base, head, method,
+family, function, lines), `freeze-p1.json` (used-set digests and counts per source, guard result, label-check
+counts, freeze digest = sha256 of the canonical manifest), `protocol-p1.md` (the scoring rules of section 4,
+committed before the first replay) and `usage-p1.jsonl`. The `private/` directory is gitignored.
+
+The guard grows by one parametrised reservation. The existing sweep becomes a list of (reservation directory,
+names): `benchmarks/independent/` for populations and `benchmarks/unseen/` for pools, including the private
+manifest when present. It also sweeps `plane/`. A second test asserts that pool and population names are disjoint,
+and prints only a count on failure. A third asserts that a frozen pool's digest matches `freeze-p1.json`, so the
+pool is never edited. Raw per-change records stay outside the repository (`~/ousast-results/unseen/` and the
+store). Committed result records under `benchmarks/measurements/` carry counts, rates, intervals and digests,
+never names.
+
+### 7. Metrics and reporting (Req 3.3, 3.4, 5.3)
+
+`benchmarks/unseen/score.py` reports per family and overall: catch and false-alarm rate with Wilson 95%
+intervals, the cluster bootstrap interval for false alarms, coverage counts per state, catch per method,
+post-cutoff and per stream, and the alerts-only rate.
+
+**Decision 7.1: arms at fixed budgets.** An arm with a score is thresholded per budget (1%, 5%, 10%). The
+threshold is the lowest score whose false-alarm rate on the slice's ordinary changes stays within the budget.
+Catch at that threshold is reported with its Wilson interval and the realised false-alarm rate. That in-slice
+threshold is labelled exploratory. For a qualification, the threshold is fixed beforehand on an earlier slice or
+on held-out folds, and it is applied unchanged. A binary arm (today's hook) has one operating point, reported with
+the budgets it meets. Rejected: AUC over changes (it averages operating points no user runs).
+
+**Decision 7.2: next to held-out folds.** Experiment manifests registered after this spec lands carry
+`unseen: {pool, slice}`. `analyse` writes an `unseen` block into `result.json` next to the fold estimates: catch at
+each budget for each arm, and the paired difference B - A with an exact McNemar test on the 100 changes. The
+fold rule decides as today. The slice is recorded in the usage ledger.
+
+**Decision 7.3: adoption gate on the unseen slice** (amended 2026-10-04, maintainer decision; Req 5.4; open
+question 4 resolved). Reporting is not enough. An arm B is adopted over the production arm A only if, on the
+untouched slice that qualifies it, both hold:
+
+- catch at the 5% budget, with B's threshold fixed beforehand (7.1), is not significantly lower than A's: exact
+  two-sided McNemar on the paired vulnerability-introducing changes, alpha 0.05;
+- for an arm with one operating point (a binary arm such as a hook setting), its false-alarm rate is not
+  significantly higher than A's: exact two-sided McNemar on the paired ordinary changes, alpha 0.05, with the
+  repository-cluster bootstrap interval of the difference reported beside it, since those changes are clustered.
+
+`analyse` writes `unseen.gate: pass | regression | not_run` into `result.json`. `not_run` (no slice, slice
+already `informed` for this change, or more than 5% `instrument_failure`) blocks adoption like `regression`. The
+fold rule and this gate must both pass.
+
+### 8. Increments
+
+1. **Used set and guard.** `eligibility.py`, the extended guard and the usage-ledger test. Exit: guard green on an
+   empty pool; a planted used repository, a renamed one and a fork are each rejected in tests.
+2. **Sourcing and extraction.** Candidates, both change kinds and the label check, on the VM. Exit: a draft for 300
+   repositories with method counts, the confirmed share and exclusion counts.
+3. **Freeze p1.** Guard bisection for v3, then manifest, freeze record and protocol committed. Exit: freeze digest
+   committed and guard green.
+4. **Dispatcher.** `benchmarks/ax/batch.py`, with the engine pass moved onto it. Exit: existing tests green and a
+   two-pin engine smoke on kube-ax matching its stored records.
+5. **Replay task.** Image stage and entry, proven on kind, then kube-ax on five changes from this repository's own
+   history (never pool changes). Exit: instrument fields show bytes read and realistic JVM time, and quick-tier
+   output matches the VM lane.
+6. **Scorer.** `score.py` on synthetic records, including every coverage state and an instrument failure. Exit:
+   unit tests green.
+7. **Baseline.** Today's hook, from a frozen export of HEAD, on slice 1 (1,300 replays). Exit: the record committed
+   (counts and digests only) and a `baseline` row in the usage ledger. That is G0's exit.
+8. **Arms on slice 1 (G1)** (added 2026-10-04 with decisions 4.0 and 7.3). The registered long-deadline arm,
+   the engine-evidence arm from the stored records, the model arms on the VM (5.4) under their budgets, and the
+   `unseen` block with the adoption gate in `analyse`. Exit: one committed table of arms on slice 1 at the 1/5/10%
+   budgets (counts, rates, intervals, digests), and `exploratory` rows in the usage ledger.
+
+## Error Handling
+
+A failed fetch, a missing commit or a hook crash gives a failure record, never an empty success. A dead sandbox is
+retried once, then the VM runs it. A store or AX cleanup error stops the dispatcher. A slice with more than 5%
+`instrument_failure` after re-runs is not scored; the cause is fixed first.
+
+## Testing Strategy
+
+Unit tests: used-set normalisation (case, `.git`, renames, forks), blame acceptance on a synthetic repository,
+ordinary-commit exclusions, Wilson values against section 3, budget thresholds, coverage states, the task manifest
+and dispatcher resume and fallback with a fake lane. Guard tests run in CI; cluster runs stay manual.
+
+## Requirement Traceability
+
+| Requirement | Design |
+| --- | --- |
+| 1.1 eligibility against every source, v3 via the guard | 1.2 |
+| 1.2 frozen manifest, never edited | 6, increment 3 |
+| 1.3 pool directory, guard extended | 6 |
+| 1.4 licences, private manifest | 1.4 |
+| 2.1 introducing changes, family, function, post-cutoff | 1.1, 1.3, 2.1 |
+| 2.2 ordinary changes across time and size | 2.2 |
+| 2.3 sizes for interval widths | 3 |
+| 3.1 default replay, skipped checks kept | 4, 5.2 |
+| 3.2 catch and false-alarm rules | 4.1, 4.2 |
+| 3.3 per family, Wilson, coverage | 4.3, 7 |
+| 3.4 arms at 1/5/10% budgets | 4.0, 7.1, increment 8 |
+| 4.1 one change one Task, prefix, image by digest | 5.1, 5.2 |
+| 4.2 HTTPS fetch, presigned record, no keys | 5.2 |
+| 4.3 generic dispatcher, concurrency, retry, fallback, resume | 5.3 |
+| 4.4 model arms on the VM within budgets | 5.4 |
+| 4.5 kind runs the same tasks | 5.3, 5 (kind paragraph) |
+| 5.1 baseline on slice 1, counts and digests | 6, increment 7 |
+| 5.2 fresh slice per decision | 3.1, usage ledger |
+| 5.3 experiments report unseen next to folds | 7.2 |
+| 5.4 adoption needs no significant unseen regression | 7.3 |
+
+## Open Questions
+
+Maintainer decisions of 2026-10-04 resolve questions 1, 2, 4 and 5; question 3 stays open for G3.
+
+1. **Resolved 2026-10-04.** Licences: only permissively licensed repositories are published by name. Copyleft and
+   all other licences go to the gitignored private manifest, covered in the committed manifest by an opaque id and
+   a digest (Req 1.4 as written; decision 1.4). Population v2's OSI-approved bar is not used.
+2. **Resolved 2026-10-04.** Concurrency stays at two AX lanes plus the VM lane. No operator placement change is
+   requested (5.3).
+3. Per-family BLOCK (G3) needs about 100 changes per family. Pool slices together, or draw a family-enriched pool
+   for G3?
+4. **Resolved 2026-10-04.** An adoption requires no significant regression on the unseen slice, not only a report
+   (Req 5.4, amended; decision 7.3).
+5. **Resolved 2026-10-04.** G1 adds a registered long-deadline arm (`--deadline 300`) next to the default 30 s, so
+   the table shows both what the user sees and what the engine could catch (decision 4.0, increment 8).

.kiro/specs/unseen-repo-evaluation/requirements.md

@@ -0,0 +1,107 @@
+# Requirements Document
+
+## Introduction
+
+The safety net is meant for repositories it has never seen. This spec builds the instrument that measures that: a
+frozen pool of unseen repositories, a push-level metric (catch rate and false-alarm rate as the hook reports them),
+and an execution path that runs the replays as AX Tasks on the kube-ax cluster. Context: `brief.md`, the roadmap's
+G0-G1 (`.kiro/steering/roadmap.md`).
+
+## Boundary Context
+
+- **In scope**: pool selection and freezing; change extraction (vulnerability-introducing and ordinary commits);
+  replay of each change through `ousast pre-push` and through registered experiment arms; the metric and its
+  confidence intervals; AX Task execution with kind/VM fallback; the baseline for today's hook; rotation of pool
+  slices.
+- **Out of scope**: any detection change; population v3 (stays reserved and unread); model calls inside the cluster.
+- **Adjacent expectations**: `learned-decision-engine` experiments use this pool as their primary unseen-repo score;
+  `pre-push-safety-net` owns the hook being replayed; `plane-on-kubernetes` and the AX executor
+  (`benchmarks/learn/engine_trace_ax.py`) supply the execution pattern; the reserved-repository guard covers every
+  population and this pool.
+
+## Requirements
+
+### Requirement 1: A pool no development step has touched
+
+**User Story:** As the maintainer, I want the evaluation repositories to be ones nothing in development has seen, so a
+good number means the approach generalises.
+
+#### Acceptance Criteria
+
+1. A repository is eligible only if it appears in no pair corpus, no population (v1, v2, v3), no harvest record, no
+   memory example, no experiment unit and no measurement record. Eligibility is checked by code against every source
+   before the pool is frozen; v3 is checked through the existing guard without being opened.
+2. The pool is frozen (a committed manifest with repository URLs, commit SHAs, licences and the freeze digest) before
+   any change in it is replayed. A frozen pool is never edited; a correction is a new pool.
+3. Pool repositories are referenced only inside the pool's own directory (like `benchmarks/independent/`), and the
+   reserved-repository guard is extended to them.
+4. Only permissively licensed repositories are published by name; others may be used with their names kept out of
+   the repository (pointers in a local, gitignored manifest), recorded as such.
+
+### Requirement 2: Two kinds of change per repository
+
+**User Story:** As the maintainer, I want both vulnerable and ordinary pushes from the same repositories, so the
+false-alarm rate is measured on the code the catch rate is measured on.
+
+#### Acceptance Criteria
+
+1. Vulnerability-introducing changes: from public advisories with a linked fix, the commit range that introduced the
+   vulnerable code where it can be determined, else the fix's parent compared with an earlier base; each labelled
+   with family and the vulnerable function. Post-model-cutoff advisories are preferred and flagged.
+2. Ordinary changes: commits from the same repositories' histories with no security fix, revert or advisory link,
+   sampled across time and size; each recorded with its changed files and lines.
+3. Per repository, at least one vulnerability-introducing change and several ordinary changes; the pool's totals are
+   sized so the false-alarm rate's 95% interval is no wider than +/- 3 points and the catch rate's no wider than
+   +/- 10 points (sizes computed in the design).
+
+### Requirement 3: The push-level metric
+
+**User Story:** As the maintainer, I want the score to be what a user would see, so the number means something.
+
+#### Acceptance Criteria
+
+1. Each change is replayed as `ousast pre-push --base <base> --head <head>` with the hook's default settings, in a
+   clean checkout, with the record of every skipped check kept.
+2. Catch: a vulnerability-introducing change counts as caught when the hook (or the arm) reports a finding of the
+   right family on the vulnerable function or its changed lines. False alarm: an ordinary change counts as a false
+   alarm when the hook reports any advisory finding.
+3. Reported per family and overall: catch rate, false-alarm rate, and both with 95% Wilson intervals; plus coverage
+   (changes where the hook could analyse the changed language) so a silent zero is visible.
+4. Arms (decision engine, combiner, engine evidence, fix mechanisms) are scored on the same replays: catch rate at
+   fixed false-alarm budgets (1%, 5%, 10%), so arms are compared at equal noise.
+
+### Requirement 4: Execution on kube-ax, designed for AX
+
+**User Story:** As the maintainer, I want the replays to run on the cluster, so measurements are fast and the VM is
+free.
+
+#### Acceptance Criteria
+
+1. One replayed change is one AX Task, named with the `ousast-engine-` prefix the operator's egress watcher matches,
+   using an image built `FROM ghcr.io/norandom/ax-task-runner:v0.3.1` with this package, Joern and php-cli, pinned by
+   digest.
+2. The task fetches the repository at the needed commits itself over HTTPS (public hosts are reachable from actors)
+   and writes its record through a presigned link; no API keys or store credentials enter the cluster.
+3. A generic AX batch dispatcher serves both this replay and the existing engine passes: items to Tasks, at most the
+   cluster's safe concurrency (two heavy tasks per worker), retry once on sandbox death, then a VM or kind fallback
+   lane; progress and resume as in the engine runner.
+4. Arms that need model calls take the cluster's records and make their calls on the VM, within registered budgets.
+5. kind runs the same tasks unchanged as the local fallback.
+
+### Requirement 5: Baseline, rotation and use
+
+**User Story:** As the maintainer, I want a baseline now and a fresh slice for each decision, so the pool is not
+worn out by tuning.
+
+#### Acceptance Criteria
+
+1. The first use is a baseline of today's hook on the first slice; its record is committed (counts and digests only,
+   no repository names outside the pool directory).
+2. A slice that informed a change can no longer qualify that change; each adoption decision draws a new, untouched
+   slice, recorded with its freeze digest.
+3. Every learned-decision-engine experiment from now on reports its unseen-repo score from this pool next to its
+   held-out-fold score.
+4. An arm is adopted only if, on the untouched slice that qualifies it, it shows no significant regression against
+   the current production arm: catch at the 5% false-alarm budget not significantly lower, and for an arm with one
+   operating point the false-alarm rate not significantly higher (exact paired tests, two-sided, alpha 0.05).
+   Reporting the unseen score alone is not enough to adopt. *(amended 2026-10-04, maintainer decision)*

.kiro/specs/unseen-repo-evaluation/spec.json

@@ -0,0 +1,22 @@
+{
+  "feature_name": "unseen-repo-evaluation",
+  "created_at": "2026-10-04T18:00:00Z",
+  "updated_at": "2026-10-04T20:00:00Z",
+  "language": "en",
+  "phase": "implementation",
+  "approvals": {
+    "requirements": {
+      "generated": true,
+      "approved": true
+    },
+    "design": {
+      "generated": true,
+      "approved": true
+    },
+    "tasks": {
+      "generated": true,
+      "approved": true
+    }
+  },
+  "ready_for_implementation": true
+}

.kiro/specs/unseen-repo-evaluation/tasks.md

@@ -0,0 +1,384 @@
+# Implementation Plan
+
+One task group per increment of `design.md` section 8, in that order. A group starts only after the previous
+group's exit is recorded. Maintainer decisions of 2026-10-04 are folded in: a registered long-deadline arm
+(`--deadline 300`) beside the hook's default 30 s (design 4.0, group 8); adoption needs no significant regression
+on the unseen slice (Req 5.4 as amended, design 7.3, tasks 6.4 and 8.2); only permissively licensed repositories
+are published by name, everything else goes to the gitignored private manifest under a digest (Req 1.4, design
+1.4); concurrency stays at two AX lanes plus the VM lane (design 5.3).
+
+Execution rules:
+
+- **Who does what.** Codex implements every coding task offline: no network, no model, no store, no cluster. The
+  coordinator runs every networked step (GitHub and OSV API calls, git fetches and clones, the S3 store, image
+  pushes, kind and kube-ax). Those tasks are marked **(coordinator, network)**. Steps that need the coordinator but
+  no network (agent label reads, adjudication, the v3 guard bisection, commits) are marked **(coordinator)**.
+  Steps that spend model money are marked **paid: needs a budget go-ahead**. Each starts only after the maintainer
+  confirms a ceiling against the provider balance read just before it.
+- **Done means.** A coding task is done when its tests pass in the host suite, `ruff check`, `ruff format --check`
+  and `mypy` are clean, and the reserved-repository guard is green. No test makes a network, model or cluster call.
+- **Names stay in the pool directory.** Pool repository names and URLs appear only under `benchmarks/unseen/`
+  (copyleft and other non-permissive ones only under the gitignored `benchmarks/unseen/private/`). Raw per-change
+  records live in `~/ousast-results/unseen/` and the store, never in the repository.
+- **Population v3 is never opened.** It is checked only through the guard test, run as a black box with its
+  output discarded.
+- **Every group ends with a measured exit** and a record at
+  `benchmarks/measurements/<date>-unseen-NN-<slug>/record.json`: commit, image digests where used, the instrument
+  checks (bytes read, JVM time, exit codes), counts and digests. No repository names. The guard sweeps these
+  records.
+- **Instrument first** (AGENTS.md). A zero, an empty draft or a fast run is treated as an instrument failure until
+  bytes read and wall time prove otherwise.
+
+- [x] 1. Used set and guard (design 1.2, 6; increment 1)
+- [x] 1.1 Used-set builder
+  - `benchmarks/unseen/eligibility.py`: normalises names (lower case, no `.git`, no trailing slash) and builds the
+    used set from every source of design table 1.2: pair catalogs, recipes and datasets under `benchmarks/pairs/`
+    plus the local pointer-pair manifests as `benchmarks/pairs/catalog_gen.py` reads them; `population-v1.toml`
+    and `population-v2.toml`; the advisory-fix batch catalogs; memory `example` rows (`repo` field) through an
+    injected `MemoryStore` (`FileStore` locally, the S3 store when the coordinator passes one); experiment units
+    under `plane/experiments/`; `benchmarks/measurements/` and `benchmarks/experiments/`; a text sweep for
+    `github.com/<owner>/<name>` over `benchmarks/`, `src/`, `tests/`, `plane/` and `.kiro/`. It never reads
+    population v3 and never prints a name. Output: counts per source and the sha256 of the sorted set.
+  - `tests/test_unseen_eligibility.py`: a synthetic tree with one repository per source, each found; case, `.git`
+    and trailing-slash variants collapse; stdout and the output JSON contain no name; a v3-shaped file planted in
+    the synthetic tree is not opened (open is patched to fail on it).
+  - _Requirements: 1.1_
+- [x] 1.2 Canonical resolution, renames and forks
+  - `eligibility.py`: `resolve(candidate, api)` with an injected GitHub client returns the canonical `full_name`,
+    former names (redirects) and the fork network (`parent`, `source`). A candidate is ineligible when any of them
+    is in the used set; one candidate per fork network is kept.
+  - Tests with a fake client: a planted used repository, a renamed one and a fork of a used one are each
+    rejected; two forks of one network keep one; the rejection reason is a source label, not a name.
+  - _Requirements: 1.1_
+- [x] 1.3 Guard extended to pools
+  - `tests/test_independent_population.py`: the sweep becomes a list of (reservation directory, names):
+    `benchmarks/independent/` for populations, `benchmarks/unseen/` for pools (public manifests, plus
+    `benchmarks/unseen/private/*.toml` when present). The sweep also covers `plane/`. New tests: pool and
+    population names are disjoint (failure prints a count only); a frozen pool's manifest digest equals its
+    `freeze-pN.json` (skipped while no pool is frozen). The existing v3 handling is unchanged.
+  - Each reservation is exempt only inside its own directory: population names (v3 included) are still swept in
+    `benchmarks/unseen/`, the gitignored `private/` files included. Task 3.1 depends on this.
+  - `.gitignore`: `benchmarks/unseen/private/`.
+  - Evidence: the guard is green with an empty `benchmarks/unseen/`, and red when a test fixture names a pool
+    repository in `benchmarks/measurements/` (shown once in the record).
+  - _Requirements: 1.2, 1.3, 1.4_
+- [x] 1.4 Usage ledger
+  - `benchmarks/unseen/ledger.py`: append-only `usage-pN.jsonl`, one row per use (slice, decision or experiment
+    id, purpose `baseline | exploratory | informed | qualifies`, date, result digest, freeze digest). It refuses a
+    `qualifies` row for a change whose slice already has an `informed` row for it, and refuses to rewrite rows.
+  - `tests/test_unseen_ledger.py`: append, the refusal, a rewritten row detected by a digest chain.
+  - _Requirements: 5.2_
+- [x] 1.5 Used-set measurement and record 01 **(coordinator, network)**
+  - Run `eligibility.py` against the real sources, the S3 memory store included (read only).
+  - Instrument: each source reports files read and bytes; a source with zero rows fails the run unless it is
+    known empty.
+  - `benchmarks/measurements/<date>-unseen-01-used-set/record.json`: counts per source, used-set digest, guard
+    result (green on the empty pool), test counts of 1.1-1.4.
+  - _Requirements: 1.1, 1.3_
+
+- [ ] 2. Sourcing and extraction (design 1.1, 1.3, 1.4, 2, 3; increment 2)
+- [x] 2.1 Advisory candidates and repository filters
+  - `benchmarks/unseen/source.py`: from GHSA advisory JSON (pip, npm, maven, composer; published 2019 or later)
+    keep those whose CWE maps to a family through `src/openultrasast/model/taxonomy.py` and that link one fix
+    commit with one parent in one repository; attach the OSV first affected version; flag `post_cutoff` when
+    published after 2026-06-01. Repository filters: anonymous HTTPS, at most 500 MB, licence at the pinned head
+    (LICENSE file, then the API's SPDX id) classed `permissive` (the design 1.4 list) or `private`.
+  - `tests/test_unseen_source.py` on recorded fixture JSON (no network): CWE mapping, multi-parent and
+    multi-repository fixes dropped, cutoff flag, licence classing (GPL, AGPL, LGPL, none and source-available go
+    private).
+  - _Requirements: 1.4, 2.1_
+- [x] 2.2 Vulnerability-introducing range
+  - `benchmarks/unseen/extract.py`: the `introducing` method of design 2.1 (blame of the sink line, or the
+    declaration for an absence bug; absence at V's first parent; no merge or root; at most 50 files and 2,000
+    lines; OSV version bracket when present; walk-back of at most five steps) and the `last_touch` fallback. Each
+    change records method, family, vulnerable function resolved in head, head-side lines, `post_cutoff`, advisory
+    id. Git only; no scanner or engine.
+  - `tests/test_unseen_extract.py` on a synthetic repository built in a temp dir: accepted V; a reformat walked
+    back; a merge rejected; size cap; bracket rejection; fallback chosen when no V passes.
+  - _Requirements: 2.1_
+- [x] 2.3 Ordinary-commit draw
+  - `extract.py`: first-parent, non-merge commits touching an analysed language; the exclusions of design 2.2
+    (security words of the narrow pattern, reverts, bot and bump commits, manifest- or lock-only, generated or
+    vendored, advisory fix commits, commits touching a known vulnerable function, more than 1,000 source lines,
+    counted). Seeded draw of 12 per repository over three time thirds by four size buckets, proportional, at least
+    one per third; a repository with fewer than 40 eligible commits is dropped.
+  - Tests: each exclusion; the narrow pattern keeps "query", "token", "session"; allocation and the seed
+    reproduce the same draw; the 40-commit floor.
+  - _Requirements: 2.2, 2.3_
+- [x] 2.4 Draft manifest and slices
+  - `benchmarks/unseen/draft.py`: writes `benchmarks/unseen/draft-p1.toml` (permissive entries by name) and
+    `benchmarks/unseen/private/draft-p1.toml` (the rest); the public file holds an opaque id and
+    sha256(URL, commits) per private entry. Seeded shuffle into three slices of 100 repositories, stratified by
+    ecosystem and `post_cutoff`; each repository has one introducing change and 12 ordinary changes.
+  - Tests: the digest covers private entries; no private URL in the public file; the same seed gives the same
+    slices; per-slice strata within one repository of proportional.
+  - _Requirements: 1.2, 1.4, 2.3, 5.2_
+- [ ] 2.5 Sourcing run **(coordinator, network)**
+  - `gh api /advisories` enumeration, OSV lookups, canonical resolution against the used set (1.2), blobless
+    clones into a scratch cache, extraction (2.2, 2.3), draft (2.4).
+  - Instrument: clones counted with bytes on disk; a repository whose clone reads zero source files is an
+    instrument failure, not a drop. Fewer than 300 eligible repositories stops the run with counts; the
+    eligibility bars are not relaxed to fill it.
+  - Observable: a draft of 300 repositories (3,900 changes) on disk.
+  - _Requirements: 1.1, 1.4, 2.1, 2.2, 2.3_
+- [ ] 2.6 Label check and record 02 **(coordinator)**
+  - Every third `introducing` change read by an agent, no model call, marked `confirmed | refactor | unclear`
+    in the draft.
+  - `benchmarks/measurements/<date>-unseen-02-draft/record.json`: candidates per ecosystem, rejections per used
+    source, licence split (public, private), method counts, post-cutoff counts, confirmed share, ordinary-commit
+    exclusion counts. No names.
+  - _Requirements: 2.1, 2.2_
+
+- [ ] 3. Freeze p1 (design 1.2, 6; increment 3)
+- [ ] 3.1 Guard bisection tool
+  - `benchmarks/unseen/bisect_guard.py`: rewrites the draft with a candidate subset, runs
+    `.venv/bin/pytest -q -p no:cacheprovider tests/test_independent_population.py -k referenced` with stdout and
+    stderr discarded, reads only the exit code, and bisects until green. Reports the dropped count only.
+  - `tests/test_unseen_bisect.py` with an injected fake guard (a hidden set): finds and drops exactly the hidden
+    members; never logs a candidate; an exit code other than 0 or 1 aborts (instrument failure).
+  - _Requirements: 1.1_
+- [ ] 3.2 Freeze writer
+  - `benchmarks/unseen/freeze.py`: canonical `pool-p1.toml` (and the private counterpart) from the bisected draft;
+    `freeze-p1.json` with used-set digest and per-source counts, guard result, dropped count, label-check counts
+    and the freeze digest (sha256 of the canonical manifest). Refuses to write over a frozen pool; a correction
+    is `p2`.
+  - Tests: digest stable under key order; overwrite refused; the guard's digest test (1.3) passes on a fixture
+    pool.
+  - _Requirements: 1.2_
+- [ ] 3.3 Protocol p1
+  - `benchmarks/unseen/protocol-p1.md`: the scoring rules of design section 4 (default flags, decision 4.0's
+    long-deadline arm, catch and location rules, false alarm, coverage states), the statistics of section 3 and
+    7, and the adoption gate of 7.3, stated before any replay.
+  - Observable: the file exists and its sha256 is listed in `freeze-p1.json`.
+  - _Requirements: 3.1, 3.2, 3.3, 3.4, 5.4_
+- [ ] 3.4 Freeze run and record 03 **(coordinator)**
+  - Run the bisection (3.1), then the freeze (3.2). Guard green, digest test active.
+  - `benchmarks/measurements/<date>-unseen-03-freeze/record.json`: freeze digest, protocol digest, dropped
+    count, final counts per slice and kind.
+  - _Requirements: 1.1, 1.2, 1.3_
+
+- [ ] 4. Dispatcher (design 5.3; increment 4)
+- [x] 4.1 Items, workloads and the AX lane
+  - `benchmarks/ax/batch.py`: `Item` (id, input objects, result object name, command, extra env, input digest),
+    `Workload` (image digest, allowed URL env names, deadline, result validator) and the `ax` lane, moved from
+    `benchmarks/learn/engine_trace_ax.py`: manifest render, egress readiness, presigned GET and PUT as `.dat`,
+    apply, poll result object and AX phase, delete, object cleanup. The `kind` lane is the same lane with the
+    kind context.
+  - `tests/test_ax_batch.py` with a fake `ax` CLI and a fake store: manifest holds only the allowed URL env
+    names and no credential; names carry the `ousast-engine-` prefix; cleanup on success and failure.
+  - _Requirements: 4.1, 4.2, 4.5_
+- [x] 4.2 Scheduler, fallback and resume
+  - `batch.py`: two AX lanes and one VM `docker` lane by default (one container at a time); sandbox death retried
+    once under a new name, a second death or a JVM OOM goes to the VM fallback queue ahead of overflow;
+    `progress.json` written by the scheduler only; a `STOP` file halts launches; resume skips items whose stored
+    record has the same input digest; a cleanup failure stops dispatching.
+  - Tests with fake lanes: retry then fallback; OOM straight to fallback; resume skips; `STOP`; cleanup failure
+    halts; never more than two AX tasks in flight.
+  - _Requirements: 4.3_
+- [x] 4.3 Engine pass on the dispatcher
+  - `benchmarks/learn/engine_trace_ax.py` and `engine_trace.run_plan` become an engine `Workload` over `batch.py`;
+    no behaviour change.
+  - Observable: `tests/test_ax_executor.py` and `tests/test_engine_trace.py` pass with their assertions unchanged
+    (moved imports only).
+  - _Requirements: 4.3_
+- [ ] 4.4 Engine smoke on kube-ax and record 04 **(coordinator, network)**
+  - Two pins through the new dispatcher on kube-ax, compared with their stored records.
+  - `benchmarks/measurements/<date>-unseen-04-dispatcher/record.json`: image digest, per-pin wall time and JVM
+    time, record digests equal to the stored ones (or the diff counted), cleanup result.
+  - _Requirements: 4.3_
+
+- [ ] 5. Replay task (design 5.1, 5.2; increment 5)
+- [x] 5.1 Image stage
+  - `plane/Dockerfile.engine-task`: second stage `FROM engine AS replay` adding `git`, the package with its
+    runtime dependencies and `benchmarks/unseen/replay_entry.py`; smoke `ousast pre-push --help`. The root image
+    stays `ghcr.io/norandom/ax-task-runner:v0.3.1`; the engine stage is unchanged.
+  - `tests/test_unseen_image.py`: parses the Dockerfile; the `replay` stage builds from `engine`; the root image
+    pin; the engine stage's text is byte-identical to before.
+  - _Requirements: 4.1_
+- [x] 5.2 Task entry
+  - `benchmarks/unseen/replay_entry.py`, standard library only, reusing `engine_task_entry.transfer`: GET the spec;
+    `git init` and depth-1 fetches of head and base over HTTPS (180 s bound), verified with `git cat-file -e`; run
+    `ousast pre-push` with the spec's hook flags (defaults, or `--deadline 300` for the long-deadline arm), heap
+    1700 MB and the cluster JVM flags; pack `push.json`, `push.txt`, `engine.json`, `instrument.json` (files and
+    bytes in the checkout, bytes of each changed file, JVM wall time, hook exit code, step timings, image
+    digest); PUT as `.dat`. Any error PUTs a failure record naming the step and status, never URL text.
+  - `tests/test_unseen_replay_entry.py`: a local bare repository and a localhost presign stub; success record
+    fields; a missing commit, a failed fetch and a hook crash each give a failure record; zero bytes read and a
+    replay faster than JVM start are flagged `suspect_fast`.
+  - _Requirements: 3.1, 4.1, 4.2_
+- [x] 5.3 Replay workload
+  - `benchmarks/unseen/replay.py`: builds `Item`s from a frozen slice and an arm (`default` or `long_deadline`);
+    the spec object carries the repository URL so names stay out of AX status and telemetry; task names
+    `ousast-engine-replay-<slug>-<digest>` with an opaque slug; the validator rejects a record without
+    `instrument.json`; records go to `~/ousast-results/unseen/<pool>/<slice>/<arm>/` and the store.
+  - Tests: no repository name in any task name or manifest; only the two presigned URLs in the env; input
+    digest changes with the arm's flags; `instrument_failure` items queued for one re-run.
+  - _Requirements: 3.1, 4.1, 4.2, 4.3_
+- [ ] 5.4 Proof on kind, then kube-ax, and record 05 **(coordinator, network)**
+  - Build and push the `replay` target, pinned by digest. Five changes from this repository's own history (never
+    pool changes) on kind, then the same five on kube-ax, then on the VM lane.
+  - Exit: instrument fields show bytes read and realistic JVM time on every lane; quick-tier findings equal
+    across lanes.
+  - `benchmarks/measurements/<date>-unseen-05-replay-task/record.json`: image digest, per-lane timings, bytes,
+    exit codes, the quick-tier comparison.
+  - _Requirements: 4.1, 4.2, 4.5_
+
+- [x] 6. Scorer (design 3, 4, 7.1, 7.3; increment 6)
+- [x] 6.1 Coverage states
+  - `benchmarks/unseen/score.py`: each record gets one state (`instrument_failure`, `unanalysable`, `quick_only`,
+    `engine_covered`) from `instrument.json` and the artifact; `suspect_fast` goes to failure; a slice above 5%
+    `instrument_failure` after re-runs is refused.
+  - `tests/test_unseen_score.py` on synthetic records: every state; the refusal; a failure never counts as quiet
+    or missed.
+  - _Requirements: 3.3_
+- [x] 6.2 Catch and false-alarm rules
+  - `score.py`: family match (engine family, quick-rule CWE through `taxonomy.py`), the location table of design
+    4.1, all three streams with the stream named per catch; false alarm on any finding of any stream; the
+    alerts-only rate.
+  - Tests: right family wrong place misses; `access_control` on the handler; `config_secrets` file scope; a
+    quick-rule-only catch; one advisory finding on an ordinary change is a false alarm.
+  - _Requirements: 3.2_
+- [x] 6.3 Rates, intervals and breakdowns
+  - `score.py`: Wilson 95% intervals; repository-cluster bootstrap for false alarms; ICC; breakdowns per family,
+    method, `post_cutoff`, stream, coverage state and the confirmed subset.
+  - Tests: the n values of design section 3 (315, 483, 756, 62, 88, 97) reproduced; bootstrap seeded; ICC on a
+    synthetic clustered set.
+  - _Requirements: 2.3, 3.3_
+- [x] 6.4 Budgets, paired tests, the gate and record 06
+  - `score.py`: per-budget thresholds (1%, 5%, 10%), labelled `exploratory` in-slice and `fixed` when supplied;
+    a binary arm's single operating point with the budgets it meets; exact two-sided McNemar for catch and for
+    false alarm; `gate(A, B)` returning `pass | regression | not_run` per design 7.3.
+  - Tests: thresholds on a synthetic score set; McNemar exact values; a binary arm with a significant
+    false-alarm increase gives `regression`; missing slice or excess failures give `not_run`.
+  - `benchmarks/measurements/<date>-unseen-06-scorer/record.json`: test counts and the digest of the scorer's
+    output on the committed synthetic fixture.
+  - _Requirements: 3.4, 5.4_
+
+- [ ] 7. Baseline (design 4, 6; increment 7; G0 exit)
+- [ ] 7.1 Frozen source and dry run **(coordinator, network)**
+  - Export HEAD to a frozen tree; build and push the `replay` image from it, pinned by digest.
+  - `benchmarks/push/python_overhead.py` run with the stub engine to size per-replay Python cost; dispatcher dry
+    run on slice 1 lists 1,300 items without launching.
+  - _Requirements: 5.1, 4.1_
+- [ ] 7.2 Baseline replay of slice 1 **(coordinator, network)**
+  - Default arm, 1,300 replays over two AX lanes and the VM lane (about 9 hours); `instrument_failure` items
+    re-run once; the run stops for a cause above 5% failures.
+  - Observable: 1,300 records in `~/ousast-results/unseen/p1/1/default/` and the store, each with
+    `instrument.json`.
+  - _Requirements: 3.1, 4.3, 5.1_
+- [ ] 7.3 Adjudication **(coordinator)**
+  - Flagged ordinary changes adjudicated against protocol-v2's standard (all if 30 or fewer, else every third);
+    the adjudicated share kept beside the false-alarm rate. ICC measured; above 0.05 noted as "next pool sizes
+    up".
+  - _Requirements: 3.2, 2.3_
+- [ ] 7.4 Record 07 and the ledger **(coordinator)**
+  - `benchmarks/measurements/<date>-unseen-07-baseline/record.json`: `score.py` output (counts, rates, Wilson and
+    bootstrap intervals, coverage per state, per family, method, stream), freeze digest, image digest, commit.
+    `usage-p1.jsonl` row `baseline`.
+  - Observable: guard green over the record; G0's exit met.
+  - _Requirements: 5.1, 5.2, 3.3_
+
+- [ ] 8. Arms on slice 1 (design 4.0, 5.4, 7.1-7.3; increment 8; G1)
+- [ ] 8.1 Registered long-deadline arm
+  - `plane/experiments/` manifest for the long-deadline arm: hook flags with `--deadline 300`, budget 0,
+    `unseen: {pool: p1, slice: 1}`, committed before its first replay. `src/openultrasast/learn/experiments.py`
+    accepts `unseen` and a hook-flag arm.
+  - `tests/test_learn_experiments.py`: the manifest loads; an `unseen` block without pool or slice is refused.
+  - _Requirements: 3.4, 5.3_
+- [ ] 8.2 `unseen` block and adoption gate in `analyse`
+  - `experiments.py` `analyse`: writes `unseen` into `result.json` (catch at each budget per arm, paired B - A,
+    McNemar, `gate`) from `score.py`; checks the usage ledger; adoption requires the fold rule and
+    `gate == pass`.
+  - Tests: `regression` and `not_run` each block adoption; an `informed` slice cannot qualify; result JSON shape.
+  - _Requirements: 5.2, 5.3, 5.4_
+- [ ] 8.3 Engine-evidence arm
+  - `benchmarks/unseen/arms.py`: a score per change from `engine.json` (witness rows on head-side changed
+    functions), no model; plugs into the budget thresholds of 6.4.
+  - Tests on synthetic records: score monotone in witness strength; a change without engine coverage gets no
+    score and is reported, not counted quiet.
+  - _Requirements: 3.4_
+- [ ] 8.4 Model-arm adapter
+  - `arms.py`: candidates from head-side changed functions of stored records, scored by the current decision
+    engine program (and the combiner when its program exists; else its row reads `not_built`); change score is the
+    maximum; metered client, dry run first, response cache, the manifest's `budget_usd` as the hard ceiling.
+  - Tests with a scripted client: dry-run estimate, ceiling stop, cache replay; no model call.
+  - _Requirements: 4.4, 3.4_
+- [ ] 8.5 Long-deadline replay of slice 1 **(coordinator, network)**
+  - 1,300 replays of the `long_deadline` arm, bounded at about 345 s each (up to about 42 hours over three
+    lanes), with `STOP` and resume across sessions; failures re-run once.
+  - Observable: 1,300 records under `~/ousast-results/unseen/p1/1/long_deadline/`, JVM time in each.
+  - _Requirements: 3.1, 3.4, 4.3_
+- [ ] 8.6 Model arms on slice 1 **(coordinator, network; paid: needs a budget go-ahead)**
+  - Dry run gives the candidate count; estimate at the injection slice's measured cost (about $0.008 per
+    candidate at k = 5); the maintainer sets the ceiling against the balance read just before. Then the decision
+    engine arm (and the combiner if built) on the VM.
+  - _Requirements: 4.4, 3.4_
+- [ ] 8.7 Record 08: the arms table **(coordinator)**
+  - `benchmarks/measurements/<date>-unseen-08-arms/record.json`: per arm (hook 30 s, hook 300 s, engine evidence,
+    decision engine, combiner or `not_built`) catch at 1/5/10% budgets with Wilson intervals and realised
+    false-alarm rates, coverage per state, spend; `exploratory` rows in `usage-p1.jsonl`.
+  - Observable: one committed table of arms on slice 1; G1's table exists.
+  - _Requirements: 3.4, 5.2, 5.3_
+
+## Implementation Notes
+
+- 2026-10-04: Tasks 1.1–1.4 implemented offline and independently reviewed. Task 1.5 remains for the
+  coordinator; no networked measurement or commit was performed. Command and API contracts:
+  `benchmarks/unseen/README.md`. Offline evidence:
+  `benchmarks/measurements/2026-10-04-unseen-01-offline/record.json`.
+- Population guard scope is preserved under the maintainer's hard rule: benchmarks/src/tests for
+  populations, plus plane for pools. Each directory exemption is reservation-specific, including private
+  pool manifests. Extending population scope to plane exposed 30 existing references; no historical
+  records were changed or exempted inside the existing population scope.
+- S3 inventory opens through `open_store(read_only=True)` because ordinary store construction writes a
+  Select capability probe. Default store behavior is unchanged; inventory uses LIST/GET only.
+- The freeze writer must match the canonical TOML digest encoding documented in the pool README and
+  exercised by `test_independent_population.py`; final pool discovery does not trust mutable status.
+
+### Group 2 implementation evidence (2026-10-04)
+
+Tasks 2.1–2.4 are implemented offline and independently reviewed. Task 2.5 is
+`python -m benchmarks.unseen.draft --root "$PWD" --s3-store s3:// --seed 20260601`
+with the documented PATH/PYTHONPATH and GH_TOKEN environment. No networked draw,
+label check, group-2 measurement record, guard bisection, or commit was performed.
+The group header remains pending its coordinator exits 2.5/2.6.
+
+The command rebuilds the group-1 used set, resolves renamed/fork repositories,
+uses disposable external blobless clones, re-reads licences at extracted heads,
+and emits counts/digests only. The README documents injected clients, method bias,
+the hard 300-repository floor, and an explicitly unmeasured resource estimate.
+Independent review caught and fixed ambiguous parent-function absence and stale
+candidate-404 handling; regression tests cover both.
+
+Validation: requested pytest selection 103 passed, 1 skipped, exit 0 (population-test
+output discarded); `ruff check .` exit 0; `ruff format --check .` exit 0 (411 files);
+`mypy src/openultrasast` exit 0 (160 source files). The CLI `--help` smoke exited 0.
+Source-read instrumentation printed 20,101 bytes for the extraction implementation.
+
+### Groups 4–6 offline implementation (2026-10-04)
+
+Tasks 4.1–4.3, 5.1–5.3 and 6.1–6.4 are implemented and independently reviewed.
+Tasks 4.4 and 5.4 remain coordinator-owned network proofs; group headers 4 and 5 stay
+pending those exits. No commit, network request, Docker, AX or kubectl command was run.
+The sourcing job's draft.py, source.py, extract.py and journal.py are unchanged.
+
+The shared dispatcher retains the engine adapter's historical questions.json object
+name to preserve its unchanged assertions; replay uses spec.dat and result.dat.
+The engine Dockerfile body is unchanged except its required AS engine alias; CI builds
+engine and replay separately. S3Store's read_only class default preserves existing
+constructor-bypassing transport fixtures; initialized read-only stores still refuse writes.
+
+Exact coordinator commands and input contracts are in benchmarks/unseen/README.md,
+including frozen TOML/private pointer verification, scoring joins, own-history proof
+preparation and anonymous measurement-record generation. JVM wall time is the hook's
+aggregate scan build/query timing and is labelled as such. Unsupported-only partitions
+with explicit language_not_covered evidence do not pretend to have launched a JVM.
+
+Record 06 contains the committed synthetic fixture and scorer-output digests. The
+reserved-repository guard was run as a black box with both output streams discarded.
+Independent review: APPROVED; all reported integration and coverage defects resolved.
+
+Final verification: requested pytest command 279 passed, 8 skipped, exit 0; separate
+`tests/test_ax_batch.py` 5 passed, exit 0; `ruff check .`, `ruff format --check .`,
+`mypy src/openultrasast`, and the output-discarded reserved-repository guard all exit 0.
+The own-history proof preparer and replay dry run read five items and exit 0.

.kiro/steering/overview.md

@@ -1,8 +1,23 @@
 # Overview — what this is, and what is actually true of it
 
-## Current state, 2026-10-02
+## Current state, 2026-10-04
+
+Everything below this section is dated history; where it disagrees with this section, this section wins. The
+direction lives in `roadmap.md`, section of 2026-10-04 (phases G0-G5).
+
+- **Goal.** A pre-push safety net that works on repositories it has never seen. Two numbers per push decide it:
+  catch rate on real vulnerability-introducing changes, and false-alarm rate on ordinary changes. Neither has been
+  measured yet; the unseen-repository pool (G0, spec `unseen-repo-evaluation`) comes first.
+- **Correction, 2026-10-04.** Infrastructure and engine work had drifted from that goal. It is now done only when
+  it removes a bottleneck for those measurements.
+- **Engine.** Joern runs on the operator's kube-ax cluster as AX Tasks, with a VM fallback lane; kind stays the
+  development profile. It is almost blind to fixes: paths differ between sides in 2 of 105 both-asked pairs.
+- **Decision engine.** Ranks but cannot block (exp-003 to exp-005 inconclusive; 710 fix pairs in memory). Fix
+  mechanisms is one G2 arm, paused after mining until the pool exists.
+- **Hook.** `ousast pre-push` is operational and advisory on any repository; rollout stays NO-GO.
+
+## State, 2026-10-02 (superseded by the section above)
 
-Everything below this section is dated history; where it disagrees with this section, this section wins.
 User-facing documentation: `README.md`, `docs/architecture.md`, `docs/evaluation.md`, `docs/decision-engine.md`,
 `ops/ax/README.md`.
 

.kiro/steering/roadmap.md

@@ -1,5 +1,100 @@
 # Roadmap
 
+## Roadmap to detection with the decision engine on kube-ax (2026-10-04)
+
+This section supersedes the ordering below it. The older sections stay as history.
+
+### Destination
+
+On every push, from a hook or CI, OpenUltraSAST answers for the changed code. The answer is advisory for every
+family, and BLOCK only for families whose precision has been proven on repositories the system never saw. The work
+is split by what each place is good at:
+
+- **kube-ax cluster** (operator k3s, Agent Substrate v0.3.0, AX v0.3.1, 2 nodes): the Joern engine, as AX Tasks.
+  No API keys and no store credentials in the cluster; tasks read and write through short-lived presigned links.
+- **The VM or CI runner**: candidate selection, the decision engine's model calls (DeepSeek), the memory store,
+  and the fallback engine lane (2.5 GB heap) for pins that do not fit the cluster's 1.7 GB heap.
+- **kind** stays the local development profile, with the same code and manifests.
+
+### The question everything answers
+
+Does the safety net work on repositories it has never seen? That is the purpose of the A/B experiments, and every
+phase below is judged by it. On an unseen repository almost every push is an ordinary change, so two numbers decide
+whether the safety net is useful, measured on the hook's advisory output exactly as a user would see it:
+
+1. **Catch rate:** on real vulnerability-introducing changes in unseen repositories, how often it flags the change.
+2. **False-alarm rate:** on ordinary changes in the same unseen repositories, how often it flags anything.
+
+Within-pair AUC on known fix pairs answers neither. It stays a development signal, not the goal.
+
+### Where we are (2026-10-04)
+
+| Area | State | Evidence |
+| --- | --- | --- |
+| Generalisation measured so far | The classifier ranks above chance on repository-held-out folds (within-pair AUC about 0.76) but cannot reach the block bar; the engine-based scan failed fresh populations (v1 1/11, v2 0/17); push-level catch and false-alarm rates on unseen repositories have never been measured | exp-003/004/005, `results-v1.json`, `results-v2.json` |
+| Pre-push hook | Operational and advisory on any repository | `2026-10-02-pre-push-operational-1/record.json` |
+| Labelled data | 710 complete real fix pairs | `2026-10-04-decision-engine-harvest-3/record.json` |
+| Engine | Almost blind to fixes (paths differ between sides in 2 of 105 both-asked pairs) | `2026-10-04-joern-slice-leak-audit/record.json` |
+| Infrastructure | Engine runs on kube-ax as AX Tasks with a VM fallback; kind for development | memory `kube-ax-cluster` |
+
+Correction recorded 2026-10-04: the previous two days went mostly into infrastructure and engine internals that did
+not move a generalisation number. From here, infrastructure and engine work is done only when it removes a
+bottleneck for the measurements below.
+
+### Phases
+
+**G0. The unseen-repository pool and the push-level metric** (new spec `unseen-repo-evaluation`)
+
+1. A frozen pool of repositories never used for training, tuning, mining or harvesting, and not in any population.
+   Two kinds of change per repository: real vulnerability-introducing commits (the parent of a fix, post-cutoff where
+   possible) and ordinary commits from the same history.
+2. The metric: replay each change through `ousast pre-push` (and the decision engine where an arm uses it) and
+   record catch rate and false-alarm rate, with confidence intervals, per family and overall.
+3. Rotation: the pool is spent per decision, like the populations; a new slice is drawn for the next one.
+
+Exit: the pool is frozen and the current production hook has a measured baseline on it.
+
+**G1. Re-score the arms we have on the pool**
+
+Current hook (quick rules + engine), decision-engine advisory (current programs), combiner, engine evidence. Each is
+an arm with a registered decision rule, scored by catch rate at a fixed false-alarm budget.
+
+Exit: one measured table of arms on unseen code; keep only what generalises.
+
+**G2. Improve what moves the unseen-repo numbers**
+
+Candidates, each admitted only if it improves the pool metric on a fresh slice: fix mechanisms in the engine
+(approved plan, now one arm, not the spine), engine coverage and TypeScript, decision-engine prompt and retrieval
+changes, more labelled pairs from more distinct repositories.
+
+**G3. Decide BLOCK per family** on the pool's false-alarm rate and the pooled precision bound (Req 6.4).
+
+**G4. Detection on push for any repository** (`plane-on-kubernetes` P4 work: `ousast plane scan --base --head`,
+engine pins on kube-ax, model calls on the VM or CI), sized by what G1-G3 kept.
+
+**G5. Qualification and rollout:** adopt, commit `prediction-v3.json`, spend population v3 once, advisory rollout,
+BLOCK only for families that passed.
+
+### Cross-cutting rules
+
+- No hand-tuning per miss. Every detection change is a registered experiment or an admission by held-out
+  separation.
+- Records carry counts, digests and pattern names, never repository names: the reserved-repository guard covers
+  every population.
+- Instrument first: a zero from an unread input is a failed run (AGENTS.md).
+- Cluster stability: sandbox deaths are retried once, then sent to the VM lane; the operator owns workers, egress
+  and Substrate.
+
+### Spec map
+
+| Spec | Role in this roadmap | State |
+| --- | --- | --- |
+| `unseen-repo-evaluation` (new) | G0, G1: the pool and the push-level metric | To specify |
+| `learned-decision-engine` | G1-G3, G5; fix-mechanisms plan is one G2 arm | Implementation; experiments 3-5 recorded |
+| `plane-on-kubernetes` | G4; the engine executors for kube-ax | Groups 1-2 done (kind); production path re-scoped to the operator's kube-ax (AX Tasks), groups 3-8 to re-plan |
+| `pre-push-safety-net` | the hook replayed in G0-G1; M4-M5 in G5 | Operational increment 1 done; M4-M5 open |
+| `ai-service-plane` | The plane the above runs on | Complete |
+
 ## Status, 2026-10-02
 
 The milestone table below dates from 2026-09-14. Since then (`.kiro/specs/pre-push-safety-net/release-milestones.md`):

Dockerfile

@@ -52,7 +52,7 @@ COPY benchmarks ./benchmarks
 # The core install is zero-dependency by design; `semantic` adds the tree-sitter grammars the candidate
 # enumerator uses. The LLM endpoint stays optional and is configured by environment, never baked in.
 RUN python3 -m venv /venv \
- && /venv/bin/pip install --no-cache-dir -e ".[semantic]"
+ && /venv/bin/pip install --no-cache-dir -e ".[semantic,s3]"
 ENV PATH="/venv/bin:${PATH}"
 
 # Analysis never needs root.

README.md

@@ -1,38 +1,53 @@
 # OpenUltraSAST
 
-OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
-repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
-and separates what it can prove from what it merely suspects. A scan combines language-scoped
-pattern rules, a tree-sitter overlay, a Joern code property graph and, optionally, a model that is
-asked only the question the graph cannot settle; every finding carries the evidence that earned
-its place. A scan that could not look at something records a degradation instead of reporting a
-clean result. `ousast` is the interface; model work over whole repositories runs on a separate
-agentic plane (google/ax), and a learned decision engine is in development. Neither is needed
-for a scan.
+OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
+repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
+keeps what it can prove apart from what it only suspects.
 
-This file is the front door; each section links to its page of the documentation site (`docs/`,
-`mkdocs.yml`; `uv sync --extra docs && uv run mkdocs serve`).
+A scan combines up to four parts:
+
+- language-scoped pattern rules;
+- a tree-sitter overlay (a syntax tree of each file, built by the tree-sitter parser);
+- a Joern code property graph (a graph of the program's syntax, control flow and data flow);
+- optionally, a model. It is asked only the question the graph cannot settle.
+
+Every finding carries the evidence that earned its place. When a scan could not look at
+something, it records a degradation. It does not report a clean result.
+
+`ousast` is the interface. Model work over whole repositories runs on a separate agentic plane
+(google/ax). A learned decision engine is in development. A scan needs neither of them.
+
+This file is the front door. Each section links to its page of the documentation site.
+The site is published at <https://norandom.github.io/OpenUltraSAST/>. Its sources are in `docs/` and
+`mkdocs.yml`; to preview it locally, run `uv sync --extra docs && uv run mkdocs serve`.
 
 ## Status
 
-**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate:
-no hook capability is enabled, and the two independent populations spent so far gave engine recall
-1 of 11 and then 0 of 17. The learned decision engine ranks but may not block: BLOCK is offered for
-no family. The project has stopped trying to reach the gates by hand; every detection change now
-has to win a controlled measurement on held-out repositories. The one page that states all of this
-with the record behind each figure is [docs/where-we-stand.md](docs/where-we-stand.md); the
-release history is in [RELEASE_NOTES.md](RELEASE_NOTES.md).
+**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate.
+
+- No hook capability is enabled.
+- Two independent populations (case sets frozen before any scan) have been spent so far. Engine
+  recall was 1 of 11 and then 0 of 17.
+- The learned decision engine ranks but may not block. BLOCK is offered for no family.
+
+The project has stopped trying to reach the gates by hand. Every detection change now has to win
+a controlled measurement on held-out repositories. One page states all of this, with the record
+behind each figure: [docs/where-we-stand.md](docs/where-we-stand.md). The release history is in
+[RELEASE_NOTES.md](RELEASE_NOTES.md).
 
 ## Install and first scan
 
-Which tier you need: a deterministic scan of a checkout or a CI gate is tier 1; the Joern engine
-(`standard`, `deep`) is tier 2; running model work over whole repositories on the plane is tier 3,
-for maintainers; tier 4 does not exist yet.
+Pick the tier you need:
+
+1. Tier 1: a deterministic scan of a checkout, or a CI gate.
+2. Tier 2: the Joern engine (`standard`, `deep`).
+3. Tier 3, for maintainers: model work over whole repositories on the plane.
+4. Tier 4 does not exist yet.
 
 ### Tier 1: quick scan, no Docker
 
 The core install depends only on PyYAML. From a checkout, `uv sync` (or `pip install .`) installs
-the `ousast` entry point; `uv sync --group dev` adds the test and lint tools.
+the `ousast` entry point. `uv sync --group dev` adds the test and lint tools.
 
 ```bash
 uv sync
@@ -41,19 +56,41 @@ uv run ousast scan . --mode quick --fail-on verified   # CI: exit 1 on any evide
 ```
 
 This runs the language-scoped pattern rules, entry-point reachability hints, ranking,
-verification and scoring: deterministic, reproducible, no network, no keys. What it cannot do:
-there is no engine, so nothing follows a value across statements or files, and the PHP quick rules
-are a low-precision, in-sample development set (per-rule precision lower bounds from 0.03 to 1.0 on
-the real pins, `benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). The optional
-`semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for the overlay.
+verification and scoring. It is deterministic and reproducible. It needs no network and no keys.
+
+The same install gives you the advisory pre-push hook for any repository. It runs the quick rules
+on the lines a push changes, names every check it could not run, and never blocks a push:
+
+```bash
+cd /path/to/your/repository && ousast pre-push install    # remove: ousast pre-push uninstall
+```
+
+What it cannot do:
+
+- There is no engine. Nothing follows a value across statements or files.
+- The PHP quick rules are a low-precision, in-sample development set. Per-rule precision lower
+  bounds run from 0.03 to 1.0 on the real pins
+  (`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
+
+The optional `semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for
+the overlay.
 
 ### Tier 2: standard and deep scans (Docker)
 
-`standard` and `deep` need Joern. The Docker image `openultrasast:dev` ships the tool, Joern
-v4.0.625 and `php-cli` (needed by the PHP frontend) in one image; it is built locally from the
-`Dockerfile` (no published image yet), and it is big: a JRE plus about 2 GB of Joern. The shell
-wrapper builds it on first use and then runs `ousast` in the container with your directory mounted
-read-only and the container network off:
+`standard` and `deep` need Joern. The Docker image ships three things in one image: the tool,
+Joern v4.0.625 and `php-cli` (the PHP frontend needs it). Each release publishes it to GHCR. It is
+big: a JRE plus about 2 GB of Joern. Pull it instead of building, and tag it with the name the
+shell wrapper uses:
+
+```bash
+docker pull ghcr.io/norandom/openultrasast:2.0.1
+docker tag ghcr.io/norandom/openultrasast:2.0.1 openultrasast:dev   # or build: docker compose build
+```
+
+The shell wrapper runs `ousast` in the container. It builds the image only if `openultrasast:dev`
+is missing. Your directory is mounted read-only and the container network is off. For the pre-push
+hook, use `ousast pre-push install --docker`: Git cannot call a shell function, so it installs the
+script `ousast-docker` beside the hook.
 
 ```bash
 source /path/to/OpenUltraSAST/ops/shell/ousast.sh   # PowerShell: ops/shell/ousast.ps1
@@ -62,90 +99,132 @@ ousast-with-judge scan . --mode standard            # network on, the model is a
 ```
 
 `docker compose build` and `TARGET=/path/to/repo docker compose run --rm ousast scan /target` do
-the same without the wrapper. `deep` adds REGRESS, which runs each reproduction snippet with
-`docker run` from wherever the scan runs; the engine container has no Docker socket, so run `deep`
-from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
-verified) and a working `docker`, or accept the recorded `sandbox_unavailable` degradation.
+the same without the wrapper.
+
+`deep` adds REGRESS. REGRESS runs each reproduction snippet with `docker run` from wherever the
+scan runs. The engine container has no Docker socket. So run `deep` in one of two ways:
+
+- from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
+  verified) and a working `docker`;
+- or accept the recorded `sandbox_unavailable` degradation.
+
 Details, including the host install, are in [ops/README.md](ops/README.md).
 
-Keys and store settings live in `.env` in the working directory (gitignored; `.env.example` lists
-the names). `DEEPSEEK_API_KEY` serves every LLM call; `OPENROUTER_API_KEY` is used for embeddings
-only (`ousast learn memory embed`). `.env` never overrides a variable already exported in your
-shell, so `unset` a stale export when a key change seems to have no effect. The plane's memory
-store defaults to a local file store under `~/ousast-results/plane/memory`; an S3-compatible bucket
-is optional (`uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT` and the
-AWS credentials in `.env`). Nothing in a scan needs it.
+Keys and store settings live in `.env` in the working directory. The file is gitignored;
+`.env.example` lists the names.
+
+- `DEEPSEEK_API_KEY` serves every LLM call.
+- `OPENROUTER_API_KEY` is used for embeddings only (`ousast learn memory embed`).
+- `.env` never overrides a variable already exported in your shell. If a key change seems to have
+  no effect, `unset` the stale export.
+- The plane's memory store defaults to a local file store under `~/ousast-results/plane/memory`.
+- An S3-compatible bucket is optional: `uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>`
+  with `S3_ENDPOINT` and the AWS credentials in `.env`. Nothing in a scan needs it.
 
 ### Tier 3: the plane on ax (maintainers)
 
-None of this is needed for tier 1 or 2. Today's profile is a single-node kind cluster on the
-maintainer's host. `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the
-PATH (it refuses to start when one is missing) and builds Agent Substrate and ax from their
-checkouts under `~/.cache/ousast/ax-src/`. Measured idle footprint with two workers: the kind node
-takes 1.7 GiB of memory (27% of the 7.7 GiB host) and 4.9 GB of disk for images, with several GB
-more of Go caches during the build; the worker pool is sized for that 7 GB host. The verify and
-roles tasks need `DEEPSEEK_API_KEY`, and memory lives in the local file store or the S3 store
-(RustFS is the tested server). Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
+Tier 1 and 2 need none of this. Today's profile is a single-node kind cluster on the maintainer's
+host.
+
+- `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the PATH. It refuses to
+  start when one is missing.
+- It builds Agent Substrate and ax from their checkouts under `~/.cache/ousast/ax-src/`.
+- Measured idle footprint with two workers: the kind node takes 1.7 GiB of memory (27% of the
+  7.7 GiB host) and 4.9 GB of disk for images. The build uses several GB more for Go caches.
+- The worker pool is sized for that 7 GB host.
+- The verify and roles tasks need `DEEPSEEK_API_KEY`.
+- Memory lives in the local file store or the S3 store. RustFS is the tested server.
+
+Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
 
 ### Tier 4: Kubernetes (k3s), planned
 
-No deployment to any cluster other than that kind cluster has been made. The target is two
-profiles of one code base, `kind` locally and `k3s` in production, where the same `ousast plane run`
-submits a Run in a remote execution mode: it needs a kubeconfig context or the cluster's ingress
-(and an `imagePullSecret` on the worker ServiceAccount for a private package), runner and engine
-images published to GHCR and pinned by digest, and the S3 store as the artifact path, with no
-receiver on a laptop. The plan, and the code changes it needs with file paths, are in
+Nothing has been deployed to any cluster other than that kind cluster. The target is two profiles
+of one code base: `kind` locally and `k3s` in production. The same `ousast plane run` would
+submit a Run in a remote execution mode. It needs:
+
+- a kubeconfig context or the cluster's ingress (and an `imagePullSecret` on the worker
+  ServiceAccount for a private package);
+- runner and engine images published to GHCR and pinned by digest;
+- the S3 store as the artifact path, with no receiver on a laptop.
+
+The plan, and the code changes it needs with file paths, are in
 [docs/deployment.md](docs/deployment.md#production-topology-ax-on-a-kubernetes-cluster).
 
 ## Scan modes
 
-`quick` runs the pattern rules, hints, ranking, verification and score and needs nothing.
-`standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and the
-Joern model layer, which builds one code property graph per repository and decides taint,
-guard-dominance and configuration questions on it; a model is asked only the residual question
-(without a key it is skipped and only the graph's entailed findings are reported). `deep` adds
-REGRESS: promoted candidates are loaded by a small snippet in a Docker sandbox and get a
-`triggerable` or `not_triggerable` verdict. A missing engine is a recorded degradation
-(`cpg_unavailable`, `sandbox_unavailable`), never a clean result.
+There are three modes. Each adds to the one before.
+
+- `quick` runs the pattern rules, hints, ranking, verification and score. It needs nothing.
+- `standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and
+  the Joern model layer. The model layer builds one code property graph per repository. It decides
+  taint, guard-dominance and configuration questions on that graph. A model is asked only the
+  residual question. Without a key, that question is skipped and only the graph's entailed
+  findings are reported.
+- `deep` adds REGRESS. Promoted candidates are loaded by a small snippet in a Docker sandbox. Each
+  gets a `triggerable` or `not_triggerable` verdict.
+
+A missing engine is a recorded degradation (`cpg_unavailable`, `sandbox_unavailable`), never a
+clean result.
+
+`--fail-on never|findings|verified|worth-fixing` sets the exit code. `--config openultrasast.toml`
+loads settings. Every run writes these files under `<target>/.openultrasast/runs/<scan-id>/`:
+`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`, `report.sarif`
+and `trace/events.jsonl`.
+
+`ousast pre-push` (experimental) analyses the commits a push would publish against their base.
+First, the quick rules run on the changed lines. Then the engine compares head with base, and only
+new or worsened defects become candidates. It is advisory by default, and it prints one line per
+check it could not run. Its capability registry is empty, so it emits no normal alert today. An
+engine finding the push introduced is shown as advisory. Install, settings, skip reasons and a CI
+recipe: [docs/scanning.md](docs/scanning.md#first-run-install-the-hook-on-any-repository).
 
-`--fail-on never|findings|verified|worth-fixing` sets the exit code and `--config openultrasast.toml`
-loads settings. Every run writes `manifest.json`, `findings.json`, `verification.json`,
-`score.json`, `report.md`, `report.sarif` and `trace/events.jsonl` under
-`<target>/.openultrasast/runs/<scan-id>/`. `ousast pre-push` (experimental) analyses the commits a
-push would publish against their base so that only new or worsened defects are candidates; it is
-advisory by default and its capability registry is empty, so no normal alert is emitted today.
 Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md).
 
 ## How detection works
 
-Three representations, from cheap to expensive: text (regex quick rules, one statement at a time),
-tree (the tree-sitter CST and an IR per function, for the overlay and the obligations) and graph
-(the Joern CPG, the only one that follows a value across statements, calls and files). Taint runs
-as fact tables, queries and verdicts on the graph; absence bugs (an operation that should have
-been guarded) are found as obligations, dominance and configuration census; a model answers only
-the residual question. [docs/detection-techniques.md](docs/detection-techniques.md) explains each
-technique with the code it comes from and what it misses;
-[docs/architecture.md](docs/architecture.md) gives the stage order, the evidence ladder (a state
-machine, not a label a model may assign), the central CWE policy and the project score, and the
-self-improving loops (`ousast improve`, with `--memory` for proposals from plane rows).
+The tool uses three representations of code, from cheap to expensive:
 
-Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP, and
-run only against files of their language. The detection gate (`uv run python -m openultrasast.gate`)
-enforces at least 90% recall and under 10% false positives per language on the bundled cheat-sheet
-corpora; on this tree it reports 93.62% recall (44/47) and 0.0% false positives. Those corpora show
-that a rule change did not break a known detection; they are not a recall estimate on real code,
-and PHP is not in the gate. Framework knowledge (WordPress, Flask, Django, Express, Spring) is
-tagged as an optional prior in `src/openultrasast/ruleset/frameworks.toml`, so its contribution can
-be measured and switched off.
+1. Text: regex quick rules, one statement at a time.
+2. Tree: the tree-sitter CST (concrete syntax tree) and an IR (intermediate representation) per
+   function. The overlay and the obligations use it.
+3. Graph: the Joern CPG. It is the only one that follows a value across statements, calls and
+   files.
+
+Taint runs as fact tables, queries and verdicts on the graph. Absence bugs are operations that
+should have been guarded. They are found as obligations, dominance and configuration census. A
+model answers only the residual question.
+
+- [docs/detection-techniques.md](docs/detection-techniques.md) explains each technique, with the
+  code it comes from and what it misses.
+- [docs/architecture.md](docs/architecture.md) gives the stage order and the evidence ladder (a
+  state machine, not a label a model may assign). It also covers the central CWE policy, the
+  project score and the self-improving loops (`ousast improve`, with `--memory` for proposals from
+  plane rows).
+
+Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP. Each
+rule runs only against files of its language.
+
+The detection gate (`uv run python -m openultrasast.gate`) enforces at least 90% recall and under
+10% false positives per language on the bundled cheat-sheet corpora. On this tree it reports
+93.62% recall (44/47) and 0.0% false positives. Those corpora show that a rule change did not
+break a known detection. They are not a recall estimate on real code, and PHP is not in the gate.
+
+Framework knowledge (WordPress, Flask, Django, Express, Spring) is tagged as an optional prior in
+`src/openultrasast/ruleset/frameworks.toml`. So its contribution can be measured and switched off.
 
 ## The plane on ax
 
-Model-driven work over whole repositories runs as a Run manifest, a DAG of ax Tasks, each in its
-own gVisor actor on Agent Substrate. ax is the only agentic executor; there is no local
-subprocess path. Each Task binds at most one Model and its own `usd`/`calls` budget, egress is
-deny-by-default per Task (hostnames only), and the provider key travels only in the Task's start
-request, never in a manifest or an image. Measured on the first increment: the plane agreed on 16
-of 20 declared sites at $0.0205 per candidate.
+Model-driven work over whole repositories runs as a Run manifest. A Run is a DAG of ax Tasks. Each
+Task runs in its own gVisor actor (an isolated sandbox) on Agent Substrate. ax is the only agentic
+executor; there is no local subprocess path.
+
+- Each Task binds at most one Model and its own `usd`/`calls` budget.
+- Egress is deny-by-default per Task, by hostname only.
+- The provider key travels only in the Task's start request, never in a manifest or an image.
+
+Measured on the first increment: the plane agreed on 16 of 20 declared sites at $0.0205 per
+candidate.
 
 ```bash
 uv run ousast plane doctor                               # kind, Agent Substrate, ax controller, runner image
@@ -155,42 +234,61 @@ uv run ousast plane remember validation-46               # ingest the run's rows
 ```
 
 `workspaces`, `harvest`, `alerts-engine` and `memory-normalise` generate Runs, their inputs and
-the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md); what runs
-where today and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
+the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md). What runs
+where today, and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
 
-HarnessX, the earlier agentic extra, was retired 2026-09-30: a `[harnessx]` section is ignored
-with one warning, and `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail
-naming the plane replacement, as do config keys nothing ever read. Fusion is deterministic.
+HarnessX, the earlier agentic extra, was retired 2026-09-30. A `[harnessx]` section is ignored
+with one warning. `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail and
+name the plane replacement. Config keys that nothing ever read fail the same way. Fusion is
+deterministic.
 
 ## Memory
 
-One store, keyed by repository and pin, holds what plane runs and the decision engine learned:
-rows (facts, verdicts, unit costs, alerts, coverage, feature records) and content-addressed blobs
-(excerpts, embeddings, cached responses, compiled programs). `OUSAST_MEMORY` selects the backend:
-a local file store, or `s3://<bucket>` on any S3-compatible server with versioning, lifecycle
-rules, object tags and S3 Select; RustFS is the tested server. An admin sets the bucket up once;
-the store verifies it at every start and refuses to run, naming each missing piece, rather than
-fall back. Layout and the read and write paths: [docs/memory.md](docs/memory.md); the bucket
-policy and admin steps: [docs/rustfs.md](docs/rustfs.md); how stored rows are meant to turn into
-better decisions: [docs/memory-and-detection.md](docs/memory-and-detection.md).
+One store holds what plane runs and the decision engine learned. It is keyed by repository and pin.
+It holds two kinds of data:
+
+- rows: facts, verdicts, unit costs, alerts, coverage and feature records;
+- content-addressed blobs: excerpts, embeddings, cached responses and compiled programs.
+
+`OUSAST_MEMORY` selects the backend. It is either a local file store or `s3://<bucket>` on any
+S3-compatible server. The S3 server must support versioning, lifecycle rules, object tags and S3
+Select. RustFS is the tested server.
+
+An admin sets the bucket up once. The store verifies it at every start. If anything is missing, it
+refuses to run and names each missing piece. It does not fall back.
+
+- Layout and the read and write paths: [docs/memory.md](docs/memory.md).
+- The bucket policy and admin steps: [docs/rustfs.md](docs/rustfs.md).
+- How stored rows are meant to turn into better decisions:
+  [docs/memory-and-detection.md](docs/memory-and-detection.md).
 
 ## Decision engine and evaluation
 
-A learned decision engine is being built to replace hand-tuned detection edits: every instrument
-produces signals, never verdicts; labels come from ground truth only; a compiled AI classifier with
-local memory turns retrieved similar cases and the candidate's signals into a calibrated
-probability with BLOCK and ADVISORY operating points. It is **in development, not adopted**: no
-scan, pre-push check or report uses it. Measured on six families, on repositories the compiled
-program was not shown, the pooled AUC of its score runs from 0.77 (injection) to 0.96 (access
-control); the stricter within-pair AUC is 0.72 to 0.93; BLOCK is offered for no family
-(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Maintainer surface:
-`ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`. Design, slices and
-the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
+A learned decision engine is being built. It is meant to replace hand-tuned detection edits. It
+follows three rules:
 
-How anything is measured on code the tool was not tuned on, the pair corpora, the independent
-populations and the M4 gates, and the fold discipline that keeps learned decisions out of their
-own training data: [docs/evaluation.md](docs/evaluation.md). Population v3 (PHP, 15 cases) is
-frozen as the one-time final check and nothing in the tree may read it.
+- Every instrument produces signals, never verdicts.
+- Labels come from ground truth only.
+- A compiled AI classifier with local memory turns retrieved similar cases and the candidate's
+  signals into a calibrated probability. It has BLOCK and ADVISORY operating points.
+
+It is **in development, not adopted**. No scan, pre-push check or report uses it.
+
+It was measured on six families, on repositories the compiled program was not shown:
+
+- The pooled AUC (area under the ROC curve) of its score runs from 0.77 (injection) to 0.96
+  (access control).
+- The stricter within-pair AUC is 0.72 to 0.93 across the same families.
+- BLOCK is offered for no family
+  (`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
+
+Maintainer surface: `ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`.
+Design, slices and the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
+
+[docs/evaluation.md](docs/evaluation.md) explains how anything is measured on code the tool was not
+tuned on. It covers the pair corpora, the independent populations and the M4 gates. It also covers
+the fold discipline that keeps learned decisions out of their own training data. Population v3
+(PHP, 15 cases) is frozen as the one-time final check. Nothing in the tree may read it.
 
 ```bash
 uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
@@ -201,19 +299,26 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dr
 
 ## Token economics
 
-Model tokens are the one running cost. At runtime the plane routes them: one Model and one budget
-per Task, a metered client that refuses the next call at the ceiling, repository facts computed
-model-free and reused, blobs replayed at $0, triage before the hunt and a tie-break only on
-disputed candidates. In development, 85.5% of the maintainers' coding-session tokens were tool
-input and output rather than reasoning (down from 90%), and the remedies in `benchmarks/dev/`
-target that. Mechanisms,
-the measured costs and the before-and-after table: [docs/token-ergonomics.md](docs/token-ergonomics.md).
+Model tokens are the one running cost. At runtime the plane routes them:
+
+- one Model and one budget per Task;
+- a metered client that refuses the next call at the ceiling;
+- repository facts computed without a model and reused;
+- blobs replayed at $0, with no new model call;
+- triage before the hunt;
+- a tie-break only on disputed candidates.
+
+In development, 85.5% of the maintainers' coding-session tokens were tool input and output rather
+than reasoning. That is down from 90%. The remedies in `benchmarks/dev/` target that share.
+Mechanisms, the measured costs and the before-and-after table:
+[docs/token-ergonomics.md](docs/token-ergonomics.md).
 
 ## Examples, threat model
 
-End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
-What the tool trusts and what it does not (scanned code is untrusted input, secret redaction,
-per-task egress, budgets, the `deep` sandbox): [docs/threat-model.md](docs/threat-model.md).
+- End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
+- What the tool trusts and what it does not: [docs/threat-model.md](docs/threat-model.md). This
+  covers scanned code as untrusted input, secret redaction, per-task egress, budgets and the
+  `deep` sandbox.
 
 ## Contributing and maintainer tooling
 
@@ -228,25 +333,111 @@ uv run python dagger/ci.py    # containerized CI pipeline
 ```
 
 An agent that can run shell commands needs nothing but the CLI. For [OpenCode](https://opencode.ai)
-there are two project skills in `.agents/skills/`: `openultrasast-scan` (modes and their
-prerequisites, `--fail-on`, the evidence ladder, recorded degradations, run artifacts) and
-`openultrasast-triage` (false-positive reasons, what an adjudication may feed, fixing and
-re-scanning). `ousast mcp` runs a narrow MCP server over stdio with ten tools (`openultrasast.scan`,
-`status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`, `explain`,
-`propose_patch`, `export_report`); no tool runs a shell, Docker or a free-form command:
+there are two project skills in `.agents/skills/`:
+
+- `openultrasast-scan`: modes and their prerequisites, `--fail-on`, the evidence ladder, recorded
+  degradations and run artifacts.
+- `openultrasast-triage`: false-positive reasons, what an adjudication may feed, and fixing and
+  re-scanning.
+
+`ousast mcp` runs a narrow MCP (Model Context Protocol) server over stdio. It has ten tools:
+`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`,
+`explain`, `propose_patch` and `export_report`. No tool runs a shell, Docker or a free-form
+command.
 
 ```jsonc
 { "command": "uv", "args": ["run", "ousast", "mcp"] }
 ```
 
 The `kiro-*` skills, `AGENTS.md` and the Kiro-style specifications under `.kiro/` are the
-maintainer's development tooling and can be ignored by users. Other maintainer commands:
-`ousast repos` (pinned known-vulnerable checkouts) and `ousast model candidates` (what the
-candidate enumerator can reach).
+maintainer's development tooling. Users can ignore them. Other maintainer commands:
+
+- `ousast repos`: pinned known-vulnerable checkouts.
+- `ousast model candidates`: what the candidate enumerator can reach.
 
 ## Licence
 
 OpenUltraSAST is licensed under the [Apache License 2.0](LICENSE). Third-party material keeps its
-own terms: Joern (Apache-2.0) is downloaded at image build time, the tree-sitter wheels of the
-`semantic` extra are MIT, and the published benchmark cases were licence-checked one by one (v1's
-OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md)).
+own terms:
+
+- Joern (Apache-2.0) is downloaded at image build time.
+- The tree-sitter wheels of the `semantic` extra are MIT.
+- The published benchmark cases were licence-checked one by one. v1's OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md).
+
+## Engine traces on kube-ax
+
+`benchmarks/learn/engine_trace.py --executor ax` submits AX v0.3.1 Tasks with
+`ax apply -f -` and removes them with `ax delete task NAME`. The operator supplies
+kubeconfig and AX access; the dispatcher creates no Kubernetes resources or secrets.
+The `ousast-engine-task` GHCR package **must be made Public once** in its package
+settings so the actors can pull it without registry credentials.
+
+The task image contains the current engine and worker code. Rebuild it after code
+changes and pass its digest with `--image`; no analyzer download is needed. The host
+materializes each pin as usual and uploads source and prepared questions under
+`engine-queue/<run>/<pin>/`. Only object-scoped presigned URLs and deadline parameters
+enter Task env. Credentials stay on the coordinator. Configure `OUSAST_MEMORY=s3://…`
+and the store credentials in the coordinator's `.env` or environment, with
+`S3_ENDPOINT=https://files.because-security.com`. Presigned links expire after the
+pin deadline plus ten minutes. IP and HTTP endpoints are rejected: the operator's
+actor EgressPolicy must allow **files.because-security.com:443**.
+
+Completion is the result object, even while AX reports Running. Failed phases and
+host deadlines produce explicit failure/timeout rows. Task and staging-object cleanup
+runs on success and failure; cleanup failure stops further dispatch. Per-pin files,
+`--rerun-status`, resume and `progress.json` retain the existing trace-runner behavior.
+AX results include `executor: "ax"`, `worker_ip`, and the image digest. Placement is
+random; `--parallel` defaults to two AX tasks and cannot exceed two.
+
+AX workers use `--heap-profile cluster` with
+`-Xms1700m -Xmx1700m -XX:MaxMetaspaceSize=256m -XX:ReservedCodeCacheSize=128m -XX:MaxDirectMemorySize=256m -Xss512k -XX:ActiveProcessorCount=1 -XX:+UseG1GC`
+on every frontend/query JVM; instruments record the flags. A `JVM out of memory`
+result is automatically re-queued to one local Docker lane with `--heap-profile vm`
+(2560m), recording `fallback: "vm-oom"` and the AX attempt. `mixed-ax` also uses this
+one Docker lane for overflow, prioritizing pending OOM fallbacks. Thus a mixed run has
+up to two AX tasks plus one Docker container. Use `--docker-image` for the locally
+available VM image; Docker never pulls it implicitly. A STOP file prevents new
+launches; interrupted fallback queues are recovered on resume.
+
+Coordinator commands, from this worktree (the AX binary and kubeconfig remain
+configurable). Build/push the actual uncommitted tree and read its digest:
+
+```bash
+cd /home/mc/Source/OpenUltraSAST/.claude/worktrees/engine-ax
+export PATH=/home/mc/Source/OpenUltraSAST/.venv/bin:$PATH
+export PYTHONPATH="$PWD/src"
+TASK_TAG="ghcr.io/norandom/ousast-engine-task:sha-$(git rev-parse --short=7 HEAD)"
+docker buildx build --platform linux/amd64 -f plane/Dockerfile.engine-task \
+  -t "$TASK_TAG" -t ghcr.io/norandom/ousast-engine-task:main \
+  --push --metadata-file /tmp/ousast-engine-task-build.json .
+export TASK_IMAGE="ghcr.io/norandom/ousast-engine-task@$(python -c 'import json; print(json.load(open("/tmp/ousast-engine-task-build.json"))["containerimage.digest"])')"
+printf '%s\n' "$TASK_IMAGE"
+docker build -f Dockerfile -t openultrasast:engine-ax .
+```
+
+The Engine image workflow also builds/pushes both images and prints each digest;
+CI needs the changes published before dispatch. The local build above includes
+uncommitted changes without publishing a git commit.
+
+With `KUBECONFIG` pointing to the operator's kube-ax config and store settings in
+`.env`, run a one-pin smoke using the default experiment units and local example
+memory (or add `--examples /path/to/frozen-examples.jsonl`):
+
+```bash
+: "${KUBECONFIG:?Set KUBECONFIG to the operator kube-ax configuration}"
+python benchmarks/learn/engine_trace.py --executor ax \
+  --ax-bin "$HOME/.local/ax-v0.3.1/ax" --kubeconfig "$KUBECONFIG" \
+  --atespace default --image "$TASK_IMAGE" --docker-image openultrasast:engine-ax \
+  --parallel 1 --limit 1 --deadline 900 \
+  --out "$HOME/ousast-results/plane/engine-ax-smoke"
+```
+
+Then run the mixed lanes (repeat to resume; defaults re-run failed/timeout pins):
+
+```bash
+python benchmarks/learn/engine_trace.py --executor mixed-ax \
+  --ax-bin "$HOME/.local/ax-v0.3.1/ax" --kubeconfig "$KUBECONFIG" \
+  --atespace default --image "$TASK_IMAGE" --docker-image openultrasast:engine-ax \
+  --parallel 2 --deadline 900 --rerun-status failed,timeout \
+  --out "$HOME/ousast-results/plane/engine-ax-mixed"
+```

benchmarks/ax/batch.py

@@ -0,0 +1,434 @@
+"""Host-only AX Task transport; credentials never enter task manifests."""
+
+from __future__ import annotations
+
+import hashlib
+import ipaddress
+import json
+import os
+import queue
+import re
+import shutil
+import subprocess
+import threading
+import time
+import uuid
+from collections import Counter, deque
+from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
+from dataclasses import dataclass, field
+from datetime import timedelta
+from pathlib import Path
+from urllib.parse import urlsplit
+
+import yaml
+
+from openultrasast.config import load_dotenv
+from openultrasast.plane.memory import PRESIGNED_ARCHIVE_SUFFIX, S3Store, open_store
+
+
+class EgressError(ValueError):
+    """Safe diagnostic without presigned URL contents."""
+
+
+class SandboxFailure(Exception):
+    """A terminal AX task without an authoritative result object."""
+
+
+def public_endpoint(url):
+    """Reject endpoints the actor's hostname-based HTTPS egress policy cannot reach."""
+    parts = urlsplit(url)
+    host = parts.hostname or ""
+    try:
+        ipaddress.ip_address(host)
+        is_ip = True
+    except ValueError:
+        is_ip = False
+    if (
+        parts.scheme != "https"
+        or not host
+        or is_ip
+        or "." not in host
+        or host.endswith((".localhost", ".local", ".internal"))
+        or parts.port not in (None, 443)
+        or parts.username
+        or parts.password
+    ):
+        raise EgressError(
+            "AX requires a public HTTPS DNS hostname in S3_ENDPOINT and presigned URLs: "
+            "the egress policy allows the store hostname on port 443 only (no IP or plain HTTP)"
+        )
+    return host.lower()
+
+
+def task_state(text, name):
+    """Accept AX YAML/JSON output and its human-readable tasks table."""
+    try:
+        doc = yaml.safe_load(text)
+    except yaml.YAMLError:
+        doc = None
+    rows = doc.get("items", doc.get("tasks", [doc])) if isinstance(doc, dict) else doc if isinstance(doc, list) else []
+    for row in rows:
+        if not isinstance(row, dict) or row.get("metadata", {}).get("name", row.get("name")) != name:
+            continue
+        status = row.get("status", {})
+        ip = next(
+            (status.get(key) or row.get(key) for key in ("workerIP", "workerIp", "worker_ip", "ip") if status.get(key) or row.get(key)),
+            None,
+        )
+        worker = status.get("worker")
+        if not ip and isinstance(worker, dict):
+            ip = worker.get("ip") or worker.get("address")
+        return str(status.get("phase", row.get("phase", ""))), ip
+    lines = [line.split() for line in text.splitlines() if line.strip()]
+    if lines:
+        headers = [re.sub(r"[^a-z]", "", cell.lower()) for cell in lines[0]]
+        for cells in lines[1:]:
+            if name not in cells:
+                continue
+            row = dict(zip(headers, cells, strict=False))
+            phase = row.get("phase", row.get("status", ""))
+            ip = row.get("workerip") or row.get("ip")
+            if not ip:
+                for cell in cells:
+                    try:
+                        ipaddress.ip_address(cell)
+                        ip = cell
+                        break
+                    except ValueError:
+                        pass
+            return phase, None if ip in {"<none>", "-", ""} else ip
+    listed = isinstance(doc, (dict, list)) or bool(lines and "name" in headers and {"phase", "status"}.intersection(headers))
+    return "Missing" if listed else "", None
+
+
+def bounded_read(read, timeout):
+    """Do not let SDK retries postpone task teardown beyond the pin deadline.
+
+    Only the read runs in a daemon thread; it cannot publish a late result to disk.
+    No executor shutdown waits for an SDK call whose socket has stopped responding.
+    """
+    if timeout <= 0:
+        raise TimeoutError("AX result deadline")
+    replies = queue.Queue(maxsize=1)
+
+    def fetch():
+        try:
+            replies.put((True, read()))
+        except Exception as exc:
+            replies.put((False, exc))
+
+    threading.Thread(target=fetch, daemon=True, name="engine-ax-result-read").start()
+    try:
+        ok, result = replies.get(timeout=timeout)
+    except queue.Empty:
+        raise TimeoutError("AX result deadline") from None
+    if not ok:
+        raise result
+    return result
+
+
+class AXLane:
+    def __init__(self, args, workload, store=None, pause=time.sleep, clock=time.monotonic):
+        self.workload = workload
+        self.args = args
+        self.pause, self.clock = pause, clock
+        self.run = uuid.uuid4().hex
+        load_dotenv()
+        self.host = public_endpoint(os.environ.get("S3_ENDPOINT", ""))
+        # Validate the image before opening a store, whose constructor probes the bucket.
+        validate_image(workload.image)
+        self.store = store if store is not None else open_store()
+        if store is None and not isinstance(self.store, S3Store):
+            raise ValueError("AX requires OUSAST_MEMORY=s3://bucket[/prefix]")
+
+    def __call__(self, item, output, attempt):
+        attempts = []
+        try:
+            done, worker = self._execute_attempt(item, output, attempt, attempts)
+        except SandboxFailure:
+            return {"status": "sandbox_failure", "ax_attempts": attempts}
+        if done is None or done.returncode:
+            return {"status": "instrument_failure", "ax_attempts": attempts}
+        record = json.loads((output / "result.json").read_text())
+        return {**record, "worker_ip": worker, "ax_attempts": attempts}
+
+    def ax(self, *args, manifest=None, timeout=30):
+        env = dict(os.environ)
+        if self.args.kubeconfig:
+            env["KUBECONFIG"] = str(Path(self.args.kubeconfig).expanduser())
+        done = subprocess.run(
+            [str(Path(self.args.ax_bin).expanduser()), *args, *(["--atespace", self.args.atespace] if args[0] != "apply" else [])],
+            input=json.dumps(manifest) if manifest is not None else None,
+            env=env,
+            capture_output=True,
+            text=True,
+            timeout=max(0.01, timeout),
+        )
+        if done.returncode:
+            # AX errors may echo manifest URLs; report only verb and exit code.
+            raise ValueError(f"ax {args[0]} failed (exit {done.returncode})")
+        return done
+
+    def _execute_attempt(self, item, output, attempt, attempts):
+        pin_id = item.id
+        digest = hashlib.sha256(f"{self.run}/{pin_id}/attempt-{attempt}".encode()).hexdigest()[:12]
+        slug = re.sub(r"[^a-z0-9]+", "-", str(pin_id).lower()).strip("-")[:22]
+        name = f"ousast-engine-{slug or 'pin'}-{digest}"
+        key_id = hashlib.sha256(str(pin_id).encode()).hexdigest()
+        prefix = f"engine-queue/{self.run}/{key_id}/attempt-{attempt}/"
+        keys = [prefix + item.object_names.get(key, key.lower() + PRESIGNED_ARCHIVE_SUFFIX) for key in item.inputs]
+        keys.append(prefix + item.result_object)
+        expires = timedelta(seconds=self.workload.deadline + 600)
+        worker_ip = None
+        submitted = False
+        started = self.clock()
+        phase = ""
+        try:
+            for key, data in zip(keys, item.inputs.values(), strict=False):
+                self.store._put(key, data)
+            urls = {env: self.store.presign_get(key, expires) for env, key in zip(item.inputs, keys, strict=False)}
+            urls["RESULT_URL"] = self.store.presign_put(keys[-1], expires)
+            if any(public_endpoint(url) != self.host for url in urls.values()):
+                raise EgressError("AX presign hostname must match S3_ENDPOINT for the store egress policy")
+            document = manifest(name, self.workload, item, urls, self.args.atespace)
+            end = self.clock() + self.workload.deadline
+            # An apply timeout can still have created the task; cleanup must cover it.
+            submitted = True
+            self.ax("apply", "-f", "-", manifest=document, timeout=min(30, self.workload.deadline))
+            while self.clock() < end:
+                result = bounded_read(lambda: self.store._get(keys[-1]), end - self.clock())
+                if self.clock() >= end:
+                    return None, worker_ip
+                phase = ""
+                try:
+                    state = self.ax("get", "tasks", timeout=min(30, end - self.clock())).stdout
+                    phase, ip = task_state(state, name)
+                    worker_ip = ip or worker_ip
+                except (ValueError, OSError, subprocess.SubprocessError):
+                    # The object is authoritative even when AX status is temporarily unavailable.
+                    pass
+                if self.clock() >= end:
+                    return None, worker_ip
+                if result is not None:
+                    self.workload.validate(result[0], output)
+                    return subprocess.CompletedProcess([], 0, "", ""), worker_ip
+                if phase.lower() in {"failed", "missing"}:
+                    # The worker may have uploaded while we were reading AX status.
+                    result = bounded_read(lambda: self.store._get(keys[-1]), end - self.clock())
+                    if self.clock() >= end:
+                        return None, worker_ip
+                    if result is not None:
+                        self.workload.validate(result[0], output)
+                        return subprocess.CompletedProcess([], 0, "", ""), worker_ip
+                    raise SandboxFailure
+                self.pause(min(2, max(0, end - self.clock())))
+            return None, worker_ip
+        except (subprocess.TimeoutExpired, TimeoutError):
+            return None, worker_ip
+        except SandboxFailure:
+            raise
+        except Exception as exc:
+            # Never expose exception strings which may contain object-scoped bearer URLs.
+            detail = str(exc) if isinstance(exc, EgressError) else type(exc).__name__
+            return subprocess.CompletedProcess([], 1, "", "AX transport failed: " + detail), worker_ip
+        finally:
+            attempts.append({"name": name, "phase": phase, "seconds": self.clock() - started})
+            errors = []
+            if submitted:
+                try:
+                    self.ax("delete", "task", name)
+                except Exception as exc:
+                    # A successful list already confirmed a disappeared task is gone.
+                    if phase != "Missing":
+                        errors.append("task: " + type(exc).__name__)
+            for key in keys:
+                try:
+                    self.store._delete(key)
+                except Exception as exc:
+                    errors.append("object: " + type(exc).__name__)
+            if errors:
+                raise RuntimeError("AX cleanup failed; stop dispatching: " + ", ".join(errors))
+
+
+@dataclass
+class Item:
+    id: str
+    inputs: dict[str, bytes]
+    result_object: str
+    command: tuple[str, ...]
+    extra_env: dict[str, str]
+    input_digest: str
+    object_names: dict[str, str] = field(default_factory=dict)
+
+
+@dataclass(frozen=True)
+class Workload:
+    image: str
+    url_env: frozenset[str]
+    deadline: float
+    validate: object
+
+
+def validate_image(image):
+    if not re.fullmatch(r"[^\s@]+@sha256:[0-9a-f]{64}", image):
+        raise ValueError("AX --image must be pinned by sha256 digest")
+
+
+def manifest(name, workload, item, urls, atespace="default", *, validate_name=True):
+    validate_image(workload.image)
+    if validate_name and (not re.fullmatch(r"ousast-engine-[a-z0-9-]+", name) or len(name.encode()) > 49):
+        raise ValueError("invalid AX task name")
+    if set(urls) != workload.url_env:
+        raise ValueError("unexpected URL environment names")
+    # Extra env is deliberately a small numeric execution contract, never inherited credentials.
+    if set(item.extra_env) - {"DEADLINE", "QUESTION_DEADLINE"} or any(
+        not re.fullmatch(r"[0-9]+(?:\.[0-9]+)?", str(v)) for v in item.extra_env.values()
+    ):
+        raise ValueError("unexpected task environment")
+    for url in urls.values():
+        public_endpoint(url)
+    env = [{"name": key, "value": str(value)} for key, value in {**urls, **item.extra_env}.items()]
+    if len(json.dumps(env).encode()) >= 32768:
+        raise ValueError("AX Task env exceeds the 32 KB limit")
+    return {
+        "apiVersion": "ax.io/v1alpha1",
+        "kind": "Task",
+        "metadata": {"name": name, "atespace": atespace},
+        "spec": {"image": workload.image, "env": env, "command": list(item.command)},
+    }
+
+
+def write_json(path, record):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(".tmp")
+    temporary.write_text(json.dumps(record, sort_keys=True) + "\n")
+    temporary.replace(path)
+
+
+def schedule(pending, lanes, out, execute, complete, *, fallbacks=(), overflow=True, progress=None):
+    """The sole progress writer; callbacks carry workload-specific records, never transport."""
+    if sum(lane in {"ax", "kind"} for lane in lanes) > 2 or lanes.count("docker") > 1:
+        raise ValueError("at most two AX lanes and one VM lane")
+    pending, fallbacks = deque(pending), deque(fallbacks)
+    progress = progress if progress is not None else {"pins_done": 0, "pins_total": len(pending) + len(fallbacks)}
+    progress["lanes"] = dict(Counter(lanes))
+    active = {}
+    failures = False
+
+    def persist():
+        progress.update(pins_in_flight=len(active), active_lanes=dict(Counter(lane for _, lane in active.values())))
+        if any(lane in {"ax", "kind"} for lane in lanes):
+            progress["fallbacks_pending"] = len(fallbacks)
+        write_json(out / "progress.json", progress)
+
+    with ThreadPoolExecutor(max_workers=len(lanes)) as pool:
+
+        def launch(lane):
+            if (out / "STOP").exists():
+                return
+            if lane == "docker" and fallbacks:
+                item, previous = fallbacks.popleft()
+            elif pending and (lane != "docker" or overflow):
+                item, previous = pending.popleft(), None
+            else:
+                return
+            active[pool.submit(execute, item, lane, previous)] = item, lane
+
+        for lane in lanes:
+            launch(lane)
+        persist()
+        try:
+            while active:
+                finished, _ = wait(active, return_when=FIRST_COMPLETED)
+                # Resolve ALL completed futures before launching; cleanup exceptions stop launches.
+                results = [(active.pop(future), future.result()) for future in finished]
+                for (item, lane), result in results:
+                    failed, fallback = complete(item, lane, result, progress)
+                    if fallback is not None:
+                        fallbacks.append((item, fallback))
+                    else:
+                        failures |= failed
+                        progress["pins_done"] += 1
+                for lane, count in (Counter(lanes) - Counter(lane for _, lane in active.values())).items():
+                    for _ in range(count):
+                        launch(lane)
+                persist()
+        finally:
+            persist()
+    return failures
+
+
+def dispatch(items, workload, out, executors, *, lanes=("ax", "ax", "docker"), overflow=True):
+    """Digest checkpoints, one instrument rerun, sandbox retry, priority VM fallback."""
+    out.mkdir(parents=True, exist_ok=True)
+    pending, fallbacks = [], []
+
+    def path(item):
+        return out / (hashlib.sha256(item.id.encode()).hexdigest() + ".json")
+
+    for item in items:
+        previous = json.loads(path(item).read_text()) if path(item).exists() else {}
+        if previous.get("input_digest") == item.input_digest:
+            if previous.get("fallback_pending"):
+                fallbacks.append((item, previous))
+                continue
+            if previous.get("done"):
+                continue
+        pending.append(item)
+
+    def execute(item, lane, previous):
+        output = out / hashlib.sha256(item.id.encode()).hexdigest()
+        output.mkdir(exist_ok=True)
+        attempts = []
+        for attempt in range(2):
+            attempt_output = output / ("attempt-" + uuid.uuid4().hex)
+            attempt_output.mkdir()
+            record = executors[lane](item, attempt_output, attempt)
+            for artifact in attempt_output.iterdir():
+                if artifact.is_file():
+                    shutil.copyfile(artifact, output / artifact.name)
+            attempts.append({"lane": lane, "status": record.get("status"), "attempt": attempt, "transport": record.get("ax_attempts", [])})
+            if record.get("status") not in {"sandbox_failure", "instrument_failure"}:
+                break
+        return {**record, "attempts": (previous or {}).get("attempts", []) + attempts, "input_digest": item.input_digest, "done": True}
+
+    def complete(item, lane, record, progress):
+        fallback = lane in {"ax", "kind"} and record.get("status") in {"sandbox_failure", "oom"} and "docker" in lanes
+        record["fallback_pending"] = fallback
+        write_json(path(item), record)
+        progress["last_item"] = hashlib.sha256(item.id.encode()).hexdigest()
+        return record.get("status") not in {"ok", "unanalysable", "quick_only", "engine_covered"}, record if fallback else None
+
+    return schedule(pending, lanes, out, execute, complete, fallbacks=fallbacks, overflow=overflow)
+
+
+class DockerLane(AXLane):
+    """VM lane uses the same presigned-object protocol and pinned replay image."""
+
+    def _execute_attempt(self, item, output, attempt, attempts):
+        # Reuse all object/validation/cleanup logic, replacing only the task runner.
+        import copy
+
+        lane = copy.copy(self)
+        lane.ax = lane._docker_command
+        lane._docker_name = None
+        return AXLane._execute_attempt(lane, item, output, attempt, attempts)
+
+    def _docker_command(self, *args, manifest=None, timeout=30):
+        if args[0] == "apply":
+            self._docker_name = manifest["metadata"]["name"]
+            command = ["docker", "run", "-d", "--name", self._docker_name, "--entrypoint", manifest["spec"]["command"][0]]
+            for env in manifest["spec"]["env"]:
+                command += ["-e", env["name"] + "=" + env["value"]]
+            command += [manifest["spec"]["image"], *manifest["spec"]["command"][1:], "--heap-profile", "vm"]
+        elif args[0] == "get":
+            command = ["docker", "inspect", "--format", "{{.State.Running}} {{.State.ExitCode}}", self._docker_name]
+        else:
+            command = ["docker", "rm", "-f", self._docker_name or args[2]]
+        done = subprocess.run(command, capture_output=True, text=True, timeout=timeout)
+        if done.returncode:
+            raise ValueError("docker command failed")
+        if args[0] == "get":
+            phase = "Running" if done.stdout.startswith("true") else "Failed"
+            done.stdout = json.dumps({"metadata": {"name": self._docker_name}, "status": {"phase": phase}})
+        return done

benchmarks/experiments/exp-003-precision-bound-blocking/registration.json

@@ -0,0 +1,39 @@
+{
+  "experiment": "exp-003-precision-bound-blocking",
+  "registered": "2026-10-03",
+  "store": "s3://sast-memory",
+  "manifest": "plane/experiments/exp-003-precision-bound-blocking.yaml",
+  "manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
+  "pin": "dd7ebc119b5ad8e77c05d418eed9cbd46e193587",
+  "units_status": "frozen",
+  "units_file": "plane/experiments/exp-003-precision-bound-blocking.units.jsonl",
+  "units_digest": "338de998b1cfa36857cd793dbc4cf7fd0a13ab517a91f6790335881e941b3006",
+  "units": {
+    "total": 353,
+    "paired": 244,
+    "groups": 146,
+    "by_family": {
+      "access_control": {"units": 41, "positives": 25, "paired": 32, "paired_positives": 16, "groups": 24},
+      "deserialization": {"units": 48, "positives": 31, "paired": 34, "paired_positives": 17, "groups": 31},
+      "injection": {"units": 119, "positives": 63, "paired": 76, "paired_positives": 38, "groups": 34},
+      "output_encoding": {"units": 45, "positives": 29, "paired": 32, "paired_positives": 16, "groups": 27},
+      "path": {"units": 39, "positives": 23, "paired": 32, "paired_positives": 16, "groups": 23},
+      "untrusted_destination": {"units": 61, "positives": 37, "paired": 38, "paired_positives": 19, "groups": 36}
+    },
+    "note": "frozen model-free on the 937-example static memory; the outer folds (k 5, seed 0, compile fraction 0.25) are the incumbent evaluation's; injection and access_control equal exp-002's 160 units"
+  },
+  "kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
+  "zero_cost": "responses replayed from the response cache only, behind a meter with zero USD and call ceilings around a refusing client; a unit with any uncached response is excluded and counted; the run is void unless the meter reads 0 calls and $0",
+  "arms": {
+    "A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds (slope CI contains 1, intercept CI contains 0, ECE <= 0.05); BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
+    "B": "selective blocking: per family and outer fold, threshold chosen on the family's paired units in the other four folds as the lowest raw score t with Wilson 95% LB of flagged precision >= 0.95 (unsure verdicts never flag); none -> no BLOCK on that fold",
+    "C": "secondary, non-deciding: as B with point precision >= 0.95 over at least 10 training flags"
+  },
+  "primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision, per family, arm B, paired population",
+  "populations": {"primary": "paired (both sides of the candidate present; excludes every positives-only group and any unpaired unit)", "secondary": ["pooled (every unit)"]},
+  "secondary_metrics": ["flagged_precision", "coverage (flagged true positives / positives)", "flagged", "repository-cluster bootstrap 95% CI of flagged precision"],
+  "decision_rule": "REJECT B if in any family B flags held-out paired units at a point precision < 0.95; else ADOPT B if in some family B's held-out paired Wilson LB >= 0.95; else INCONCLUSIVE (the incumbent rule stays)",
+  "m4_reading": "a family reaches the M4 precision bar on held-out repositories when its held-out paired Wilson LB >= 0.95; point precision reported beside it",
+  "power": "a Wilson LB of 0.95 needs >= 73 flagged units with no false flag; the largest family holds 38 paired positives over all five folds, so arm B cannot offer a BLOCK in any family by construction; expected verdict inconclusive",
+  "status": "run and analysed 2026-10-03: inconclusive -- no arm flags any held-out unit (result.json); meter 0 client calls, $0.0; 142 of 353 units excluded for an uncached response"
+}

benchmarks/experiments/exp-003-precision-bound-blocking/result.json

@@ -0,0 +1,981 @@
+{
+  "arms": {
+    "A": {
+      "label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
+      "rule": {
+        "name": "calibrated_block"
+      }
+    },
+    "B": {
+      "label": "selective blocking -- per-family raw-score threshold from the other folds' paired units, lowest t with Wilson 95% LB of flagged precision >= 0.95",
+      "rule": {
+        "bound": "wilson",
+        "name": "precision_bound_block",
+        "precision": 0.95,
+        "select_on": "paired"
+      }
+    },
+    "C": {
+      "label": "secondary, non-deciding -- as B with point precision >= 0.95 over at least 10 training flags",
+      "rule": {
+        "bound": "point",
+        "min_flagged": 10,
+        "name": "precision_bound_block",
+        "precision": 0.95,
+        "select_on": "paired"
+      }
+    }
+  },
+  "code_commit": "7bb97ad0bca62b5f986c5768297d8b8fca50fae4",
+  "decision": "inconclusive",
+  "experiment": "exp-003-precision-bound-blocking",
+  "families": {
+    "access_control": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.077,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 16,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 32
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 24,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 41
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 16,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 32
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 24,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 41
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 16,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 32
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 24,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 41
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 41,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 12,
+          "outer-1": 13,
+          "outer-2": 13,
+          "outer-3": 13,
+          "outer-4": 13
+        }
+      }
+    },
+    "deserialization": {
+      "A": {
+        "calibration_holds": false,
+        "ece": null,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 3,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 2,
+          "true_flagged": 0,
+          "units": 5
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 12,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 11,
+          "true_flagged": 0,
+          "units": 14
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 3,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 2,
+          "true_flagged": 0,
+          "units": 5
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 12,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 11,
+          "true_flagged": 0,
+          "units": 14
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 3,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 2,
+          "true_flagged": 0,
+          "units": 5
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 12,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 11,
+          "true_flagged": 0,
+          "units": 14
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 14,
+        "excluded_replay_miss": 34,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 0,
+          "outer-2": 2,
+          "outer-3": 2,
+          "outer-4": 2
+        }
+      }
+    },
+    "injection": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.1067,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 26,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 76
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 34,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 63,
+          "true_flagged": 0,
+          "units": 119
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 26,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 76
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 34,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 63,
+          "true_flagged": 0,
+          "units": 119
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 26,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 76
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 34,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 63,
+          "true_flagged": 0,
+          "units": 119
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 119,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 29,
+          "outer-1": 33,
+          "outer-2": 33,
+          "outer-3": 26,
+          "outer-4": 31
+        }
+      }
+    },
+    "output_encoding": {
+      "A": {
+        "calibration_holds": false,
+        "ece": null,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 8,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 8,
+          "true_flagged": 0,
+          "units": 16
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 15,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 24
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 8,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 8,
+          "true_flagged": 0,
+          "units": 16
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 15,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 24
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 8,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 8,
+          "true_flagged": 0,
+          "units": 16
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 15,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 24
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 24,
+        "excluded_replay_miss": 21,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 7,
+          "outer-1": 6,
+          "outer-2": 6,
+          "outer-3": 7,
+          "outer-4": 6
+        }
+      }
+    },
+    "path": {
+      "A": {
+        "paired": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        },
+        "pooled": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        },
+        "pooled": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        },
+        "pooled": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        }
+      },
+      "capacity": {
+        "evaluated": 0,
+        "excluded_replay_miss": 39,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {}
+      }
+    },
+    "untrusted_destination": {
+      "A": {
+        "calibration_holds": false,
+        "ece": null,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 6,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 4,
+          "true_flagged": 0,
+          "units": 9
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 9,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 7,
+          "true_flagged": 0,
+          "units": 13
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 6,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 4,
+          "true_flagged": 0,
+          "units": 9
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 9,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 7,
+          "true_flagged": 0,
+          "units": 13
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 6,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 4,
+          "true_flagged": 0,
+          "units": 9
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 9,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 7,
+          "true_flagged": 0,
+          "units": 13
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null
+        }
+      },
+      "capacity": {
+        "evaluated": 13,
+        "excluded_replay_miss": 48,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 4,
+          "outer-1": 3,
+          "outer-2": 3,
+          "outer-3": 2
+        }
+      }
+    }
+  },
+  "hypothesis": "The calibrated-probability BLOCK rule (Platt map + calibration gate, ECE <= 0.05) is the wrong instrument for M4, which asks for precision on what is flagged. Selective blocking -- a per-family raw-score threshold chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95 -- reaches the M4 precision bar on held-out repositories where the calibrated rule offers no BLOCK.\n",
+  "instrument_check": {
+    "families": {
+      "access_control": {
+        "distinct_s": 30,
+        "paired": {
+          "blockable": 28,
+          "positives_before_first_negative": 10
+        },
+        "pooled": {
+          "blockable": 37,
+          "positives_before_first_negative": 19
+        },
+        "verdicts": {
+          "not_vulnerable": 19,
+          "unsure": 4,
+          "vulnerable": 18
+        }
+      },
+      "deserialization": {
+        "distinct_s": 5,
+        "paired": {
+          "blockable": 4,
+          "positives_before_first_negative": 1
+        },
+        "pooled": {
+          "blockable": 13,
+          "positives_before_first_negative": 10
+        },
+        "verdicts": {
+          "not_vulnerable": 2,
+          "unsure": 1,
+          "vulnerable": 11
+        }
+      },
+      "injection": {
+        "distinct_s": 48,
+        "paired": {
+          "blockable": 50,
+          "positives_before_first_negative": 3
+        },
+        "pooled": {
+          "blockable": 77,
+          "positives_before_first_negative": 11
+        },
+        "verdicts": {
+          "not_vulnerable": 46,
+          "unsure": 42,
+          "vulnerable": 31
+        }
+      },
+      "output_encoding": {
+        "distinct_s": 14,
+        "paired": {
+          "blockable": 11,
+          "positives_before_first_negative": 0
+        },
+        "pooled": {
+          "blockable": 19,
+          "positives_before_first_negative": 8
+        },
+        "verdicts": {
+          "not_vulnerable": 9,
+          "unsure": 5,
+          "vulnerable": 10
+        }
+      },
+      "untrusted_destination": {
+        "distinct_s": 9,
+        "paired": {
+          "blockable": 4,
+          "positives_before_first_negative": 1
+        },
+        "pooled": {
+          "blockable": 8,
+          "positives_before_first_negative": 4
+        },
+        "verdicts": {
+          "not_vulnerable": 3,
+          "unsure": 5,
+          "vulnerable": 5
+        }
+      }
+    },
+    "note": "descriptive, all folds together, ranking by raw score s among non-unsure verdicts; not nested, chooses nothing. It shows the scores are real and the zero is the rule's, not an unread input."
+  },
+  "manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
+  "meter": {
+    "client_calls": 0,
+    "decided": 211,
+    "excluded_replay_miss": 142,
+    "folds_digest": "a1f78e141a572a4adcc9cd57eb8a784c440c177f0b60d5ef5bed9e94c8753c5f",
+    "metered_usd": 0.0,
+    "replay_priced_usd_not_spent": 0.582876,
+    "replayed_responses": 1056
+  },
+  "min_flagged_for_bound": 73,
+  "overall": {
+    "A": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 59,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 68,
+        "true_flagged": 0,
+        "units": 138
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 89,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 122,
+        "true_flagged": 0,
+        "units": 211
+      }
+    },
+    "B": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 59,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 68,
+        "true_flagged": 0,
+        "units": 138
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 89,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 122,
+        "true_flagged": 0,
+        "units": 211
+      }
+    },
+    "C": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 59,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 68,
+        "true_flagged": 0,
+        "units": 138
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 89,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 122,
+        "true_flagged": 0,
+        "units": 211
+      }
+    }
+  },
+  "populations": {
+    "primary": "paired",
+    "secondary": [
+      "pooled"
+    ]
+  },
+  "primary": [
+    {
+      "name": "flagged_precision_lb",
+      "target": 0.95
+    }
+  ],
+  "reason": "B offers no BLOCK on held-out paired units in any family; the incumbent stays",
+  "resamples": 2000,
+  "secondary": [
+    {
+      "name": "flagged_precision"
+    },
+    {
+      "name": "coverage"
+    },
+    {
+      "name": "flagged"
+    }
+  ],
+  "spend_usd": {
+    "score": 0.0
+  },
+  "target_precision": 0.95,
+  "units": {
+    "by_label": {
+      "0": 89,
+      "1": 122
+    },
+    "evaluated": 211,
+    "excluded_replay_miss": 142,
+    "frozen": 353,
+    "groups": 89,
+    "not_run": 0,
+    "paired_evaluated": 138,
+    "unsure": 57
+  }
+}

benchmarks/experiments/exp-004-pooled-block-gate/instrument_check.py

@@ -0,0 +1,53 @@
+"""Instrument check for exp-004: are the scores real, and how clean is the pooled top of the ranking?
+
+Reads the run's arm_outcome rows from the memory store (OUSAST_MEMORY) and the frozen units file; makes no model call.
+usage: python benchmarks/experiments/exp-004-pooled-block-gate/instrument_check.py   (from the repository root)
+"""
+
+from __future__ import annotations
+
+import collections
+import json
+from pathlib import Path
+
+from openultrasast.config import load_dotenv
+from openultrasast.learn import experiments as ex
+from openultrasast.plane.memory import open_store
+
+EXPERIMENT = "exp-004-pooled-block-gate"
+UNITS = Path(f"plane/experiments/{EXPERIMENT}.units.jsonl")
+
+
+def main() -> None:
+    load_dotenv(Path(".env"))
+    rows = [r.row for r in open_store(None).rows(repo=ex.experiment_repo(EXPERIMENT), kind="arm_outcome")]
+    with UNITS.open() as handle:
+        units = {u["unit"]: u for u in (json.loads(line) for line in handle)}
+    print("arm_outcome rows", len(rows), "units file", len(units))
+    by_family: dict[str, list[tuple[float | None, int, str | None, str]]] = collections.defaultdict(list)
+    for row in rows:
+        unit = units.get(row.get("unit"))
+        if unit:
+            by_family[unit["family"]].append((row.get("s"), unit["label"], unit.get("pair"), str(row.get("verdict") or "")))
+    for family, xs in sorted(by_family.items()):
+        scored = [x for x in xs if x[0] is not None]
+        pairs: dict[str, dict[int, float]] = collections.defaultdict(dict)
+        for score, label, pair, _ in scored:
+            if pair:
+                pairs[pair][label] = score
+        full = [p for p in pairs.values() if 0 in p and 1 in p]
+        wins = sum(1 for p in full if p[1] > p[0])
+        ties = sum(1 for p in full if p[1] == p[0])
+        top = sorted((x for x in scored if x[2]), key=lambda x: -(x[0] or 0.0))
+        first_negative = next((i for i, x in enumerate(top) if x[1] == 0), len(top))
+        print(
+            f"{family}: n={len(xs)} distinct_scores={len({round(x[0] or 0.0, 4) for x in scored})} "
+            f"unsure={sum(1 for x in xs if x[3] == 'unsure')} pairs={len(full)} within_pair_win={wins} tie={ties} "
+            f"pos_above_first_neg={first_negative}"
+        )
+    pooled = sorted((x for xs in by_family.values() for x in xs if x[0] is not None and x[2]), key=lambda x: -(x[0] or 0.0))
+    print("pooled raw top-30 labels:", "".join(str(x[1]) for x in pooled[:30]))
+
+
+if __name__ == "__main__":
+    main()

benchmarks/experiments/exp-004-pooled-block-gate/record.json

@@ -0,0 +1,81 @@
+{
+  "experiment": "exp-004-pooled-block-gate",
+  "date": "2026-10-03",
+  "fill": {
+    "client_calls": 2073,
+    "usd": 1.238223,
+    "ceiling_usd": 2.0,
+    "still_missing": 0,
+    "dry_run_estimate_usd": 5.582,
+    "note": "the dry-run estimator over-priced 4.5x (no prompt-cache pricing); measured spend is the meter's"
+  },
+  "run": {
+    "units": 501,
+    "client_calls": 0,
+    "usd": 0.0,
+    "replayed": 2509,
+    "replay_miss": 0
+  },
+  "verdict": "inconclusive",
+  "reason": "no t_block on any outer fold, pooled or per family; B offers no BLOCK",
+  "instrument_check": {
+    "per_family": {
+      "access_control": {
+        "n": 66,
+        "distinct_scores": 41,
+        "unsure": 5,
+        "pairs": 28,
+        "within_pair_wins": 22,
+        "ties": 1,
+        "positives_above_first_negative": 12
+      },
+      "deserialization": {
+        "n": 64,
+        "distinct_scores": 38,
+        "unsure": 7,
+        "pairs": 25,
+        "within_pair_wins": 18,
+        "ties": 4,
+        "positives_above_first_negative": 5
+      },
+      "injection": {
+        "n": 149,
+        "distinct_scores": 52,
+        "unsure": 54,
+        "pairs": 53,
+        "within_pair_wins": 30,
+        "ties": 17,
+        "positives_above_first_negative": 6
+      },
+      "output_encoding": {
+        "n": 69,
+        "distinct_scores": 31,
+        "unsure": 32,
+        "pairs": 28,
+        "within_pair_wins": 18,
+        "ties": 6,
+        "positives_above_first_negative": 2
+      },
+      "path": {
+        "n": 66,
+        "distinct_scores": 43,
+        "unsure": 15,
+        "pairs": 29,
+        "within_pair_wins": 25,
+        "ties": 3,
+        "positives_above_first_negative": 13
+      },
+      "untrusted_destination": {
+        "n": 87,
+        "distinct_scores": 36,
+        "unsure": 33,
+        "pairs": 32,
+        "within_pair_wins": 18,
+        "ties": 4,
+        "positives_above_first_negative": 3
+      }
+    },
+    "pooled_raw_top30_labels": "111111111110111110111111110111",
+    "reading": "scores are real (31-52 distinct values per family); pooled raw top 30 holds 27 positives (0.90), first negative at rank 12; the Wilson bound needs >= 73 flags with no error, so precision at the top, not data volume alone, now binds"
+  }
+}

benchmarks/experiments/exp-004-pooled-block-gate/registration.json

@@ -0,0 +1,35 @@
+{
+  "experiment": "exp-004-pooled-block-gate",
+  "registered": "2026-10-03",
+  "store": "s3://sast-memory",
+  "manifest": "plane/experiments/exp-004-pooled-block-gate.yaml",
+  "manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
+  "pin": "2ad9c1db7ae673c2a2e48a0afadcaa795dfb1ced",
+  "units_status": "pending",
+  "units_file": "plane/experiments/exp-004-pooled-block-gate.units.jsonl",
+  "units_digest": "",
+  "units": "frozen at run time by `ousast learn experiment units` from the memory after the negatives harvest (merged in 5b2bd9b, 1133 static examples), committed, and registered again before the run; until then the run refuses (check_registered: the units file does not exist)",
+  "requirement": "learned-decision-engine Req 6.4, amended 2026-10-03 by maintainer decision (refines the BLOCK rule of Req 7.3 and 7.4)",
+  "kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
+  "zero_cost": "the run replays the response cache behind a zero-ceiling meter; a unit with any uncached response is excluded and counted. The harvest changed the memory the programs retrieve from, so a paid cache fill, approved separately with its own ceiling, must precede the run",
+  "normalisation": {
+    "choice": "per-family Platt scaling: learn.calibrate.fit_map with its existing floors (own slope and intercept per family at >= 20 positive and negative repository groups, a pooled slope with a per-family intercept at >= 10, no map below that and so no BLOCK for that family on that fold), fitted on the training folds' paired units only",
+    "why": "a rank-to-quantile map gives every family the same flag rate whatever its precision, so one pooled threshold would flag the top slice of a weak family as readily as of a strong one; Platt puts each family on the scale of its own training precision, which a pooled precision bound needs. The map is a normaliser only: precision is shown by the held-out bound, not by calibration (exp-003: ECE <= 0.05 does not hold per family)"
+  },
+  "arms": {
+    "A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds; BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
+    "B": "pooled gate: per outer fold j, training units normalised by an inner cross-fit; t_block = the lowest normalised score whose Wilson 95% LB of precision over the training paired flags of all families together is >= 0.95 (unsure never flags); t_quiet per family = the highest normalised score keeping >= 90% of that family's training paired positives; fold j's units BLOCK at p >= t_block, ADVISORY at p >= t_quiet (or without a map or t_quiet), silent below"
+  },
+  "family_floor": {
+    "F": 10,
+    "rule": "a family may block only if its own held-out paired flags (all five folds) number >= 10 with zero errors; otherwise its flags are reported as ADVISORY. The floor uses held-out labels to decide which families block and moves no threshold",
+    "why": "zero errors in 10 flags is unlikely for a family much worse than the bar (probability 0.056 at precision 0.75, 0.107 at 0.80); a larger F is out of reach of the smaller families (16-19 paired positives in exp-003's units over all folds); 10 is exp-003 arm C's training minimum"
+  },
+  "primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision over arm B's paired flags of all families together, before the family floor (so dropping a failing family cannot raise it)",
+  "populations": {"primary": "paired (both sides of the candidate present)", "secondary": ["pooled (every unit)"]},
+  "secondary_metrics": ["per-family flagged precision and Wilson LB", "the family-floor table", "quiet_recall per family (held-out positives at or above t_quiet / positives) against 0.90", "after-floor totals (what users would get)", "repository-cluster bootstrap 95% CI of flagged precision", "coverage", "flagged"],
+  "decision_rule": "on arm B's held-out paired flags of all families together, before the floor: REJECT B if their point precision < 0.95; else ADOPT B if their Wilson 95% LB >= 0.95 and at least one family passes the floor (>= 10 held-out paired flags, zero errors) -- those families block, every other family stays advisory; else INCONCLUSIVE (the incumbent rule stays)",
+  "power": "the pooled bound needs >= 73 flags with no error (110 with one, 142 with two). On exp-003's units (all 353 scored, result-complete-cache.json) few paired positives rank above the first negative per family on the raw score (access control 10, deserialization 7, injection 2, output encoding 0, path 8, untrusted destination 5); normalisation keeps each family's order, so the pooled top set reaches 73 only if the harvest's pairs separate better. An inconclusive verdict is a plausible outcome",
+  "tests": "tests/test_learn_rules.py: test_one_pooled_threshold_blocks_where_no_family_alone_reaches_the_bound, test_no_held_out_label_moves_a_threshold_or_the_map, test_a_family_with_one_held_out_error_cannot_block, test_quiet_threshold_keeps_ninety_percent_of_positives_and_nothing_is_silenced_blind, test_the_pooled_verdict_follows_the_registered_rule, test_the_exp004_manifest_registers_the_procedure_before_its_units",
+  "status": "registered 2026-10-03, units pending, not run"
+}

benchmarks/experiments/exp-004-pooled-block-gate/result.json

@@ -0,0 +1,822 @@
+{
+  "arms": {
+    "A": {
+      "label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
+      "rule": {
+        "name": "calibrated_block"
+      }
+    },
+    "B": {
+      "label": "pooled gate -- one t_block over all families on a per-family Platt score (training folds, inner cross-fit), t_quiet per family at 90% recall, family floor 10 clean held-out flags",
+      "rule": {
+        "floor": 10,
+        "name": "pooled_block_gate",
+        "normalise": "platt",
+        "precision": 0.95,
+        "recall": 0.9,
+        "select_on": "paired"
+      }
+    }
+  },
+  "code_commit": "90a2642019df414cb67b5dcea6e4ec235f35650a",
+  "decision": "inconclusive",
+  "experiment": "exp-004-pooled-block-gate",
+  "families": {
+    "access_control": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0715,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "quiet_recall": {
+          "paired": 0.8929,
+          "pooled": 0.9211
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.44900641988143913,
+          "outer-1": 0.3723384028044276,
+          "outer-2": 0.4202772526614184,
+          "outer-3": 0.4481849709797749,
+          "outer-4": 0.36346174463707154
+        }
+      },
+      "capacity": {
+        "evaluated": 66,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 22,
+          "outer-1": 22,
+          "outer-2": 22,
+          "outer-3": 23,
+          "outer-4": 23
+        }
+      }
+    },
+    "deserialization": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0725,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 25,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 50
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 39,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 25,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 50
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 39,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "quiet_recall": {
+          "paired": 0.88,
+          "pooled": 0.9231
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.17507194490500488,
+          "outer-1": 0.22766488322270723,
+          "outer-2": 0.2163745870743959,
+          "outer-3": 0.3112201878893598,
+          "outer-4": 0.363243722721776
+        }
+      },
+      "capacity": {
+        "evaluated": 64,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 20,
+          "outer-1": 20,
+          "outer-2": 20,
+          "outer-3": 20,
+          "outer-4": 20
+        }
+      }
+    },
+    "injection": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.1042,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 41,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 53,
+          "true_flagged": 0,
+          "units": 106
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 78,
+          "true_flagged": 0,
+          "units": 149
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 41,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 53,
+          "true_flagged": 0,
+          "units": 106
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 78,
+          "true_flagged": 0,
+          "units": 149
+        },
+        "quiet_recall": {
+          "paired": 0.8679,
+          "pooled": 0.8974
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.287983504097999,
+          "outer-1": 0.17524866355153343,
+          "outer-2": 0.12918182665453967,
+          "outer-3": 0.14521040470628938,
+          "outer-4": 0.23005714687697673
+        }
+      },
+      "capacity": {
+        "evaluated": 149,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 38,
+          "outer-1": 44,
+          "outer-2": 45,
+          "outer-3": 43,
+          "outer-4": 42
+        }
+      }
+    },
+    "output_encoding": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0736,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 41,
+          "true_flagged": 0,
+          "units": 69
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 41,
+          "true_flagged": 0,
+          "units": 69
+        },
+        "quiet_recall": {
+          "paired": 0.8571,
+          "pooled": 0.9024
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.3723166587726901,
+          "outer-1": 0.5329072757249385,
+          "outer-2": 0.3785066578507705,
+          "outer-3": 0.4389011844848602,
+          "outer-4": 0.43855966643440264
+        }
+      },
+      "capacity": {
+        "evaluated": 69,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 23,
+          "outer-1": 22,
+          "outer-2": 22,
+          "outer-3": 22,
+          "outer-4": 23
+        }
+      }
+    },
+    "path": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0621,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 29,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 29,
+          "true_flagged": 0,
+          "units": 58
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 37,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 29,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 29,
+          "true_flagged": 0,
+          "units": 58
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 37,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "quiet_recall": {
+          "paired": 0.8621,
+          "pooled": 0.8919
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.4680507207548809,
+          "outer-1": 0.3955717127313657,
+          "outer-2": 0.43694703358760895,
+          "outer-3": 0.4450583743716199,
+          "outer-4": 0.46396657833309796
+        }
+      },
+      "capacity": {
+        "evaluated": 66,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 23,
+          "outer-1": 23,
+          "outer-2": 24,
+          "outer-3": 23,
+          "outer-4": 23
+        }
+      }
+    },
+    "untrusted_destination": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.1143,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 31,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 32,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 50,
+          "true_flagged": 0,
+          "units": 87
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 31,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 32,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 50,
+          "true_flagged": 0,
+          "units": 87
+        },
+        "quiet_recall": {
+          "paired": 1.0,
+          "pooled": 1.0
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.13676746287025585,
+          "outer-1": 0.11255775703980551,
+          "outer-2": 0.13763698150768486,
+          "outer-3": 0.09787938647374639,
+          "outer-4": 0.13763698150768486
+        }
+      },
+      "capacity": {
+        "evaluated": 87,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 26,
+          "outer-1": 26,
+          "outer-2": 24,
+          "outer-3": 26,
+          "outer-4": 26
+        }
+      }
+    }
+  },
+  "hypothesis": "One BLOCK threshold pooled across families on a per-family Platt-normalised score, chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95, reaches that bound on held-out repositories' paired units, where per-family thresholds (exp-003) could not for lack of flags; a per-family floor (>= 10 held-out flags, none wrong) keeps a family that the pooling masks from blocking.\n",
+  "manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
+  "min_flagged_for_bound": 73,
+  "overall": {
+    "A": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 182,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 195,
+        "true_flagged": 0,
+        "units": 390
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 220,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 283,
+        "true_flagged": 0,
+        "units": 501
+      }
+    },
+    "B": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 182,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 195,
+        "true_flagged": 0,
+        "units": 390
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 220,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 283,
+        "true_flagged": 0,
+        "units": 501
+      }
+    }
+  },
+  "overall_after_floor": {
+    "B": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 182,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 195,
+        "true_flagged": 0,
+        "units": 390
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 220,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 283,
+        "true_flagged": 0,
+        "units": 501
+      }
+    }
+  },
+  "populations": {
+    "primary": "paired",
+    "secondary": [
+      "pooled"
+    ]
+  },
+  "primary": [
+    {
+      "name": "flagged_precision_lb",
+      "target": 0.95
+    }
+  ],
+  "reason": "B offers no BLOCK on held-out paired units; the incumbent stays",
+  "resamples": 2000,
+  "secondary": [
+    {
+      "name": "flagged_precision"
+    },
+    {
+      "name": "coverage"
+    },
+    {
+      "name": "flagged"
+    },
+    {
+      "name": "quiet_recall"
+    }
+  ],
+  "spend_usd": {
+    "score": 0.0
+  },
+  "target_precision": 0.95,
+  "units": {
+    "by_label": {
+      "0": 218,
+      "1": 283
+    },
+    "evaluated": 501,
+    "excluded_replay_miss": 0,
+    "frozen": 501,
+    "groups": 220,
+    "not_run": 0,
+    "paired_evaluated": 390,
+    "unsure": 146
+  }
+}

benchmarks/experiments/exp-005-graph-slice/fill-B.json

@@ -0,0 +1,84 @@
+{
+  "arms": [
+    "B"
+  ],
+  "client_calls": 2394,
+  "experiment": "exp-005-graph-slice",
+  "families": {
+    "access_control": {
+      "cached": 0,
+      "conditional_retries": 330,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 0.9956892000000007,
+      "filled": 330,
+      "missing": 330,
+      "requests": 330,
+      "still_missing": 0,
+      "units": 66
+    },
+    "deserialization": {
+      "cached": 0,
+      "conditional_retries": 310,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 0.8843295999999988,
+      "filled": 310,
+      "missing": 310,
+      "requests": 310,
+      "still_missing": 0,
+      "units": 64
+    },
+    "injection": {
+      "cached": 0,
+      "conditional_retries": 675,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 2.270278999999999,
+      "filled": 675,
+      "missing": 675,
+      "requests": 675,
+      "still_missing": 0,
+      "units": 149
+    },
+    "output_encoding": {
+      "cached": 0,
+      "conditional_retries": 325,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 1.0014180000000004,
+      "filled": 328,
+      "missing": 325,
+      "requests": 328,
+      "still_missing": 0,
+      "units": 69
+    },
+    "path": {
+      "cached": 0,
+      "conditional_retries": 320,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 0.9399192000000005,
+      "filled": 325,
+      "missing": 320,
+      "requests": 325,
+      "still_missing": 0,
+      "units": 66
+    },
+    "untrusted_destination": {
+      "cached": 0,
+      "conditional_retries": 425,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 1.174316,
+      "filled": 426,
+      "missing": 425,
+      "requests": 426,
+      "still_missing": 0,
+      "units": 87
+    }
+  },
+  "reason": "",
+  "status": "done",
+  "usd": 1.5890382240000016
+}

benchmarks/experiments/exp-005-graph-slice/result.json

@@ -0,0 +1,165 @@
+{
+  "alpha": 0.05,
+  "arms": {
+    "A": {
+      "budget_usd": 0.0,
+      "label": "six slice-2 incumbents unchanged, replaying exp-004's response cache",
+      "repeats": 1,
+      "replay_only": true
+    },
+    "B": {
+      "budget_usd": 6.0,
+      "label": "the same six programs with the in-function slice added to their prompts",
+      "repeats": 1,
+      "replay_only": false
+    }
+  },
+  "code_commit": "0acbc7ec41dcaeca805c4720bc3f6d02cb940ee7",
+  "decision": "inconclusive",
+  "experiment": "exp-005-graph-slice",
+  "hypothesis": "Adding the in-function def-use slice to the prompt improves within-pair ranking of the six slice-2 incumbent programs on held-out repositories, at no more than 1.5 times the cost per candidate.\n",
+  "looks": [
+    {
+      "boundary": 1.959964,
+      "crossed": false,
+      "estimate": {
+        "a": 0.7615384615384615,
+        "b": 0.7461538461538462,
+        "ci": [
+          -0.0628019323671497,
+          0.03155339805825241
+        ],
+        "diff": -0.01538461538461533,
+        "groups": 220,
+        "metric": "within_pair_auc",
+        "resamples": 10000,
+        "se": 0.024030780235707972,
+        "units": 501
+      },
+      "fraction": 1.0,
+      "groups": 220,
+      "z": -0.6402045723740141
+    }
+  ],
+  "manifest_digest": "1b1ec814cde12801491376d2086c8c2d6db70f4a86fc308c0adc5a530398e554",
+  "mde": 0.0,
+  "primary": [
+    {
+      "name": "within_pair_auc"
+    }
+  ],
+  "reason": "the CI of within_pair_auc spans 0; the incumbent stays",
+  "resamples": 10000,
+  "secondary": [
+    {
+      "name": "top_30_precision"
+    },
+    {
+      "name": "unsure_rate"
+    },
+    {
+      "name": "usd_per_candidate"
+    }
+  ],
+  "spend_usd": {
+    "A": 1.444503,
+    "B": 1.589038
+  },
+  "tests": {
+    "top_30_precision": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 1.0,
+        "b": 1.0,
+        "ci": [
+          0.0,
+          0.0
+        ],
+        "diff": 0.0,
+        "groups": 220,
+        "metric": "top_30_precision",
+        "resamples": 10000,
+        "se": 0.0006666666666666664,
+        "units": 501
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "unsure_rate": {
+      "ci_vs_zero": "favour",
+      "direction": -1,
+      "estimate": {
+        "a": 0.29141716566866266,
+        "b": 0.23952095808383234,
+        "ci": [
+          -0.08583690987124465,
+          -0.01996007984031939
+        ],
+        "diff": -0.05189620758483032,
+        "groups": 220,
+        "metric": "unsure_rate",
+        "resamples": 10000,
+        "se": 0.01685991406098136,
+        "units": 501
+      },
+      "mcnemar": {
+        "b": 47,
+        "c": 21,
+        "concordant": 433,
+        "p_value": 0.0021862431776861705
+      },
+      "target": null
+    },
+    "usd_per_candidate": {
+      "ci_vs_zero": "against",
+      "direction": -1,
+      "estimate": {
+        "a": 0.002883239520958083,
+        "b": 0.0031717325349301397,
+        "ci": [
+          0.0002458205128205199,
+          0.0003319800399201574
+        ],
+        "diff": 0.0002884930139720565,
+        "groups": 220,
+        "metric": "usd_per_candidate",
+        "resamples": 10000,
+        "se": 2.1746161270931252e-05,
+        "units": 501
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "within_pair_auc": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 0.7615384615384615,
+        "b": 0.7461538461538462,
+        "ci": [
+          -0.0628019323671497,
+          0.03155339805825241
+        ],
+        "diff": -0.01538461538461533,
+        "groups": 220,
+        "metric": "within_pair_auc",
+        "resamples": 10000,
+        "se": 0.024030780235707972,
+        "units": 501
+      },
+      "mcnemar": null,
+      "target": null
+    }
+  },
+  "tost": null,
+  "units": {
+    "by_label": {
+      "0": 218,
+      "1": 283
+    },
+    "groups": 220,
+    "outcomes": 1002,
+    "paired": 501
+  }
+}

benchmarks/learn/build_features.py

@@ -7,7 +7,7 @@ from:
 - quick rules and deterministic roles, run here on the unit's tree: a pair side materialised from its excerpt, a
   population or recipe pin exported from its clone or checkout (one at a time, deleted after, disk-guarded);
 - the engine: ``benchmarks/learn/engine_pairs.py`` records for pairs, the recorded protocol scans
-  (``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>.json``) for populations;
+  (``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>[_a|_b].json``) for populations;
 - verify passes a/b and ``agree`` from the harvest Run (``harvest-verify``) and, for population v2, the recorded
   ``validation-46`` Run; model roles from ``harvest-roles``;
 - callers and entry-point distance on a repository (``not_applicable`` on a pair excerpt: no repository exists).
@@ -21,6 +21,11 @@ An instrument that has not run for a unit yet (the engine job still running, a R
 memory store (``OUSAST_MEMORY``, default ``~/ousast-results/plane/memory``) as kind ``features``; a rerun replaces
 them by id. ``--counts`` writes the counts-only coverage record (no candidate, path or repository name).
 
+``--dry-run`` instead compares existing local store examples with the corrected engine features, printing
+per-source transitions from not-run/none to ran (and recorded failures separately). It reads only the spent
+v1/v2 manifests, saved scans and local Git objects; it writes neither the store nor ``--counts`` and does not
+rebuild other instruments. No label snapshot, harvest units or scratch directory is needed in this mode.
+
 Usage::
 
     python benchmarks/learn/build_features.py --labels labels-<sha>.jsonl --units <harvest plane>/units.json --counts counts.json
@@ -30,9 +35,11 @@ from __future__ import annotations
 
 import argparse
 import json
+import os
 import shutil
 import subprocess
 import tempfile
+import tomllib
 from collections import Counter, defaultdict
 from collections.abc import Mapping
 from pathlib import Path
@@ -61,6 +68,7 @@ from openultrasast.learn.features import (
     source_part,
     verify_part,
 )
+from openultrasast.learn.labels import Clone
 from openultrasast.learn.roles import RoleSet, from_model_roles, infer_for_checkout
 from openultrasast.pairs import DEFAULT_CATALOG, _materialize_side, _targets, load_pair_catalog
 from openultrasast.plane import memory
@@ -89,6 +97,110 @@ def read_json(path: Path) -> Any:
     return json.loads(path.read_text(encoding="utf-8")) if path.is_file() else None
 
 
+def resolve_scan(scans: Path, case: str, side: str) -> dict[str, Any] | None:
+    """Read the v1 single scan, otherwise combine the v2 repeats for this side."""
+    single = read_json(scans / f"{case}--{side}.json")
+    if single is not None:
+        return dict(single)
+    repeats = [read_json(scans / f"{case}--{side}_{repeat}.json") for repeat in ("a", "b")]
+    available = [r for r in repeats if r is not None]
+    if not available:
+        return None
+    # Identical rule for vulnerable and fixed: a finding in either repeat counts;
+    # exact duplicates count once. engine_part takes the highest rung and minimum
+    # witness steps across the union. Completion is sum(completed)/sum(questions).
+    # A successful repeat survives a failed/missing peer, which marks degradation.
+    ran = [r for r in available if engine_for(r) is None]
+    findings = {json.dumps(f, sort_keys=True): f for r in ran for f in r.get("findings", [])}
+    degradations = sorted({d for r in available for d in r.get("degradations", [])})
+    if len(ran) != len(repeats):
+        degradations.append("repeat_missing_or_failed")
+    return {
+        "state": "ran" if ran else "none" if all(engine_for(r) == NONE for r in available) else "failed",
+        "questions": sum(int(r.get("questions") or 0) for r in available),
+        "completed": sum(int(r.get("completed") or 0) for r in available),
+        "findings": list(findings.values()),
+        "degradations": degradations,
+        "image": "+".join(sorted({str(r.get("image") or "recorded") for r in available})),
+    }
+
+
+def recorded_engine(language: str, findings: list, result: Mapping | None, repository: bool) -> Part:
+    # The language list describes standalone scans. Repository scans also analyze
+    # TypeScript; recorded function findings take precedence over that list.
+    if language not in ENGINE_LANGUAGES and not findings and not (repository and language == "typescript" and result is not None):
+        return NONE
+    state = engine_for(result)
+    if state is not None:
+        return state
+    result = result or {}
+    questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
+    return engine_part(
+        findings, completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
+        vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
+    )  # fmt: skip
+
+
+class PinSourceIndex(SourceIndex):
+    """Resolve findings with the same matcher as a checkout, without exporting it."""
+
+    def __init__(self, clone: Path, pin: str) -> None:
+        super().__init__(clone)
+        self.clone, self.pin = Clone(clone), pin
+
+    def lines(self, path: str) -> list[str]:
+        if path not in self._lines:
+            self._lines[path] = self.clone.git("show", f"{self.pin}:{path}").splitlines()
+        return self._lines[path] or []
+
+
+def dry_run(store: memory.FileStore, results: Path, cache: Path) -> dict[str, Any]:
+    """Count existing examples whose engine becomes available; never write rows or blobs.
+
+    Only spent v1/v2 populations are read. Counts are examples, not the two
+    static/plane feature records. Other sources are reported with zero changes.
+    """
+    root = Path(__file__).resolve().parents[2]
+    cases = {}
+    for population in ("population-v1", "population-v2"):
+        manifest = root / "benchmarks" / "independent" / f"{population}.toml"
+        for case in tomllib.loads(manifest.read_text(encoding="utf-8"))["case"]:
+            for side in ("vulnerable", "fixed"):
+                cases[(population, memory.repo_key(case["repo"]), case[side], side)] = case["id"]
+    rows = [r.row for r in store.rows(kind="example")]
+    if not rows:
+        raise ValueError("dry run read no examples from the store")
+    counts: dict[str, Counter[str]] = defaultdict(Counter)
+    for row in rows:
+        count = counts[row["source"]]
+        count["examples"] += 1
+        old = row["instruments"]["engine"]
+        reason = "not_run" if old.get("version") == "missing:not-run" else "none" if old["state"] == "none" else None
+        if reason is None:
+            continue
+        key = (row["split"], row["repo"], row["pin"], row["pin_role"])
+        case_id = cases.get(key)
+        if case_id is None:
+            continue
+        scans = results / row["split"].replace("population", "independent") / "scans"
+        result = resolve_scan(scans, case_id, row["pin_role"])
+        path, _, function = row["candidate"].partition("::")
+        engine = {}
+        if engine_for(result) is None:
+            index = PinSourceIndex(cache / "independent" / case_id, row["pin"])
+            lines = index.lines(path)  # fail loudly if the candidate's input cannot be read
+            count["source_bytes_read"] += len("\n".join(lines).encode("utf-8"))
+            selected = dict(result or {})
+            selected["findings"] = [f for f in selected.get("findings", []) if f["site"].partition(":")[0] == path]
+            _, engine = _signals(index, [], selected, {})
+        part = recorded_engine(row["language"], engine.get((path, function, row["family"]), []), result, True)
+        if part.state == "ran":
+            count[f"{reason}_to_ran"] += 1
+        elif result is not None and part.state == "failed":
+            count[f"{reason}_to_recorded_failed"] += 1
+    return {source: {**{"not_run_to_ran": 0, "none_to_ran": 0}, **dict(count)} for source, count in sorted(counts.items())}
+
+
 def done(run: Path, entry: str) -> bool:
     state = (read_json(run / "state.json") or {}).get("tasks", {})
     return state.get(entry, {}).get("status") == "done"
@@ -188,17 +300,7 @@ def static_parts(root: Path, index: SourceIndex, quick: Mapping, engine: Mapping
     key = (path, function, family)
     parts: dict[str, Part] = {}
     parts["quick"] = quick_part(quick.get(key, []), rules, quick_version) if covers(QUICK_LANGUAGES, language) else NONE
-    if language not in ENGINE_LANGUAGES:
-        parts["engine"] = NONE
-    elif engine_state is not None:
-        parts["engine"] = engine_state
-    else:
-        result = engine_result or {}
-        questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
-        parts["engine"] = engine_part(
-            engine.get(key, []), completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
-            vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
-        )  # fmt: skip
+    parts["engine"] = recorded_engine(language, engine.get(key, []), engine_result, repository)
     lines = index.lines(path)
     parts["source"] = source_part(lines, language, function)
     parts["roles"] = roles_part(function_of(lines, path, language, function), inferred, language)
@@ -237,21 +339,34 @@ def export(clone: Path, pin: str, target: Path) -> bool:
 
 def main() -> int:
     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
-    parser.add_argument("--labels", type=Path, required=True)
-    parser.add_argument("--units", type=Path, required=True, help="units.json of the harvest plane")
+    parser.add_argument("--labels", type=Path)
+    parser.add_argument("--units", type=Path, help="units.json of the harvest plane")
     parser.add_argument("--catalog", type=Path, default=DEFAULT_CATALOG)
     parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "openultrasast")
     parser.add_argument("--engine", type=Path, default=RESULTS / "plane" / "harvest-engine")
     parser.add_argument("--verify-run", type=Path, default=RESULTS / "plane" / "harvest-verify")
     parser.add_argument("--roles-run", type=Path, default=RESULTS / "plane" / "harvest-roles")
     parser.add_argument("--v46", type=Path, default=RESULTS / "plane" / "validation-46")
-    parser.add_argument("--scratch", type=Path, required=True)
+    parser.add_argument("--scratch", type=Path)
     parser.add_argument("--min-free-gb", type=float, default=1.2)
-    parser.add_argument("--counts", type=Path, required=True)
+    parser.add_argument("--counts", type=Path)
     parser.add_argument("--store", help="OUSAST_MEMORY spec (default: the environment's)")
     parser.add_argument("--limit", type=int, default=0, help="stop after this many targets (a test)")
     parser.add_argument("--kinds", default="pairs,pin", help="which targets: pairs, pin (population and recipe pins)")
+    parser.add_argument("--results-root", type=Path, default=RESULTS)
+    parser.add_argument("--dry-run", action="store_true", help="read local stored examples and print engine changes by source; no writes")
     args = parser.parse_args()
+    if args.dry_run:
+        spec = args.store or os.environ.get("OUSAST_MEMORY") or ""
+        if spec and not spec.startswith("file://"):
+            parser.error("--dry-run requires a local file:// store (no network)")
+        store = memory.open_store(spec)
+        if not isinstance(store, memory.FileStore):
+            parser.error("--dry-run requires a local file:// store (no network)")
+        print(json.dumps({"dry_run": True, "by_source": dry_run(store, args.results_root, args.cache)}, sort_keys=True))
+        return 0
+    if any(getattr(args, name) is None for name in ("labels", "units", "scratch", "counts")):
+        parser.error("--labels, --units, --scratch and --counts are required unless --dry-run is used")
 
     labels = [r for r in jsonl(args.labels) if r["unit"] == "pin" and r["source"] not in CONDITIONAL_SOURCES and not r.get("conditional")]
     units = json.loads(args.units.read_text(encoding="utf-8"))
@@ -314,7 +429,7 @@ def main() -> int:
                         continue
                     split = {r["split"] for r in rows} | {unit["source"]}
                     population = "independent-v1" if "population-v1" in split else "independent-v2"
-                    engine_result = read_json(RESULTS / population / "scans" / f"{case_id}--{rows[0]['pin_role']}.json")
+                    engine_result = resolve_scan(args.results_root / population / "scans", case_id, rows[0]["pin_role"])
                 stems = [(args.verify_run, n) for n in names] + [(args.v46, unit["ref"])]
                 owner = next(
                     (n for n in names if f"{n}-roles" in {e for e in (read_json(args.roles_run / "state.json") or {}).get("tasks", {})}),

benchmarks/learn/engine_task_entry.py

@@ -0,0 +1,173 @@
+"""AX task entry: object-scoped transfers and stdlib only; result PUT is completion."""
+
+from __future__ import annotations
+
+import http.client
+import json
+import os
+import shutil
+import subprocess
+import sys
+import tarfile
+import time
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+
+class TaskEntryError(RuntimeError):
+    """A task-entry diagnostic containing only locally controlled text."""
+
+
+def transfer(request, destination=None, *, readiness_budget=None):
+    """Wait for actor egress on startup; bound later transfers to three attempts."""
+    deadline = time.monotonic() + readiness_budget if readiness_budget is not None else None
+    delay = 0.5
+    attempt = 0
+    while True:
+        remaining = deadline - time.monotonic() if deadline is not None else 60
+        if remaining <= 0:
+            raise TaskEntryError("egress not open after 60 s")
+        attempt += 1
+        try:
+            with urllib.request.urlopen(request, timeout=min(60, remaining)) as response:
+                if destination is not None:
+                    with destination.open("wb") as output:
+                        shutil.copyfileobj(response, output)
+            return
+        except urllib.error.HTTPError as exc:
+            exc.close()
+            if exc.code != 403 and not 500 <= exc.code < 600:
+                raise
+            if deadline is None and attempt >= 3:
+                raise
+        except (urllib.error.URLError, OSError, http.client.HTTPException):
+            if deadline is None and attempt >= 3:
+                raise
+        if deadline is not None:
+            remaining = deadline - time.monotonic()
+            if remaining <= 0:
+                raise TaskEntryError("egress not open after 60 s") from None
+            time.sleep(min(delay, remaining))
+            delay = min(delay * 2, 8)
+        else:
+            time.sleep(attempt)
+
+
+def download(url, destination, *, readiness_budget=None):
+    transfer(url, destination, readiness_budget=readiness_budget)
+
+
+def unpack(source, destination):
+    """Reject links and traversal; source archives contain only regular files/directories."""
+    with tarfile.open(source) as archive:
+        for member in archive.getmembers():
+            target = (destination / member.name).resolve()
+            if not target.is_relative_to(destination.resolve()) or not (member.isfile() or member.isdir()):
+                raise ValueError("unsafe source archive member")
+        archive.extractall(destination, filter="data")
+
+
+def failure_record(pin, reason):
+    return {
+        **pin,
+        "done": True,
+        "status": "failed",
+        "reason": reason,
+        "units": [
+            {
+                **unit,
+                "status": "failed" if unit.get("supported", True) else "unsupported",
+                "reason": reason,
+                "instrument": {"files": [], "bytes": 0, "jvm": []},
+                "questions": {},
+                "questions_asked": [],
+                "questions_completed": [],
+                "witness_rows": [],
+                "traces": [],
+            }
+            for unit in pin.get("units", [])
+        ],
+    }
+
+
+def main(*, work_root=Path("/tmp"), worker_script=None, readiness_budget=60):
+    pin = {}
+    failed = False
+    output = work_root / "out"
+    result = output / "result.json"
+    step = "prepare output"
+    try:
+        output.mkdir(parents=True, exist_ok=True)
+        result.unlink(missing_ok=True)
+        questions = work_root / "questions.json"
+        step = "GET questions"
+        download(os.environ["QUESTIONS_URL"], questions, readiness_budget=readiness_budget)
+        step = "parse questions"
+        pin = json.loads(questions.read_text())
+        if not isinstance(pin, dict):
+            pin = {}
+            raise ValueError("questions object must contain a prepared pin")
+        source = work_root / "source.tar"
+        step = "GET source"
+        download(os.environ["SOURCE_URL"], source)
+        step = "unpack source"
+        case = work_root / "case"
+        case.mkdir(parents=True, exist_ok=True)
+        unpack(source, case)
+        step = "configure worker"
+        deadline = int(os.environ.get("DEADLINE", "900"))
+        command = [
+            sys.executable,
+            str(worker_script or Path(__file__).with_name("engine_trace_worker.py")),
+            "--worker",
+            str(questions),
+            "--root",
+            str(case),
+            "--output",
+            str(result),
+            "--heap-profile",
+            "cluster",
+            "--deadline",
+            str(deadline),
+            "--question-deadline",
+            os.environ.get("QUESTION_DEADLINE", "120"),
+        ]
+        step = "run worker"
+        completed = subprocess.run(command, timeout=deadline + 5, check=False)
+        if completed.returncode:
+            raise TaskEntryError(f"worker exited {completed.returncode}")
+        step = "read worker result"
+        if not result.is_file() or not json.loads(result.read_text()).get("done"):
+            raise TaskEntryError("worker did not write a completed result")
+    except Exception as exc:
+        failed = True
+        # Avoid including exception text containing a presigned URL in durable output.
+        if isinstance(exc, urllib.error.HTTPError):
+            detail = f"HTTP {exc.code}"
+        else:
+            detail = str(exc) if isinstance(exc, TaskEntryError) else type(exc).__name__
+        reason = f"{step}: {detail}"
+        try:
+            output.mkdir(parents=True, exist_ok=True)
+            result.write_text(json.dumps(failure_record(pin, reason)) + "\n")
+        except Exception:
+            print("task entry could not write failure result", file=sys.stderr)
+    step = "pack result"
+    try:
+        archive = work_root / "result.tar"
+        with tarfile.open(archive, "w") as tar:
+            for path in sorted(output.iterdir()):
+                tar.add(path, arcname=path.name)
+        data = archive.read_bytes()
+        step = "PUT result"
+        request = urllib.request.Request(os.environ["RESULT_URL"], data=data, method="PUT", headers={"Content-Type": "application/x-tar"})
+        transfer(request, readiness_budget=readiness_budget if failed else None)
+    except Exception as exc:
+        detail = f"HTTP {exc.code}" if isinstance(exc, urllib.error.HTTPError) else type(exc).__name__
+        print(f"{step}: {detail}; dispatcher deadline remains authoritative", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/learn/engine_trace.py

@@ -0,0 +1,985 @@
+"""Resumable labelled-function traces with Docker and Kubernetes execution lanes.
+
+Pair blob identities resolve through catalogs to real source commits. The analyzer
+is git archive HEAD src plus a hashed snapshot of this uncommitted tool.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import copy
+import fcntl
+import hashlib
+import json
+import os
+import re
+import shutil
+import subprocess
+import sys
+import tempfile
+import textwrap
+import threading
+import time
+import tomllib
+from collections import Counter, defaultdict
+from pathlib import Path
+
+from engine_trace_k8s import IMAGE, Kubernetes
+from engine_trace_queue import Dispatcher
+from engine_trace_worker import EXCLUDED_DIRS, safe_path, unit_record, write_json
+
+from openultrasast.learn.examples import read_jsonl
+from openultrasast.learn.labels import load_sources, repo_name
+from openultrasast.model.specs import taint_specs
+from openultrasast.model.taint import request_params
+from openultrasast.pairs import _materialize_side, load_pair_catalog
+from openultrasast.plane.engine_alerts import export_pin
+from openultrasast.plane.memory import FileStore
+from openultrasast.plane.tasks.alerts import engine_languages
+from openultrasast.preprocess import detect_language
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
+from benchmarks.ax.batch import schedule
+
+ROOT = Path(__file__).resolve().parents[2]
+DEFAULT_OUT = Path.home() / "ousast-results/plane/engine-trace"
+SNAPSHOT_FILES = (
+    "benchmarks/learn/engine_trace.py",
+    "benchmarks/learn/engine_trace_worker.py",
+    "benchmarks/learn/engine_trace_k8s.py",
+    "benchmarks/learn/engine_trace_queue.py",
+    "benchmarks/learn/engine_trace_ax.py",
+    "benchmarks/ax/batch.py",
+    "benchmarks/learn/engine_task_entry.py",
+    "benchmarks/learn/engine_worker_service.py",
+    "benchmarks/learn/engine_trace_parse.py",
+    "src/openultrasast/model/taint.py",
+    "src/openultrasast/cpg/queries/taint.sc",
+)
+MATERIALIZATION_VERSION = 3
+EXPORT_CAP = 20 * 1024 * 1024
+
+STATUSES = ("path", "asked-nothing", "failed", "timeout", "unsupported")
+
+
+def digest(value):
+    return hashlib.sha256(json.dumps(value, sort_keys=True).encode()).hexdigest()
+
+
+def pin_name(pin):
+    return digest([pin["repo"], pin["pin"]]) + ".json"
+
+
+def join_units(units, examples, include_typescript=False):
+    index = {row["id"]: row for row in examples}
+    missing = [u["unit"] for u in units if u["unit"] not in index]
+    if missing:
+        raise ValueError(f"example snapshot missing {len(missing)}/{len(units)} unit IDs; supply --examples JSONL; first: {missing[0]}")
+    covered = engine_languages() - {"typescript"}
+    if include_typescript:
+        covered |= {"typescript"}
+    result = []
+    for unit in units:
+        example = index[unit["unit"]]
+        if any(example[key] != unit[key] for key in ("family", "label", "group")):
+            raise ValueError(f"unit/example identity mismatch: {unit['unit']}")
+        file, sep, function = example["candidate"].partition("::")
+        if not sep or not function:
+            raise ValueError(f"unit has no labelled function: {unit['unit']}")
+        language = detect_language(Path(file)) or example.get("language", "other")
+        model_language = "javascript" if language == "typescript" else language
+        spec = taint_specs(language=model_language).get(unit["family"]) if language in covered else None
+        result.append(
+            {
+                **unit,
+                "repo": example["repo"],
+                "pin": example["pin"],
+                "file": file,
+                "function": function,
+                "language": language,
+                "source": example["source"],
+                "excerpt": example.get("excerpt", example["source"] == "pairs" or example["source"].startswith("advisory-fixes")),
+                "pin_role": example.get("pin_role", ""),
+                "supported": spec is not None,
+                "reason": "" if spec else f"no admitted taint model: {language}/{unit['family']}",
+            }
+        )
+    return result
+
+
+def make_plan(
+    units,
+    out,
+    only_family=None,
+    limit=0,
+    *,
+    include_typescript=False,
+    only_source=(),
+    rerun_status=("failed",),
+    allow_excerpt_fallback=False,
+):
+    grouped = defaultdict(list)
+    for unit in units:
+        if (not only_family or unit["family"] == only_family) and (not only_source or unit["source"] in only_source):
+            # Include admission options even when a unit's support happens to stay
+            # unchanged. Old results cannot mark a different selection as complete.
+            identity = {
+                "unit": unit,
+                "include_typescript": include_typescript,
+                "materialization_version": MATERIALIZATION_VERSION,
+                "allow_excerpt_fallback": allow_excerpt_fallback,
+            }
+            grouped[unit["repo"], unit["pin"]].append({**unit, "input_digest": digest(identity)})
+    plan = []
+    for (repo, pin), rows in sorted(grouped.items()):
+        item = {
+            "repo": repo,
+            "pin": pin,
+            "materialization_version": MATERIALIZATION_VERSION,
+            "units": sorted(rows, key=lambda r: r["unit"]),
+        }
+        item["selection"] = digest(item["units"])
+        done = out / pin_name(item)
+        if done.exists():
+            old = json.loads(done.read_text())
+            recorded = {r["unit"]: r.get("input_digest") for r in old.get("units", [])}
+            if (
+                old.get("done")
+                and not old.get("fallback_pending")
+                and old.get("materialization_version") == MATERIALIZATION_VERSION
+                and not any(r.get("status") in rerun_status for r in old.get("units", []) if r["unit"] in {u["unit"] for u in rows})
+                and all(recorded.get(r["unit"]) == r["input_digest"] for r in rows)
+            ):
+                continue
+        sources = {r["source"] for r in rows}
+        # Inventory medians: harvest sides 31s, advisory sides 47s; v1 pins 182s, v2 pins 498s.
+        item["estimated_seconds"] = (
+            (47 if any(s.startswith("advisory") for s in sources) else 31)
+            if any(r.get("excerpt") for r in rows) or any(s.startswith("advisory") for s in sources)
+            else (182 if sources == {"population-v1"} else 498)
+        )
+        if not any(r["supported"] for r in rows):
+            item["estimated_seconds"] = 0
+        plan.append(item)
+    # limit counts runnable pins, not unsupported records.
+    if limit:
+        chosen = []
+        count = 0
+        for pin in plan:
+            runnable = any(r["supported"] for r in pin["units"])
+            if runnable and count >= limit:
+                continue
+            chosen.append(pin)
+            count += runnable
+        return chosen
+    return plan
+
+
+class Inputs:
+    """Resolve spent corpora; fetch pair source commits into the host cache."""
+
+    def __init__(self, root, cache, allow_excerpt_fallback=False):
+        self.cache = cache
+        self.allow_excerpt_fallback = allow_excerpt_fallback
+        self.catalog_rows = defaultdict(list)
+        for catalog in (root / "benchmarks/pairs").rglob("catalog.toml"):
+            for item in tomllib.loads(catalog.read_text()).get("pair", []):
+                self.catalog_rows[item["name"]].append((catalog.parent.name, item))
+        self.pairs = []
+        self.repos = defaultdict(list)
+        sources = load_sources()
+        for source in sources.sources:
+            if source.kind == "pairs":
+                self.pairs.extend(load_pair_catalog(root / source.files[0]))
+            elif source.kind == "population":
+                if source.id not in {"population-v1", "population-v2"}:
+                    continue
+                data = tomllib.loads((root / source.files[0]).read_text())
+                for case in data.get("case", []):
+                    self.repos[repo_name(case["repo"])].append((cache / "independent" / case["id"], case))
+            elif source.kind == "recipes":
+                for file in source.files:
+                    data = tomllib.loads((root / file).read_text())
+                    recipe = data.get("repo", data)
+                    # Recipes carry a top-level URL and a pinned checkout cache.
+                    if isinstance(recipe, dict):
+                        self.repos[repo_name(str(recipe.get("url", "")))].append((cache / "repos" / str(data.get("name", "")), data))
+        self.hashes = {}
+
+    def pair_for(self, row):
+        if row["pin_role"] not in {"fixed", "vulnerable", "vuln"}:
+            raise ValueError(f"ambiguous pair side: {row['pin_role']}")
+        side = "fixed" if row["pin_role"] == "fixed" else "vuln"
+        candidates = [
+            c for c in self.pairs if c.relpath == row["file"] and repo_name(c.repo or f"{c.slice}:{c.name}") == repo_name(row["repo"])
+        ]
+        matches = []
+        for case in candidates:
+            path = case.fixed_file if side == "fixed" else case.vuln_file
+            if path.is_file():
+                if path not in self.hashes:
+                    data = path.read_bytes()
+                    # Harvest hashes decoded text, so universal newline conversion is part
+                    # of its identity (one recorded ThreatByte excerpt uses CRLF).
+                    normalized = path.read_text().encode("utf-8")
+                    self.hashes[path] = {
+                        hashlib.sha1(b"blob " + str(len(body)).encode() + b"\0" + body).hexdigest() for body in (data, normalized)
+                    }
+                if row["pin"] in self.hashes[path]:
+                    matches.append(case)
+        if len(matches) == 1:
+            return matches[0], side
+        if len(candidates) == 1:
+            # Earlier stores used label_pin rather than the excerpt digest.
+            from openultrasast.learn.examples import label_pin
+
+            case = candidates[0]
+            if row["pin"] == label_pin({"source": "pairs", "source_ref": case.name, "pin_role": row["pin_role"]}):
+                return case, side
+        raise ValueError(f"cannot uniquely resolve excerpt {row['repo']}@{row['pin']}:{row['file']}")
+
+    def catalog_for(self, case):
+        matches = [
+            (source, item)
+            for source, item in self.catalog_rows[case.name]
+            if item.get("relpath") == case.relpath and repo_name(item.get("repo", "")) == repo_name(case.repo)
+        ]
+        if len(matches) != 1:
+            raise ValueError(f"ambiguous catalog source for {case.name}")
+        return matches[0]
+
+    def source_identity(self, case, side):
+        _, item = self.catalog_for(case)
+        parent, commit = item.get("parent", ""), item.get("commit", "")
+        if side not in {"vuln", "fixed"} or not all(re.fullmatch(r"[0-9a-fA-F]{40}", sha) for sha in (parent, commit)) or parent == commit:
+            raise ValueError(f"ambiguous catalog commits for {case.name}: need distinct parent and commit SHAs")
+        repo = item.get("repo", "")
+        if not repo:
+            raise ValueError(f"missing catalog repository for {case.name}")
+        return repo, parent if side == "vuln" else commit
+
+    def fetch_source(self, repo, commit, roots):
+        clone = self.cache / "trace-sources" / digest([repo, commit, roots])
+        clone.mkdir(parents=True, exist_ok=True)
+        url = repo if repo.startswith("https://") else f"https://github.com/{repo.removesuffix('.git')}.git"
+        env = {**os.environ, "GIT_TERMINAL_PROMPT": "0"}
+        env.pop("GIT_NO_LAZY_FETCH", None)  # sparse checkout may fetch the selected blobs
+
+        def git(*args):
+            return subprocess.run(["git", "-C", str(clone), *args], env=env, capture_output=True, text=True, check=True, timeout=300)
+
+        git("init")
+        git("config", "remote.origin.url", url)
+        git("config", "remote.origin.promisor", "true")
+        git("config", "remote.origin.partialclonefilter", "blob:none")
+        git("fetch", "--depth", "1", "--filter=blob:none", url, commit)
+        git("sparse-checkout", "init", "--cone")
+        git("sparse-checkout", "set", "--", *roots)
+        git("checkout", "--detach", "--force", commit)
+        actual = git("rev-parse", "HEAD").stdout.strip()
+        if actual != commit:
+            raise ValueError(f"fetched commit mismatch: {actual} != {commit}")
+        return clone
+
+    def export_local(self, clone, pin, target):
+        with tempfile.TemporaryDirectory(prefix="trace-export-", dir=target.parent) as scratch:
+            export_pin(clone, pin["pin"], Path(scratch))
+            roots = sorted({source_root(r["file"], r["language"]) for r in pin["units"]})
+            proof = export_sources(Path(scratch), target, pin["units"], roots)
+        return {**proof, "mode": "source", "commit": pin["pin"]}
+
+    def materialize(self, pin, target):
+        rows = pin["units"]
+        for row in rows:
+            safe_path(target, row["file"])
+            if row.get("mapping_error"):
+                raise ValueError(row["mapping_error"])
+        if any(r.get("excerpt") or r["source"].startswith("advisory-fixes") for r in rows):
+            resolved = [self.pair_for(row) for row in rows]
+            identities = set()
+            catalog_files = []
+            for case, side in resolved:
+                _, item = self.catalog_for(case)
+                parent, commit = item.get("parent", ""), item.get("commit", "")
+                if parent != commit and all(re.fullmatch(r"[0-9a-fA-F]{40}", sha) for sha in (parent, commit)):
+                    identities.add(self.source_identity(case, side))
+                else:
+                    file = case.fixed_file if side == "fixed" else case.vuln_file
+                    data = file.read_text()
+                    if is_catalog_excerpt(data):
+                        data = parseable_excerpt(file, data)
+                        mode = "excerpt-parseable"
+                    else:
+                        # Validate BOTH twins before treating this as a complete-file pair.
+                        for twin in (case.vuln_file, case.fixed_file):
+                            verify_catalog_file(twin)
+                        mode = "catalog-file"
+                    catalog_files.append((case, side, mode, data))
+            if catalog_files:
+                if identities:
+                    raise ValueError("mixed source and catalog-file materialization at one pair pin")
+                for case, side, mode, data in catalog_files:
+                    _materialize_side(target, case, side=side)
+                    if mode == "excerpt-parseable":
+                        safe_path(target, case.relpath).write_text(data)
+                verify_functions(target, rows)
+                modes = {case.relpath: mode for case, _, mode, _ in catalog_files}
+                for row in rows:
+                    if modes[row["file"]] == "excerpt-parseable":
+                        row["evidence_scope"] = "in-function-only"
+                return {
+                    "mode": "excerpt-parseable" if "excerpt-parseable" in modes.values() else "catalog-file",
+                    "files": sum(p.is_file() for p in target.rglob("*")),
+                    "bytes": sum(p.stat().st_size for p in target.rglob("*") if p.is_file()),
+                    "roots": sorted({source_root(r["file"], r["language"]) for r in rows}),
+                }
+            if len(identities) != 1:
+                raise ValueError("ambiguous source commits at one pair pin")
+            repo, commit = identities.pop()
+            roots = sorted({source_root(row["file"], row["language"]) for row in rows})
+            try:
+                clone = self.fetch_source(repo, commit, roots)
+                proof = export_sources(clone, target, rows, roots)
+            except (OSError, ValueError, subprocess.SubprocessError) as exc:
+                if not self.allow_excerpt_fallback:
+                    raise
+                shutil.rmtree(target, ignore_errors=True)
+                for case, side in resolved:
+                    _materialize_side(target, case, side=side)
+                return {"mode": "excerpt-fallback", "reason": str(exc), "repo": repo, "commit": commit}
+            verify_functions(target, rows)
+            return {**proof, "mode": "source", "repo": repo, "commit": commit}
+        if all(r["source"] == "dev-php" for r in rows):
+            # Export the exact requested commit from the local recipe checkout.
+            for recipe in (ROOT / "benchmarks/repos").glob("*.toml"):
+                data = tomllib.loads(recipe.read_text())
+                if repo_name(str(data.get("url", ""))) == repo_name(pin["repo"]):
+                    checkout = self.cache / "repos" / data["name"] / pin["pin"][:12]
+                    return self.export_local(checkout, pin, target)
+        for clone, _ in self.repos[repo_name(pin["repo"])]:
+            check = subprocess.run(["git", "-C", str(clone), "cat-file", "-e", pin["pin"] + "^{commit}"], capture_output=True)
+            if check.returncode == 0:
+                return self.export_local(clone, pin, target)
+        raise ValueError(f"no local clone contains {pin['repo']}@{pin['pin']}")
+
+
+def is_catalog_excerpt(data):
+    # Older Python exports have this header but no upstream_start field.
+    return re.match(r"\s*(?://|#)\s*Provenance:", data) is not None
+
+
+def parseable_excerpt(path, data):
+    """Dedent the code independently of its unindented provenance comments."""
+    language = detect_language(path)
+    lines = data.splitlines(keepends=True)
+    start = 0
+    while start < len(lines) and (not lines[start].strip() or lines[start].lstrip().startswith(("#", "//"))):
+        start += 1
+    header = "".join(lines[:start])
+    if language == "python":
+        header = re.sub(r"(?m)^(\s*)//", r"\1#", header)
+    data = header + textwrap.dedent("".join(lines[start:]))
+    if not "".join(lines[start:]).strip():
+        raise ValueError(f"excerpt does not parse: empty code ({path.name})")
+    if language == "python":
+        try:
+            ast.parse(data, filename=str(path))
+        except SyntaxError as exc:
+            raise ValueError(f"excerpt does not parse: {exc}") from exc
+    else:
+        from openultrasast.semantic.cst import parse_with_cst
+
+        parsed = parse_with_cst(str(path), data, language)
+        if parsed is None:
+            raise ValueError(f"excerpt does not parse: no cheap parser for {language} ({path.name})")
+        if not parsed.parse_ok:
+            raise ValueError(f"excerpt does not parse: {language} {parsed.reason} ({path.name})")
+    return data
+
+
+def verify_catalog_file(path):
+    """Admit complete source only; syntactically valid provenance excerpts still fail."""
+    data = path.read_text()
+    if is_catalog_excerpt(data):
+        raise ValueError(f"catalog side is an upstream excerpt without distinct commits: {path.name}")
+    language = detect_language(path)
+    if not data.strip():
+        raise ValueError(f"catalog side is empty: {path.name}")
+    if language == "python":
+        try:
+            ast.parse(data, filename=str(path))
+        except SyntaxError as exc:
+            raise ValueError(f"catalog side is not complete Python source: {path.name}: {exc.msg}") from exc
+    else:
+        from openultrasast.semantic.extra import grammar_for
+
+        grammar = grammar_for(language)
+        if grammar is None:
+            raise ValueError(f"cannot verify complete catalog source for {language}: {path.name}")
+        from tree_sitter import Parser
+
+        if Parser(grammar).parse(data.encode()).root_node.has_error:
+            raise ValueError(f"catalog side is not complete {language} source: {path.name}")
+
+
+def source_root(file, language):
+    parts = Path(file).parts
+    if language == "java":
+        for i in range(len(parts) - 2):
+            if parts[i : i + 3] == ("src", "main", "java"):
+                return "/".join(parts[: i + 3])
+    # Monorepo and source-layout containers are not themselves a package.
+    if len(parts) > 2 and parts[0] in {"packages", "apps", "src", "web"}:
+        return "/".join(parts[:2])
+    return parts[0] if len(parts) > 1 else "."
+
+
+def local_imports(source, rows):
+    """Cheap one-level Python imports from labelled files, without importing code."""
+    found = set()
+    for row in rows:
+        if row["language"] != "python":
+            continue
+        file = safe_path(source, row["file"])
+        try:
+            tree = ast.parse(file.read_bytes())
+        except (SyntaxError, ValueError):
+            continue
+        for node in ast.walk(tree):
+            if isinstance(node, ast.Import):
+                modules, bases = [alias.name for alias in node.names], [source, source / "src"]
+            elif isinstance(node, ast.ImportFrom):
+                modules = [node.module or ""]
+                modules += [".".join(filter(None, (node.module, alias.name))) for alias in node.names if alias.name != "*"]
+                bases = [file.parents[node.level - 1]] if node.level and node.level <= len(file.parents) else [source, source / "src"]
+            else:
+                continue
+            for base in bases:
+                for module in modules:
+                    path = base.joinpath(*module.split("."))
+                    for candidate in (path.with_suffix(".py"), path / "__init__.py"):
+                        if candidate.is_relative_to(source) and candidate.is_file() and not candidate.is_symlink():
+                            rel = candidate.relative_to(source).as_posix()
+                            if not any(part in EXCLUDED_DIRS for part in Path(rel).parts):
+                                found.add(rel)
+    return found
+
+
+def export_sources(source, target, rows, roots=(".",), cap=EXPORT_CAP):
+    """Copy labels intact, narrowing oversized roots before bounding context bytes."""
+    labelled = {r["file"] for r in rows}
+    languages = {r["language"] for r in rows}
+    files = set()
+    protected = {parent.as_posix() for rel in labelled for parent in Path(rel).parents}
+    for directory, dirs, names in os.walk(source):
+        dirs[:] = sorted(
+            d
+            for d in dirs
+            if d != ".git"
+            and not (Path(directory) / d).is_symlink()
+            and (d not in EXCLUDED_DIRS or (Path(directory) / d).relative_to(source).as_posix() in protected)
+        )
+        for name in names:
+            file = Path(directory) / name
+            rel = file.relative_to(source).as_posix()
+            excluded = any(part in EXCLUDED_DIRS for part in Path(rel).parts[:-1])
+            if not file.is_symlink() and (
+                rel in labelled
+                or (
+                    not excluded
+                    and detect_language(file) in languages
+                    and any(root == "." or rel == root or rel.startswith(root + "/") for root in roots)
+                )
+            ):
+                files.add(rel)
+    missing = labelled - files
+    if missing:
+        raise ValueError(f"labelled files missing or excluded from export: {sorted(missing)}")
+    imports = local_imports(source, rows)
+    files.update(imports)
+    sizes = {rel: safe_path(source, rel).stat().st_size for rel in files}
+    original_roots = sorted(roots)
+    roots = original_roots
+    # Avoid lexicographically truncating a whole large package such as vllm.
+    if sum(sizes.values()) > cap:
+        roots = sorted({str(Path(rel).parent) for rel in labelled})
+        files = {
+            rel for rel in files if rel in labelled or rel in imports or any(root == "." or rel.startswith(root + "/") for root in roots)
+        }
+    proof = {
+        "bytes": 0,
+        "files": 0,
+        "cap_bytes": cap,
+        "cap_hit": False,
+        "excludes": sorted(EXCLUDED_DIRS),
+        "roots": roots,
+        "requested_roots": original_roots,
+        "scope_narrowed": roots != original_roots,
+        "import_files": sorted(imports),
+    }
+    for rel in sorted(files, key=lambda rel: (rel not in labelled, rel)):
+        file = safe_path(source, rel)
+        size = sizes[rel]
+        # The cap bounds context only. No labelled source is ever cut or omitted.
+        if proof["bytes"] + size > cap:
+            proof["cap_hit"] = True
+            if rel not in labelled:
+                continue
+        dest = safe_path(target, rel)
+        dest.parent.mkdir(parents=True, exist_ok=True)
+        dest.write_bytes(file.read_bytes())
+        proof["bytes"] += size
+        proof["files"] += 1
+    return proof
+
+
+def verify_functions(root, rows):
+    for row in rows:
+        data = safe_path(root, row["file"]).read_text(errors="replace")
+        if not re.search(r"(?<![\w$])" + re.escape(row["function"]) + r"(?![\w$])", data):
+            raise ValueError(f"labelled function not found in fetched source: {row['file']}::{row['function']}")
+
+
+def freeze_source(target):
+    target.mkdir(parents=True)
+    archive = target.parent / "source.tar"
+    with archive.open("wb") as stream:
+        subprocess.run(["git", "-C", str(ROOT), "archive", "HEAD", "src"], stdout=stream, check=True)
+    subprocess.run(["tar", "-xf", str(archive), "-C", str(target)], check=True)
+    archive.unlink()
+    overlay = {}
+    for name in SNAPSHOT_FILES:
+        data = (ROOT / name).read_bytes()
+        dest = target / name
+        dest.parent.mkdir(parents=True, exist_ok=True)
+        dest.write_bytes(data)
+        overlay[name] = hashlib.sha256(data).hexdigest()
+    head = subprocess.run(["git", "-C", str(ROOT), "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
+    return {"head": head, "overlay_sha256": overlay}
+
+
+def docker_command(source, checkout, out, name, image, deadline, question_deadline):
+    return [
+        "docker",
+        "run",
+        "--rm",
+        "--init",
+        "--name",
+        name,
+        "--network",
+        "none",
+        "--memory",
+        "3g",
+        "--pull",
+        "never",
+        "--entrypoint",
+        "python",
+        "-e",
+        "PYTHONPATH=/frozen/src",
+        "-v",
+        f"{source}:/frozen:ro",
+        "-v",
+        f"{checkout}:/case:ro",
+        "-v",
+        f"{out}:/out",
+        image,
+        "/frozen/benchmarks/learn/engine_trace_worker.py",
+        "--worker",
+        "/out/pin.json",
+        "--deadline",
+        str(deadline),
+        "--heap-profile",
+        "vm",
+        "--question-deadline",
+        str(question_deadline),
+    ]
+
+
+def prepare_questions(pin, root, question_deadline):
+    """Prepare metadata on the host, before entering the dependency-minimal image."""
+    active = [u for u in pin["units"] if u["supported"]]
+    from openultrasast.learn.features import entry_names
+    from openultrasast.mapping import php_hook_callbacks
+    from openultrasast.plane.harvest import declares
+    from openultrasast.preprocess import preprocess_repository
+
+    _, targets = preprocess_repository(root)
+    entries, _ = entry_names(root, {u["file"] for u in active})
+    hooks = ";".join(f"{h}:{f}" for h, functions in php_hook_callbacks(root, targets).items() for f in functions)
+    requests = {}
+    for u in active:
+        specs = taint_specs(language="javascript" if u["language"] == "typescript" else u["language"])
+        if not declares(safe_path(root, u["file"]).read_text(errors="replace").splitlines(), u["file"], u["function"]):
+            raise ValueError(f"labelled function not declared: {u['file']}::{u['function']}")
+        excerpt = u.get("evidence_scope") == "in-function-only" or pin.get("materialization", {}).get("mode") == "excerpt-fallback"
+        params = request_params(
+            specs[u["family"]],
+            file=u["file"],
+            function=u["function"],
+            trace=True,
+            parameter_sources=excerpt or u["function"] in entries,
+            call_depth=0 if excerpt else 3,
+            hook_callbacks=hooks,
+        )
+        params["questionDeadline"] = str(question_deadline)
+        requests[u["unit"]] = params
+    return requests
+
+
+def run_pin(pin, source, provenance, inputs, out, args):
+    executor = getattr(args, "executor", "docker")
+    node = None
+    with tempfile.TemporaryDirectory(prefix="trace-pin-", dir=out) as scratch:
+        scratch = Path(scratch)
+        checkout, output = scratch / "case", scratch / "output"
+        output.mkdir()
+        try:
+            with getattr(args, "input_lock", threading.Lock()):
+                pin = {**pin, "materialization": inputs.materialize(pin, checkout)}
+            write_json(output / "pin.json", {**pin, "questions": prepare_questions(pin, checkout, args.question_deadline)})
+            name = "ousast-trace-" + pin_name(pin)[:16]
+            started = time.monotonic()
+            if executor in {"k8s", "k8s-jobs"}:
+                done, node = args.kubernetes.execute(source, checkout, output, json.loads((output / "pin.json").read_text()))
+                status, reason = "failed", done.stderr
+                if done.returncode:
+                    (output / "container.log").write_text(reason)
+            elif executor == "ax":
+                done, node = args.ax_dispatcher.execute(checkout, output, json.loads((output / "pin.json").read_text()), pin_name(pin)[:-5])
+                status = "timeout" if done is None else "failed"
+                reason = f"AX deadline {args.deadline}s" if done is None else done.stderr
+            elif executor == "queue":
+                done, node = args.dispatcher.execute(checkout, output, json.loads((output / "pin.json").read_text()), pin_name(pin)[:-5])
+                status, reason = "timeout", f"queue wait deadline {args.queue_timeout}s"
+            else:
+                command = docker_command(source, checkout, output, name, args.image, args.deadline, args.question_deadline)
+                started = time.monotonic()
+                done = None
+                try:
+                    done = subprocess.run(command, capture_output=True, text=True, timeout=args.deadline)
+                    status, reason = "failed", f"container exit {done.returncode}: {done.stderr[-1000:]}"
+                    (output / "container.log").write_text(done.stdout + "\n" + done.stderr)
+                except subprocess.TimeoutExpired:
+                    done = None
+                    status, reason = "timeout", f"container deadline {args.deadline}s"
+                finally:
+                    # Never start the next pin unless this container is known to be gone.
+                    try:
+                        removed = subprocess.run(["docker", "rm", "-f", name], capture_output=True, timeout=20)
+                    except subprocess.SubprocessError as exc:
+                        raise RuntimeError(f"cannot confirm container {name} stopped; refusing another launch") from exc
+                    if done is None and removed.returncode != 0:
+                        raise RuntimeError(f"cannot remove timed-out container {name}; refusing another launch")
+            result_path = output / "result.json"
+            if not result_path.exists() and reason != "AX sandbox failed twice":
+                reason = f"container produced no result: {reason}"
+            record = json.loads(result_path.read_text()) if result_path.exists() else {**pin, "units": []}
+            if not record.get("done"):
+                previous = {u["unit"]: u for u in record["units"]}
+                record["units"] = []
+                for unit in pin["units"]:
+                    row = previous.get(unit["unit"], unit_record(unit, status))
+                    if row["status"] not in {"path", "asked-nothing"}:
+                        row.update(
+                            status=status if unit["supported"] else "unsupported", reason=reason if unit["supported"] else unit["reason"]
+                        )
+                    record["units"].append(row)
+            if executor != "queue":
+                record["container_exit"] = done.returncode if done else None
+            else:
+                record.setdefault("container_exit", None)
+            if done is not None and done.returncode != 0:
+                for row in record["units"]:
+                    if row["supported"]:
+                        row.update(status="failed", reason=reason)
+            record.update(
+                done=True,
+                seconds=time.monotonic() - started,
+                analyzer={"image": args.image, "code": "baked-in-image"} if executor == "ax" else provenance,
+                image=record.get("image", args.image) if executor == "queue" else args.image,
+            )
+            if (output / "container.log").exists():
+                shutil.copyfile(output / "container.log", out / pin_name(pin).replace(".json", ".log"))
+        except (OSError, ValueError, subprocess.SubprocessError) as exc:
+            record = {
+                **pin,
+                "done": True,
+                "analyzer": provenance,
+                "units": [unit_record(u, "failed" if u["supported"] else "unsupported", str(exc)) for u in pin["units"]],
+            }
+        if executor == "ax":
+            record.update(executor="ax", worker=node, worker_ip=node, image=args.image)
+            record["analyzer"] = {"image": args.image, "code": "baked-in-image"}
+            if (output / "ax_attempts.json").exists():
+                record["ax_attempts"] = json.loads((output / "ax_attempts.json").read_text())
+        if getattr(args, "fallback", None):
+            record.update(executor="docker", fallback=args.fallback, ax_attempt=args.ax_attempt, fallback_pending=False)
+            if "ax_attempts" in args.ax_attempt:
+                record["ax_attempts"] = args.ax_attempt["ax_attempts"]
+        if executor in {"k8s", "k8s-jobs"}:
+            record.update(executor="k8s", node=node, image=args.image)
+        if executor == "queue":
+            record.update(executor="queue", worker=record.get("worker", node), pod=record.get("pod"))
+        save_pin(out / pin_name(pin), record)
+        print(
+            json.dumps({"repo": pin["repo"], "pin": pin["pin"], "statuses": dict(Counter(u["status"] for u in record["units"]))}),
+            flush=True,
+        )
+        return any(u["status"] in {"failed", "timeout"} for u in record["units"])
+
+
+def save_pin(path, record):
+    if path.exists():
+        old = json.loads(path.read_text())
+        selected = {u["unit"] for u in record["units"]}
+        record["units"].extend(u for u in old.get("units", []) if u["unit"] not in selected)
+    write_json(path, record)
+
+
+def summary(out):
+    if not out.is_dir():
+        raise ValueError(f"results root does not exist: {out}")
+    family, source = defaultdict(Counter), defaultdict(Counter)
+    pairs = defaultdict(dict)
+    units = {}
+    result_files = 0
+    for path in sorted(out.glob("*.json")):
+        record = json.loads(path.read_text())
+        if not record.get("done"):
+            continue
+        result_files += 1
+        for unit in record["units"]:
+            if unit["unit"] in units:
+                raise ValueError(f"duplicate result for {unit['unit']}")
+            units[unit["unit"]] = unit
+            family[unit["family"]][unit["status"]] += 1
+            source[unit["source"]][unit["status"]] += 1
+            if unit["pair"]:
+                pairs[unit["family"], unit["pair"]][unit["label"]] = unit["status"]
+    asymmetry = Counter()
+    completed = Counter()
+    for sides in pairs.values():
+        if set(sides) != {0, 1}:
+            asymmetry["incomplete-pair"] += 1
+            continue
+        key = {(True, False): "vulnerable-only", (False, True): "fixed-only", (True, True): "both", (False, False): "neither"}[
+            sides[1] == "path", sides[0] == "path"
+        ]
+        asymmetry[key] += 1
+        if all(s in {"path", "asked-nothing"} for s in sides.values()):
+            completed[key] += 1
+    return {
+        "result_files_read": result_files,
+        "units": len(units),
+        "by_family": {k: {s: v[s] for s in STATUSES} for k, v in sorted(family.items())},
+        "by_source": {k: {s: v[s] for s in STATUSES} for k, v in sorted(source.items())},
+        "pair_asymmetry": dict(asymmetry),
+        "pair_asymmetry_both_answered": dict(completed),
+    }
+
+
+def run_plan(plan, source, provenance, inputs, args):
+    """One task per fixed executor lane; progress is written only by the scheduler."""
+    lanes = ["k8s"] * (args.parallel - 1) + ["docker"] if args.executor == "mixed" else [args.executor] * args.parallel
+    if args.executor in {"ax", "mixed-ax"}:
+        lanes = ["ax"] * args.parallel + ["docker"]
+    args.input_lock = threading.Lock()
+    progress = {
+        "pins_done": 0,
+        "pins_total": len(plan),
+        "last_pin": None,
+        "statuses": {},
+        "lanes": dict(Counter(lanes)),
+        "pins_in_flight": 0,
+        "active_lanes": {},
+    }
+    counts = Counter()
+    pending = list(plan)
+    fallbacks = []
+    if args.executor in {"ax", "mixed-ax"}:
+        pending.clear()
+        for pin in plan:
+            path = args.out / pin_name(pin)
+            old = json.loads(path.read_text()) if path.exists() else {}
+            recorded = {u["unit"]: u.get("input_digest") for u in old.get("units", [])}
+            if old.get("fallback_pending") and all(recorded.get(u["unit"]) == u.get("input_digest") for u in pin["units"]):
+                fallbacks.append((pin, old))
+            else:
+                pending.append(pin)
+
+    def execute(pin, lane, attempt=None):
+        if not any(u["supported"] for u in pin["units"]):
+            save_pin(
+                args.out / pin_name(pin), {**pin, "done": True, "units": [unit_record(u, "unsupported", u["reason"]) for u in pin["units"]]}
+            )
+            return False
+        options = copy.copy(args)
+        options.executor = lane
+        if attempt is not None:
+            options.fallback = attempt.get("fallback", "vm-oom")
+            options.ax_attempt = attempt
+        if lane == "docker":
+            options.image = getattr(args, "docker_image", args.image)
+        return run_pin(pin, source, provenance, inputs, args.out, options)
+
+    def complete(pin, lane, failed, progress):
+        result = json.loads((args.out / pin_name(pin)).read_text())
+        selected = {u["unit"] for u in pin["units"]}
+        reasons = [u.get("reason", "") for u in result["units"] if u["unit"] in selected]
+        sandbox_failed = args.executor == "mixed-ax" and result.get("container_exit") == 1 and "AX sandbox failed twice" in reasons
+        if lane == "ax" and (sandbox_failed or any("JVM out of memory" in reason for reason in reasons)):
+            result.update(fallback="vm-sandbox" if sandbox_failed else "vm-oom", fallback_pending=True)
+            write_json(args.out / pin_name(pin), result)
+            return False, result
+        counts.update(u["status"] for u in result["units"] if u["unit"] in selected)
+        progress.update(last_pin={"repo": pin["repo"], "pin": pin["pin"]}, statuses=dict(counts))
+        return failed, None
+
+    return schedule(pending, lanes, args.out, execute, complete, fallbacks=fallbacks, overflow=args.executor != "ax", progress=progress)
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--units", type=Path, default=ROOT / "plane/experiments/exp-005-graph-slice.units.jsonl")
+    parser.add_argument("--examples", type=Path, help="frozen example JSONL instead of the local memory store")
+    parser.add_argument("--memory", type=Path, default=Path.home() / "ousast-results/plane/memory")
+    parser.add_argument("--cache", type=Path, default=Path.home() / ".cache/openultrasast")
+    parser.add_argument("--out", type=Path, default=DEFAULT_OUT)
+    parser.add_argument("--image", help="default: openultrasast:dev for Docker; public 2.0.1 image for k8s/mixed")
+    parser.add_argument("--executor", choices=("docker", "k8s", "k8s-jobs", "mixed", "queue", "ax", "mixed-ax"), default="docker")
+    parser.add_argument("--queue-status", action="store_true")
+    parser.add_argument("--queue-timeout", type=int, default=3600, help="total queue wait including execution, seconds")
+    parser.add_argument("--parallel", type=int, help="AX lanes (default 2, maximum 2); otherwise total lanes (default 1)")
+    parser.add_argument("--ax-bin", default="ax")
+    parser.add_argument("--kubeconfig", type=Path, help="AX kubeconfig; otherwise inherited KUBECONFIG/default CLI config")
+    parser.add_argument("--atespace", default="default")
+    parser.add_argument("--docker-image", default="openultrasast:dev", help="local VM image for AX OOM fallback/mixed overflow")
+    parser.add_argument("--kube-context")
+    parser.add_argument("--kube-namespace", default="ousast-engine")
+    parser.add_argument("--keep-jobs", action="store_true", help="retain Jobs, Secrets and staging objects for debugging")
+    parser.add_argument("--k8s-check", action="store_true")
+    parser.add_argument("--deadline", type=int, default=900)
+    parser.add_argument("--question-deadline", type=int, default=120)
+    parser.add_argument("--limit", type=int, default=0)
+    parser.add_argument("--only-family")
+    parser.add_argument("--only-source", default="", help="comma-separated source catalog IDs")
+    parser.add_argument("--rerun-status", default="failed,timeout", help="comma-separated statuses to re-run (including timeout)")
+    parser.add_argument(
+        "--allow-excerpt-fallback", action="store_true", help="allow fragments only if fetching/exporting resolved source fails"
+    )
+    parser.add_argument("--include-typescript", action="store_true")
+    parser.add_argument("--dry-run", action="store_true")
+    parser.add_argument("--summary", action="store_true")
+    args = parser.parse_args()
+    if args.parallel is None:
+        args.parallel = 2 if args.executor in {"ax", "mixed-ax"} else 1
+    if args.executor in {"ax", "mixed-ax"}:
+        if args.parallel > 2:
+            parser.error("AX allows at most 2 engine tasks in flight")
+        if not args.image or not re.fullmatch(r"[^\s@]+@sha256:[0-9a-f]{64}", args.image):
+            parser.error("AX requires --image by sha256 digest")
+    if args.queue_status:
+        print(json.dumps(Dispatcher(args).status(), indent=2))
+        return 0
+    if args.queue_timeout <= 0:
+        parser.error("--queue-timeout must be positive")
+    if args.executor == "queue" and (not args.image or "@sha256:" not in args.image):
+        parser.error("queue requires --image with the deployed image digest")
+    if args.executor == "queue" and args.deadline > 900:
+        parser.error("queue deadline must be <= Deployment termination grace (900s)")
+    if min(args.deadline, args.question_deadline) <= 0 or args.limit < 0:
+        parser.error("deadlines must be positive; limit must be nonnegative")
+    if args.parallel < 1 or (args.executor == "mixed" and args.parallel < 2):
+        parser.error("--parallel must be positive; mixed needs at least 2 total lanes")
+    if (args.executor in {"k8s", "k8s-jobs", "mixed"} or args.k8s_check) and not args.kube_context:
+        parser.error("--kube-context is required for Kubernetes")
+    args.image = args.image or (IMAGE if args.executor != "docker" or args.k8s_check else "openultrasast:dev")
+    if args.k8s_check:
+        return Kubernetes(args).check()
+    rerun_status = set(filter(None, args.rerun_status.split(",")))
+    if rerun_status - set(STATUSES):
+        parser.error("unknown --rerun-status value")
+    if args.summary:
+        print(json.dumps(summary(args.out), indent=2))
+        return 0
+    # Even inherited opt-ins must not make catalog resolution fetch.
+    os.environ["GIT_NO_LAZY_FETCH"] = "1"
+    os.environ.pop("OPENULTRASAST_PAIRS_NETWORK", None)
+    os.environ.pop("OPENULTRASAST_REPOS_NETWORK", None)
+    examples = read_jsonl(args.examples) if args.examples else [r.row for r in FileStore(args.memory).rows(kind="example")]
+    try:
+        units = join_units(read_jsonl(args.units), examples, args.include_typescript)
+    except ValueError as exc:
+        parser.error(str(exc))
+    inputs = Inputs(ROOT, args.cache, args.allow_excerpt_fallback)
+    # Resolve pair source names for leak-audit coverage before calculating the resume key.
+    for unit in units:
+        if unit.get("excerpt") or unit["source"].startswith("advisory-fixes"):
+            try:
+                case, _ = inputs.pair_for(unit)
+                catalog_source, _ = inputs.catalog_for(case)
+                unit["source"] = catalog_source if catalog_source.startswith("advisory-fixes") else case.slice
+                unit["excerpt"] = True
+            except ValueError as exc:
+                unit["mapping_error"] = str(exc)
+    plan = make_plan(
+        units,
+        args.out,
+        args.only_family,
+        args.limit,
+        include_typescript=args.include_typescript,
+        only_source=set(filter(None, args.only_source.split(","))),
+        rerun_status=rerun_status,
+        allow_excerpt_fallback=args.allow_excerpt_fallback,
+    )
+    # Planning is read-only, including unsupported pins. All persistence stays below this return.
+    if args.dry_run:
+        print(
+            json.dumps(
+                {
+                    "pins": len(plan),
+                    "runnable_pins": sum(any(u["supported"] for u in p["units"]) for p in plan),
+                    "units": sum(len(p["units"]) for p in plan),
+                    "estimated_seconds": sum(p["estimated_seconds"] for p in plan),
+                    "estimate_basis": "inventory medians: excerpt 31/47s, repository 182/498s; not a deadline guarantee",
+                    "plan": plan,
+                },
+                indent=2,
+            )
+        )
+        return 0
+    args.out.mkdir(parents=True, exist_ok=True)
+    lock = (args.out / ".lock").open("w")
+    try:
+        fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+    except BlockingIOError:
+        parser.error("another trace runner holds this results root")
+    failures = False
+    if args.executor not in {"ax", "mixed-ax"}:
+        args.docker_image = args.image
+    if args.executor in {"docker", "mixed"} and any(u["supported"] for pin in plan for u in pin["units"]):
+        image = subprocess.run(
+            ["docker", "image", "inspect", "--format", "{{.Id}}", args.image], capture_output=True, text=True, check=True
+        )
+        args.docker_image = image.stdout.strip()
+        if not args.docker_image:
+            raise ValueError("Docker returned no local image identity")
+    if args.executor in {"k8s", "k8s-jobs", "mixed"} and any(u["supported"] for pin in plan for u in pin["units"]):
+        args.kubernetes = Kubernetes(args)
+        args.kubernetes.namespace()
+    if args.executor == "queue":
+        args.dispatcher = Dispatcher(args)
+    if args.executor in {"ax", "mixed-ax"}:
+        from engine_trace_ax import AX
+
+        args.ax_dispatcher = AX(args)
+    with tempfile.TemporaryDirectory(prefix="trace-source-", dir=args.out) as scratch:
+        source = Path(scratch) / "frozen"
+        provenance = {"image": args.image, "executor": "queue"} if args.executor == "queue" else freeze_source(source)
+        failures = run_plan(plan, source, provenance, inputs, args)
+    return 2 if failures else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/learn/engine_trace_ax.py

@@ -0,0 +1,108 @@
+"""Engine workload adapter for the shared AX transport."""
+
+from __future__ import annotations
+
+import io
+import json
+import shutil
+import subprocess
+import sys
+import tarfile
+import tempfile
+import time
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
+from engine_trace_k8s import pack
+
+from benchmarks.ax.batch import AXLane, Item, SandboxFailure, Workload, manifest
+from benchmarks.ax.batch import task_state as task_state
+from openultrasast.plane.memory import open_store as open_store
+
+
+def task_manifest(name, args, urls):
+    return manifest(
+        name, engine_workload(args, lambda *_: None), engine_item({}, "pin", b"", args), urls, args.atespace, validate_name=False
+    )
+
+
+def engine_workload(args, validate):
+    return Workload(args.image, frozenset({"SOURCE_URL", "QUESTIONS_URL", "RESULT_URL"}), args.deadline, validate)
+
+
+def engine_item(pin, pin_id, source, args):
+    return Item(
+        pin_id,
+        {"SOURCE_URL": source, "QUESTIONS_URL": json.dumps(pin).encode()},
+        "result.dat",
+        ("python3", "/app/benchmarks/learn/engine_task_entry.py"),
+        {"DEADLINE": str(args.deadline), "QUESTION_DEADLINE": str(args.question_deadline)},
+        "",
+        {"SOURCE_URL": "source.dat", "QUESTIONS_URL": "questions.json"},
+    )
+
+
+class AX(AXLane):
+    def __init__(self, args, store=None, pause=time.sleep, clock=time.monotonic):
+        super().__init__(args, engine_workload(args, lambda *_: None), store, pause, clock)
+
+    def receive(self, data, output, pin):
+        # Validate in isolation: malformed results must never become usable checkpoints.
+        with tempfile.TemporaryDirectory(prefix="engine-ax-result-") as temporary:
+            root = Path(temporary)
+            with tarfile.open(fileobj=io.BytesIO(data)) as archive:
+                archive.extractall(root, filter="data")
+            record = json.loads((root / "result.json").read_text())
+            units = record.get("units", [])
+            if record.get("done") and record.get("status") == "failed" and not units:
+                # Questions download can fail before the actor learns any unit identities.
+                # Keep that explicit failure, filling identities only from the host input.
+                record = {
+                    **pin,
+                    **record,
+                    "units": [
+                        {
+                            **unit,
+                            "status": "failed" if unit["supported"] else "unsupported",
+                            "reason": record.get("reason", "AX task entry failed"),
+                            "instrument": {"files": [], "bytes": 0, "cpg_path": None, "cpg_bytes": 0, "jvm": []},
+                            "questions": {},
+                            "questions_asked": [],
+                            "questions_completed": [],
+                            "witness_rows": [],
+                            "traces": [],
+                        }
+                        for unit in pin["units"]
+                    ],
+                }
+                units = record["units"]
+                (root / "result.json").write_text(json.dumps(record))
+            expected = [u["unit"] for u in pin["units"]]
+            actual = [u["unit"] for u in units]
+            if not record.get("done") or sorted(actual) != sorted(expected):
+                raise ValueError("AX Task returned incomplete or mismatched units")
+            output.mkdir(parents=True, exist_ok=True)
+            # Keep the artifacts without interpreting any remote paths as host destinations.
+            shutil.copytree(root, output, dirs_exist_ok=True)
+
+    def execute(self, checkout, output, pin, pin_id):
+        import copy
+
+        self = copy.copy(self)
+        self.workload = engine_workload(self.args, lambda data, target: self.receive(data, target, pin))
+        attempts = []
+        try:
+            for attempt in range(2):
+                try:
+                    return self._execute_attempt(engine_item(pin, pin_id, pack(checkout), self.args), output, attempt, attempts)
+                except SandboxFailure:
+                    if attempt == 0:
+                        self.pause(10)
+            return subprocess.CompletedProcess([], 1, "", "AX sandbox failed twice"), None
+        finally:
+            (output / "ax_attempts.json").write_text(json.dumps(attempts))
+            result_path = output / "result.json"
+            if result_path.exists():
+                record = json.loads(result_path.read_text())
+                record["ax_attempts"] = attempts
+                result_path.write_text(json.dumps(record))

benchmarks/learn/engine_trace_k8s.py

@@ -0,0 +1,259 @@
+"""Host Kubernetes transport; the embedded bootstrap uses only the standard library."""
+
+from __future__ import annotations
+
+import hashlib
+import io
+import json
+import re
+import subprocess
+import tarfile
+import time
+import uuid
+from datetime import timedelta
+
+from openultrasast.plane.memory import PRESIGNED_ARCHIVE_SUFFIX, S3Store, open_store
+
+IMAGE = "ghcr.io/norandom/openultrasast:2.0.1"
+BOOTSTRAP = r"""
+import io, json, os, pathlib, subprocess, sys, tarfile, urllib.request
+
+def get(name):
+    with urllib.request.urlopen(os.environ[name], timeout=120) as response:
+        return response.read()
+
+def put(data):
+    request = urllib.request.Request(os.environ['RESULT_URL'], data=data, method='PUT')
+    with urllib.request.urlopen(request, timeout=120) as response:
+        response.read()
+
+try:
+    if os.environ.get('CHECK') == '1':
+        data = get('SOURCE_URL')
+        assert len(data) == 1024
+        subprocess.run([sys.executable, '-c', 'print(42)'], check=True)
+        put(data)
+    else:
+        for name, target in [('SOURCE_URL', '/case'), ('ANALYZER_URL', '/frozen')]:
+            pathlib.Path(target).mkdir(parents=True, exist_ok=True)
+            with tarfile.open(fileobj=io.BytesIO(get(name))) as archive:
+                archive.extractall(target, filter='data')
+        out = pathlib.Path('/out')
+        out.mkdir(exist_ok=True)
+        (out / 'pin.json').write_text(os.environ['PIN'])
+        env = dict(os.environ, PYTHONPATH='/frozen/src')
+        for key in ['SOURCE_URL', 'ANALYZER_URL', 'RESULT_URL']:
+            env.pop(key, None)
+        with (out / 'container.log').open('w') as log:
+            done = subprocess.run([sys.executable, '/frozen/benchmarks/learn/engine_trace_worker.py',
+                '--worker', '/out/pin.json', '--deadline', os.environ['DEADLINE'],
+                '--question-deadline', os.environ['QUESTION_DEADLINE']], env=env, stdout=log, stderr=log)
+        (out / 'exit.json').write_text(json.dumps(done.returncode))
+        buffer = io.BytesIO()
+        with tarfile.open(fileobj=buffer, mode='w') as archive:
+            for path in out.iterdir():
+                archive.add(path, arcname=path.name)
+        put(buffer.getvalue())
+        sys.exit(done.returncode)
+except Exception as exc:
+    # urllib exceptions can contain bearer URLs. Never print exception text.
+    print('engine bootstrap failed: ' + type(exc).__name__, file=sys.stderr)
+    sys.exit(1)
+"""
+
+
+def pack(path):
+    buffer = io.BytesIO()
+    with tarfile.open(fileobj=buffer, mode="w") as archive:
+        for child in sorted(path.iterdir()):
+            archive.add(child, arcname=child.name)
+    return buffer.getvalue()
+
+
+def manifests(name, run, args, urls, pin, check=False):
+    labels = {"engine-trace-run": run, "engine-trace-job": name, "engine-trace": "true"}
+    secret = {
+        "apiVersion": "v1",
+        "kind": "Secret",
+        "metadata": {"name": name, "namespace": args.kube_namespace, "labels": labels},
+        "type": "Opaque",
+        "stringData": urls,
+    }
+    env = [{"name": key, "valueFrom": {"secretKeyRef": {"name": name, "key": key}}} for key in urls]
+    env += [
+        {"name": key, "value": value}
+        for key, value in {
+            "PIN": json.dumps(pin),
+            "DEADLINE": str(args.deadline),
+            "QUESTION_DEADLINE": str(args.question_deadline),
+            "CHECK": "1" if check else "0",
+        }.items()
+    ]
+    job = {
+        "apiVersion": "batch/v1",
+        "kind": "Job",
+        "metadata": {"name": name, "namespace": args.kube_namespace, "labels": labels},
+        "spec": {
+            "activeDeadlineSeconds": args.deadline,
+            "backoffLimit": 0,
+            "ttlSecondsAfterFinished": 86400,
+            "template": {
+                "metadata": {"labels": labels},
+                "spec": {
+                    "restartPolicy": "Never",
+                    "automountServiceAccountToken": False,
+                    "securityContext": {"runAsUser": 1000, "runAsGroup": 1000, "fsGroup": 1000},
+                    "volumes": [{"name": name, "emptyDir": {}} for name in ("case", "frozen", "out")],
+                    "affinity": {
+                        "podAntiAffinity": {
+                            "requiredDuringSchedulingIgnoredDuringExecution": [
+                                {
+                                    "labelSelector": {"matchLabels": {"engine-trace": "true"}},
+                                    "topologyKey": "kubernetes.io/hostname",
+                                }
+                            ]
+                        }
+                    },
+                    "containers": [
+                        {
+                            "name": "engine",
+                            "image": args.image,
+                            "command": ["python", "-c", BOOTSTRAP],
+                            "env": env,
+                            "volumeMounts": [{"name": name, "mountPath": "/" + name} for name in ("case", "frozen", "out")],
+                            "resources": {"requests": {"memory": "3Gi", "cpu": "1"}, "limits": {"memory": "3Gi"}},
+                        }
+                    ],
+                },
+            },
+        },
+    }
+    return job, secret
+
+
+def redact(text):
+    return re.sub(r"https?://[^\s\"'<>]+", "[redacted URL]", text)
+
+
+class Kubernetes:
+    def __init__(self, args, store=None):
+        self.args = args
+        self.run = uuid.uuid4().hex
+        self.store = store if store is not None else open_store()
+        if not isinstance(self.store, S3Store):
+            raise ValueError("Kubernetes requires OUSAST_MEMORY=s3://bucket[/prefix]")
+
+    def kubectl(self, *args, manifest=None, check=True):
+        result = subprocess.run(
+            ["kubectl", "--context", self.args.kube_context, "--namespace", self.args.kube_namespace, *args],
+            input=json.dumps(manifest) if manifest is not None else None,
+            capture_output=True,
+            text=True,
+            timeout=30,
+        )
+        if check and result.returncode:
+            # kubectl can echo submitted Secret data on errors; never expose stderr.
+            raise ValueError(f"kubectl {args[0]} failed (exit {result.returncode})")
+        return result
+
+    def namespace(self):
+        self.kubectl("get", "nodes", "-o", "json")
+        self.kubectl("apply", "-f", "-", manifest={"apiVersion": "v1", "kind": "Namespace", "metadata": {"name": self.args.kube_namespace}})
+
+    def execute(self, source, checkout, output, pin, check=False):
+        name = "engine-" + uuid.uuid4().hex[:24]
+        pin_key = hashlib.sha256(json.dumps([pin.get("repo"), pin.get("pin")]).encode()).hexdigest()
+        prefix = f"engine-jobs/{self.run}/{pin_key}/"
+        keys = [prefix + name + PRESIGNED_ARCHIVE_SUFFIX for name in ("source", "analyzer", "result")]
+        expiry = timedelta(seconds=self.args.deadline + 900)
+        node = None
+        try:
+            self.store._put(keys[0], b"x" * 1024 if check else pack(checkout))
+            self.store._put(keys[1], b"" if check else pack(source))
+            urls = {
+                "SOURCE_URL": self.store.presign_get(keys[0], expiry),
+                "ANALYZER_URL": self.store.presign_get(keys[1], expiry),
+                "RESULT_URL": self.store.presign_put(keys[2], expiry),
+            }
+            job, secret = manifests(name, self.run, self.args, urls, pin, check)
+            # create avoids kubectl's last-applied annotation duplicating Secret values.
+            created = json.loads(self.kubectl("create", "-f", "-", "-o", "json", manifest=job).stdout)
+            secret["metadata"]["ownerReferences"] = [
+                {
+                    "apiVersion": "batch/v1",
+                    "kind": "Job",
+                    "name": name,
+                    "uid": created["metadata"]["uid"],
+                }
+            ]
+            self.kubectl("create", "-f", "-", manifest=secret)
+            end = time.monotonic() + self.args.deadline + 60
+            reason = "host deadline waiting for Job"
+            while time.monotonic() < end:
+                state = json.loads(self.kubectl("get", "job", name, "-o", "json").stdout).get("status", {})
+                pods = json.loads(self.kubectl("get", "pods", "-l", f"engine-trace-job={name}", "-o", "json").stdout)
+                node = next((p.get("spec", {}).get("nodeName") for p in pods.get("items", []) if p.get("spec", {}).get("nodeName")), node)
+                result = self.store._get(keys[2])
+                failed = next((c for c in state.get("conditions", []) if c.get("type") == "Failed" and c.get("status") == "True"), None)
+                if failed or state.get("failed"):
+                    reason = (failed or {}).get("reason", "Job failed") + ": " + (failed or {}).get("message", "")
+                    break
+                if state.get("succeeded") and result is not None:
+                    if check:
+                        if result[0] != b"x" * 1024:
+                            raise ValueError("round-trip bytes mismatch")
+                    else:
+                        with tarfile.open(fileobj=io.BytesIO(result[0])) as archive:
+                            archive.extractall(output, filter="data")
+                        if not (output / "result.json").is_file():
+                            raise ValueError("Job produced no result.json")
+                        record = json.loads((output / "result.json").read_text())
+                        if not record.get("done") or {u["unit"] for u in record["units"]} != {u["unit"] for u in pin["units"]}:
+                            raise ValueError("Job returned incomplete or mismatched units")
+                    return subprocess.CompletedProcess([], 0, "", ""), node
+                time.sleep(2)
+            logs = self.kubectl("logs", f"job/{name}", "--tail=40", check=False)
+            reason = redact(reason + "\n" + logs.stdout + logs.stderr)
+            return subprocess.CompletedProcess([], 1, "", reason), node
+        except Exception as exc:
+            reason = "Kubernetes transport failed: " + type(exc).__name__ + ": " + redact(str(exc))
+            try:
+                logs = self.kubectl("logs", f"job/{name}", "--tail=40", check=False)
+                reason += "\n" + redact(logs.stdout + logs.stderr)
+            except (OSError, ValueError, subprocess.SubprocessError):
+                pass
+            return subprocess.CompletedProcess([], 1, "", reason), node
+        finally:
+            if not self.args.keep_jobs:
+                # Attempt every cleanup even if an earlier deletion fails.
+                errors = []
+                for kind in ("job", "secret"):
+                    try:
+                        self.kubectl("delete", kind, name, "--ignore-not-found", "--wait=true", "--timeout=30s")
+                    except Exception as exc:
+                        errors.append(type(exc).__name__)
+                for key in keys:
+                    try:
+                        self.store._delete(key)
+                    except Exception as exc:
+                        errors.append(type(exc).__name__)
+                if errors:
+                    raise RuntimeError("engine Job cleanup failed: " + ", ".join(errors))
+
+    def check(self):
+        self.namespace()
+        nodes = json.loads(self.kubectl("get", "nodes", "-o", "json").stdout)
+        for node in nodes["items"]:
+            print(json.dumps({"node": node["metadata"]["name"], "allocatable_memory": node["status"]["allocatable"]["memory"]}))
+        done, node = self.execute(None, None, None, {}, check=True)
+        print(
+            json.dumps(
+                {
+                    "k8s_check": "passed" if done.returncode == 0 else "failed",
+                    "node": node,
+                    "reason": done.stderr,
+                    "round_trip_bytes": 1024 if done.returncode == 0 else 0,
+                }
+            )
+        )
+        return done.returncode

benchmarks/learn/engine_trace_parse.py

@@ -0,0 +1,51 @@
+"""Lossless parsing of the optional shortest-path trace, with explicit evidence limits."""
+
+from __future__ import annotations
+
+import re
+from collections.abc import Mapping
+from typing import Any
+
+_STEP = re.compile(r"^(.*):(-?\d+) \[([^\]]+)\] ([\s\S]*)$")
+
+
+def parse_trace(row: Mapping[str, Any]) -> dict[str, Any]:
+    """Preserve row-level discharge evidence; never guess which node sanitized a path.
+
+    The query emits basename-only files and truncated code. A bound can be off-path;
+    it remains in ``guard`` even when no trace step exactly matches it. Invalid steps
+    raise instead of turning an unreadable trace into an empty path.
+    """
+    raw = row.get("trace", [])
+    if not isinstance(raw, list):
+        raise ValueError("trace must be an array")
+    steps = []
+    bound = str(row.get("bound") or "")
+    for index, text in enumerate(raw):
+        match = _STEP.fullmatch(text) if isinstance(text, str) else None
+        if match is None:
+            raise ValueError(f"invalid trace step: {text!r}")
+        location, line, node_label, code = match.groups()
+        file, separator, method = location.partition(":")
+        roles = ["source"] if index == 0 else ["step"]
+        if index == len(raw) - 1:
+            roles = ["source", "sink"] if index == 0 else ["sink"]
+        if bound and code == bound:
+            roles.append("guard")
+        steps.append(
+            {
+                "file": file,
+                "method": method if separator else "?",
+                "line": int(line),
+                "node_label": node_label,
+                "code": code,
+                "roles": roles,
+                "raw": text,
+            }
+        )
+    return {
+        "steps": steps,
+        "sanitized": bool(row.get("sanitized", False)),
+        "sanitizer": {"on_path": bool(row.get("sanitized", False)), "step": None},
+        "guard": {"text": bound, "bounded": bool(row.get("bounded", False))},
+    }

benchmarks/learn/engine_trace_queue.py

@@ -0,0 +1,52 @@
+"""Host queue transport: source and task metadata only, never analyzer code."""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import time
+import uuid
+
+from engine_worker_service import PENDING, client_from_env, pack, put, read, result_key, status
+
+from openultrasast.plane.memory import PRESIGNED_ARCHIVE_SUFFIX
+
+
+class Dispatcher:
+    def __init__(self, args, client=None, *, pause=time.sleep, clock=time.monotonic):
+        self.client = client if client is not None else client_from_env()
+        self.run = uuid.uuid4().hex
+        self.args, self.pause, self.clock = args, pause, clock
+
+    def execute(self, checkout, output, pin, pin_id):
+        task_id = f"{self.run}-{pin_id}"
+        task = dict(
+            run=self.run,
+            id=pin_id,
+            task_id=task_id,
+            pin=pin,
+            input=f"engine-queue/inputs/{self.run}/{pin_id}/source{PRESIGNED_ARCHIVE_SUFFIX}",
+            image=self.args.image,
+            deadline=self.args.deadline,
+            question_deadline=self.args.question_deadline,
+        )
+        try:
+            self.client.put(task["input"], pack(checkout), {})
+            put(self.client, PENDING + task_id, task)
+            end = self.clock() + self.args.queue_timeout
+            while self.clock() < end:
+                result = read(self.client, result_key(task))
+                if result is not None:
+                    (output / "result.json").write_text(json.dumps(result))
+                    self.client.delete(task["input"])
+                    return subprocess.CompletedProcess([], 0, "", ""), result.get("worker")
+                self.pause(2)
+            # Removing pending cancels work that has not been claimed; retain source for a live lease.
+            self.client.delete(PENDING + task_id)
+            return None, None
+        except Exception as exc:
+            # SDK errors can contain endpoint URLs or credentials. Do not persist their text.
+            raise OSError("queue transport failed") from exc
+
+    def status(self):
+        return status(self.client)

benchmarks/learn/engine_trace_worker.py

@@ -0,0 +1,644 @@
+"""Container-only Joern trace worker; host prepares all corpus and question metadata.
+
+Keep this module independent of learn/plane and their optional dependencies. The
+frozen host invokes this file directly in the shipped, offline analyzer image.
+"""
+
+from __future__ import annotations
+
+import argparse
+import contextlib
+import hashlib
+import json
+import os
+import selectors
+import signal
+import subprocess
+import tempfile
+import time
+from collections import Counter
+from pathlib import Path
+
+from engine_trace_parse import parse_trace
+
+from openultrasast.cpg.backend import DATAFLOW_OVERLAY, JoernBackend, _render, extract_payload
+from openultrasast.cpg.session import EngineSession
+from openultrasast.model.contracts import ExecutionBudget
+
+
+def write_json(path, value):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(".tmp")
+    tmp.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n")
+    tmp.replace(path)
+
+
+def safe_path(root, name):
+    path = (root / name).resolve()
+    if not path.is_relative_to(root.resolve()):
+        raise ValueError(f"path escapes input root: {name}")
+    return path
+
+
+FRONTENDS = {"php2cpg", "pysrc2cpg", "jssrc2cpg", "javasrc2cpg", "c2cpg", "joern-parse"}
+MIN_CPG_BYTES = 1024
+HEAP_MB = 2560
+CLUSTER_JVM_FLAGS = (
+    "-Xms1700m -Xmx1700m -XX:MaxMetaspaceSize=256m -XX:ReservedCodeCacheSize=128m "
+    "-XX:MaxDirectMemorySize=256m -Xss512k -XX:ActiveProcessorCount=2 -XX:+UseG1GC"
+)
+ENGINE_BUG = "engine bug: jssrc2cpg ObjectPropertyCallLinker"
+EXCLUDED_DIRS = frozenset({"tests", "test", "docs", "node_modules", "vendor", "dist", "build", "templates", "migrations"})
+
+
+def instrument_failure(proof):
+    if any(stage.get("oom") for stage in proof.get("jvm", [])):
+        return "JVM out of memory"
+    if not proof.get("files") or not proof.get("bytes"):
+        return "instrument read zero files or zero bytes"
+    if proof.get("cpg_bytes", 0) <= MIN_CPG_BYTES:
+        return "frontend wrote no CPG"
+    for stage in proof.get("jvm", []):
+        if stage["command"] in FRONTENDS:
+            if stage.get("cpg_bytes", 0) <= MIN_CPG_BYTES:
+                return "frontend wrote no CPG"
+        elif (
+            stage["command"] == "joern" or (stage["command"] == "joern-session" and stage.get("step") == "start" and stage.get("success"))
+        ) and stage["seconds"] < 5:
+            return f"implausibly fast JVM step: {stage['command']} in {stage['seconds']:.3f}s (<5s)"
+    if not proof.get("jvm"):
+        return "no JVM step recorded"
+    return ""
+
+
+class MeasuredBackend(JoernBackend):
+    """Use the shipped backend with process and resident-session watchdogs."""
+
+    def __init__(self, deadline, question_deadline, checkpoint, heap_profile="vm"):
+        if heap_profile not in {"vm", "cluster"}:
+            raise ValueError(f"unknown heap profile: {heap_profile}")
+        self.heap_profile = heap_profile
+        super().__init__(
+            build_timeout=deadline,
+            query_timeout=deadline,
+            session_transport=False,
+            heap_mb=1700 if heap_profile == "cluster" else HEAP_MB,
+        )
+        self.deadline = time.monotonic() + deadline
+        self.question_deadline = question_deadline
+        self.checkpoint = checkpoint
+        self.stages = []
+        self.asked = []
+        self.answers = {}
+        self.completion_counts = Counter()
+        self.timed_out = False
+        self.fatal = ""
+        self.labelled_files = []
+        self.retry_pending = False
+        self.retries = []
+        self.session_fallback = ""
+
+    def _jvm_env(self):
+        env = super()._jvm_env()
+        if self.heap_profile == "cluster":
+            # Bound every JVM, including the child spawned by the Joern launcher.
+            env["JAVA_TOOL_OPTIONS"] = CLUSTER_JVM_FLAGS
+            env.pop("_JAVA_OPTIONS", None)
+            env.pop("JDK_JAVA_OPTIONS", None)
+        return env
+
+    def _session_stage(self, name, action):
+        started = time.monotonic()
+        result = None
+        try:
+            result = action()
+            return result
+        finally:
+            self.stages.append(
+                {
+                    "command": "joern-session",
+                    "step": name,
+                    "success": result is not None and result is not False,
+                    "seconds": time.monotonic() - started,
+                    "heap_mb": self._heap_mb(),
+                    "java_tool_options": self._jvm_env()["JAVA_TOOL_OPTIONS"],
+                }
+            )
+            self.checkpoint()
+
+    def _fallback_session(self, reason):
+        self.session_fallback = reason
+        if self._session is not None:
+            detail = self._session.log_tail()
+            if "OutOfMemoryError" in detail:
+                self.fatal = "JVM out of memory"
+                self.stages[-1]["oom"] = True
+            self.stages[-1]["output_tail"] = detail
+        self.close_session()
+        self.checkpoint()
+
+    def _session_for(self, cpg_path):
+        if self.session_fallback or self.fatal or self.timed_out:
+            return None
+        try:
+            if self._session is None:
+                self._session = EngineSession(
+                    cpg_path.parent / "engine-session",
+                    ExecutionBudget(self.deadline, 5),
+                    self._jvm_env(),
+                    startup_allowance_seconds=min(180, max(20, (self.deadline - time.monotonic()) / 3)),
+                )
+                if not self._session_stage("start", self._session.start):
+                    self._fallback_session(self._session.failure or "session_start_failed")
+                    return None
+            if self._session.loaded_graph != str(cpg_path):
+                if not self._session_stage("load", lambda: self._session.load(cpg_path, timeout=self.query_timeout)):
+                    self._fallback_session(self._session.failure or "session_load_failed")
+                    return None
+                self._defined.clear()
+            return self._session
+        except (OSError, RuntimeError) as exc:
+            self._fallback_session(f"session_start_or_load:{exc}")
+            return None
+
+    def _session_payload(self, cpg_path, query, params):
+        session = self._session_for(cpg_path)
+        if session is None:
+            return None
+        end = min(self.deadline, time.monotonic() + self.query_timeout)
+        question_end = end
+        prior_asked = list(self.asked)
+        prior_answers = dict(self.answers)
+        prior_counts = self.completion_counts.copy()
+        consumed = 0
+        streamed = ""
+
+        def observe(output):
+            nonlocal consumed, question_end, streamed
+            streamed = output
+            lines = output[consumed:].splitlines(keepends=True)
+            for line in lines:
+                if not line.endswith("\n"):
+                    break
+                consumed += len(line)
+                try:
+                    event = json.loads(line)
+                except ValueError:
+                    continue
+                if not isinstance(event, dict):
+                    continue
+                if "__question__" in event:
+                    self.asked.append(event["__question__"])
+                    question_end = min(end, time.monotonic() + self.question_deadline)
+                if "id" in event and "rows" in event:
+                    self.answers[event["id"]] = event["rows"]
+                    self.completion_counts[event["id"]] += 1
+                    question_end = end
+                self.checkpoint()
+            if time.monotonic() >= min(end, question_end):
+                self.timed_out = True
+                return False
+            return True
+
+        def run():
+            source = (self.queries_dir / f"{query}.sc").read_text()
+            # Same shipped script/parameters as JoernBackend, using the graph loaded above.
+            if source.count("importCpg(cpgFile)") != 1:
+                return None
+            symbol = f"ousast_{query}"
+            if query not in self._defined:
+                source = source.replace("importCpg(cpgFile)", "()", 1)
+                source = source.replace("@main def exec(", f"def {symbol}(", 1)
+                if session.define(source, timeout=max(0, end - time.monotonic())) is None:
+                    return None
+                self._defined.add(query)
+            arguments = ", ".join(f"{name} = {json.dumps(_render(value))}" for name, value in sorted(params.items()))
+            session.output_observer = observe
+            try:
+                answer = session.evaluate(f"{symbol}({arguments})", timeout=max(0, end - time.monotonic()))
+            finally:
+                session.output_observer = None
+            if answer is not None:
+                observe(answer.stdout)
+                return answer.stdout
+            return None
+
+        failure = ""
+        try:
+            body = self._session_stage(query, run)
+        except (OSError, RuntimeError) as exc:
+            body = None
+            failure = f"session_{query}:{exc}"
+        if self.timed_out:
+            self.close_session()
+            return streamed
+        parsed = extract_payload(body) if body is not None else None
+        if not isinstance(parsed, dict):
+            # Discard markers from an unverified attempt before replaying the script.
+            self.asked[:] = prior_asked
+            self.answers = prior_answers
+            self.completion_counts = prior_counts
+            self._fallback_session(failure or session.failure or f"session_{query}_payload_missing")
+            return None
+        if query == "census":
+            self.stages[-1]["files"] = int(parsed.get("files", 0))
+            if self.stages[-1]["files"] <= 0:
+                self.fatal = "query census read zero files"
+        return body
+
+    def _apply_overlays(self, cpg_path, scratch):
+        session = self._session_for(cpg_path)
+        result = False
+        if session is not None:
+            body = self._session_payload(cpg_path, "overlay", {"cpgFile": str(cpg_path)})
+            saved = session.scratch / "workspace" / cpg_path.name / "cpg.bin"
+            if body is not None and saved.is_file():
+                import shutil
+
+                shutil.copyfile(saved, cpg_path)
+                result = True
+            elif body is not None:
+                self._fallback_session("session_overlay_saved_graph_missing")
+        if not result:
+            result = super()._apply_overlays(cpg_path, scratch)
+        # An optional overlay failure must not prevent the taint-only attempt.
+        if self.fatal == ENGINE_BUG and not self.timed_out:
+            self.retry_pending = True
+            self.fatal = ""
+        return result
+
+    def _batch_once(self, cpg_path, query, requests):
+        if self.fatal or self.timed_out:
+            return None
+        if not self.retry_pending:
+            answers = super()._batch_once(cpg_path, query, requests)
+            if self.fatal != ENGINE_BUG:
+                return answers
+        if query != "taint" or self.timed_out or self.retries:
+            self.fatal = ENGINE_BUG
+            return None
+        self.retry_pending = False
+        self.fatal = ""
+        retry = {"reason": ENGINE_BUG, "mode": "taint-only", "completed": False}
+        self.retries.append(retry)
+        # Do not reuse rows/markers from the failed attempt as completion evidence.
+        self.asked.clear()
+        self.answers.clear()
+        self.completion_counts.clear()
+        original = self.queries_dir
+        with tempfile.TemporaryDirectory(prefix="trace-taint-", dir=cpg_path.parent) as scratch:
+            script = (original / "taint.sc").read_text()
+            marker = "importCpg(cpgFile)"
+            if script.count(marker) != 1:
+                self.fatal = ENGINE_BUG
+                return None
+            # Loading without enhancement skips the failing JS postprocessing passes.
+            # A raw graph without dataflow cannot establish a negative answer.
+            script = script.replace(
+                marker,
+                "importCpg(cpgFile, enhance = false)\n"
+                f'  require(cpg.metaData.overlays.l.contains("{DATAFLOW_OVERLAY}"), '
+                '"taint-only retry requires saved dataflowOss overlay")',
+            )
+            self.queries_dir = Path(scratch)
+            (self.queries_dir / "taint.sc").write_text(script)
+            try:
+                answers = super()._batch_once(cpg_path, query, requests)
+            finally:
+                self.queries_dir = original
+        retry["completed"] = answers is not None and not self.fatal and not self.timed_out
+        if not retry["completed"]:
+            self.fatal = ENGINE_BUG
+        return answers
+
+    def _run(self, command, *, timeout, cwd=None):
+        if self.fatal or self.timed_out:
+            return None
+        if time.monotonic() >= self.deadline:
+            self.timed_out = True
+            return None
+        started = time.monotonic()
+        end = min(self.deadline, started + timeout)
+        question_end = end
+        binary = Path(command[0]).name
+        frontend = binary in FRONTENDS
+        query = binary == "joern" and "--script" in command
+        output = bytearray()
+        pending = b""
+        process = subprocess.Popen(
+            command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=cwd, env=self._jvm_env(), start_new_session=True
+        )
+        try:
+            with selectors.DefaultSelector() as selector:
+                selector.register(process.stdout, selectors.EVENT_READ)
+                while selector.get_map():
+                    if time.monotonic() >= min(end, question_end):
+                        self.timed_out = True
+                        break
+                    for key, _ in selector.select(min(0.2, max(0, min(end, question_end) - time.monotonic()))):
+                        chunk = os.read(key.fileobj.fileno(), 65536)
+                        if not chunk:
+                            selector.unregister(key.fileobj)
+                            continue
+                        output.extend(chunk)
+                        pending += chunk
+                        while b"\n" in pending:
+                            line, pending = pending.split(b"\n", 1)
+                            try:
+                                event = json.loads(line)
+                            except ValueError:
+                                continue
+                            if not isinstance(event, dict):
+                                continue
+                            if "__question__" in event:
+                                self.asked.append(event["__question__"])
+                                question_end = min(end, time.monotonic() + self.question_deadline)
+                            if "id" in event and "rows" in event:
+                                self.answers[event["id"]] = event["rows"]
+                                self.completion_counts[event["id"]] += 1
+                                question_end = end
+                            self.checkpoint()
+            if not self.timed_out:
+                process.wait(timeout=max(0.01, end - time.monotonic()))
+        finally:
+            with contextlib.suppress(ProcessLookupError):
+                os.killpg(process.pid, signal.SIGKILL)
+            process.wait(timeout=5)
+            process.stdout.close()
+        elapsed = time.monotonic() - started
+        if frontend or query:
+            stage = {
+                "command": binary,
+                "argv": command,
+                "heap_mb": self._heap_mb(),
+                "java_tool_options": self._jvm_env()["JAVA_TOOL_OPTIONS"],
+                "seconds": elapsed,
+                "exit": process.returncode,
+                "oom": b"OutOfMemoryError" in output,
+            }
+            lines = output.decode(errors="replace").splitlines()
+            stage["labelled_file_logs"] = {
+                file: [line for line in lines if file in line or Path(file).name in line]
+                for file in self.labelled_files
+                if any(file in line or Path(file).name in line for line in lines)
+            }
+            self.stages.append(stage)
+            if query and process.returncode != 0:
+                # stderr is merged into stdout above; drop JVM frames before taking
+                # the tail so they cannot crowd out the exception's message.
+                lines = [
+                    line.strip()
+                    for line in output.decode(errors="replace").splitlines()
+                    if line.strip() and not line.strip().startswith(("at ", "... "))
+                ]
+                stage["output_tail"] = "\n".join(lines[-20:])
+                if not self.timed_out:
+                    self.fatal = f"joern query exited {process.returncode}"
+            if frontend:
+                # Capture the exact frontend output before build failure cleanup or overlays.
+                output_arg = next((i + 1 for i, arg in enumerate(command[:-1]) if arg in {"-o", "--output"}), None)
+                cpg_path = (Path(cwd or Path.cwd()) / command[output_arg]).resolve() if output_arg is not None else None
+                stage["cpg_path"] = str(cpg_path) if cpg_path is not None else None
+                stage["cpg_bytes"] = cpg_path.stat().st_size if cpg_path is not None and cpg_path.is_file() else 0
+                if stage["cpg_bytes"] <= MIN_CPG_BYTES:
+                    self.fatal = "frontend wrote no CPG"
+            elif elapsed < 5 and not self.fatal:
+                self.fatal = f"implausibly fast JVM step: {command[0]} in {elapsed:.3f}s (<5s)"
+        if b"ObjectPropertyCallLinker failed" in output and b"Assignment statement with 3 arguments" in output:
+            self.fatal = ENGINE_BUG
+            stage["engine_bug"] = ENGINE_BUG
+        if b"OutOfMemoryError" in output:
+            self.fatal = "JVM out of memory"
+        self.checkpoint()
+        return subprocess.CompletedProcess(command, process.returncode if not self.fatal else 1, output.decode(errors="replace"), "")
+
+
+def unit_record(unit, status, reason="", **extra):
+    return {
+        **unit,
+        "status": status,
+        "reason": reason,
+        "witness_rows": [],
+        "traces": [],
+        "instrument": {"files": [], "bytes": 0, "cpg_path": None, "cpg_bytes": 0, "jvm": []},
+        "questions": {},
+        "questions_asked": [],
+        "questions_completed": [],
+        **extra,
+    }
+
+
+def worker_language(pin, root, out, deadline, question_deadline, persist=write_json, heap_profile="vm"):
+    proof = {
+        "files": [],
+        "bytes": 0,
+        "cpg_path": None,
+        "cpg_bytes": 0,
+        "jvm": [],
+        "heap_mb": 1700 if heap_profile == "cluster" else HEAP_MB,
+        "heap_profile": heap_profile,
+        "java_tool_options": CLUSTER_JVM_FLAGS
+        if heap_profile == "cluster"
+        else f"{os.environ.get('JAVA_TOOL_OPTIONS', '').strip()} -Xmx{HEAP_MB}m".strip(),
+        "excludes": sorted(EXCLUDED_DIRS),
+        "materialization": pin.get("materialization", {}),
+    }
+    record = {
+        **pin,
+        "done": False,
+        "instrument": proof,
+        "units": [unit_record(u, "unsupported" if not u["supported"] else "timeout", u["reason"]) for u in pin["units"]],
+    }
+
+    def checkpoint():
+        # Checkpoints retain raw completed rows even if the outer container deadline fires.
+        # They remain timeout until the finished JVM's instrument proof is validated.
+        for stage in backend.stages:
+            if "cpg_path" in stage:
+                proof["cpg_path"] = stage["cpg_path"]
+                proof["cpg_bytes"] = stage["cpg_bytes"]
+        if getattr(backend, "session_fallback", ""):
+            proof["session_fallback"] = backend.session_fallback
+        for row in record["units"]:
+            rid = row["unit"]
+            row["instrument"] = proof
+            row["questions_asked"] = [rid] if rid in backend.asked else []
+            if rid in backend.answers:
+                row["questions_completed"] = [rid]
+                row["witness_rows"] = [r for r in backend.answers[rid] if isinstance(r, dict) and "kind" not in r and "sink" in r]
+        persist(out, record)
+
+    backend = MeasuredBackend(deadline, question_deadline, checkpoint, **({"heap_profile": heap_profile} if heap_profile != "vm" else {}))
+    proof["jvm"] = backend.stages
+    proof["retries"] = getattr(backend, "retries", [])
+    backend.labelled_files = sorted({u["file"] for u in pin["units"] if u["supported"]})
+    cpg = None
+    try:
+        active = [u for u in pin["units"] if u["supported"]]
+        for file in sorted({u["file"] for u in active}):
+            data = safe_path(root, file).read_bytes()
+            proof["files"].append({"file": file, "bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()})
+            proof["bytes"] += len(data)
+            print(f"read {file}: {len(data)} bytes", flush=True)
+        if not proof["bytes"]:
+            raise ValueError("instrument read zero files or zero bytes")
+        languages = {"javascript" if u["language"] == "typescript" else u["language"] for u in active}
+        if len(languages) != 1:
+            raise ValueError(f"one pin has multiple frontend languages: {sorted(languages)}")
+        language = next(iter(languages))
+        requests = {rid: {key: _render(value) for key, value in params.items()} for rid, params in pin["questions"].items()}
+        if set(requests) != {u["unit"] for u in active}:
+            raise ValueError("prepared questions do not match supported units")
+        record["questions"] = requests
+        for unit_row in record["units"]:
+            unit_row["questions"] = {unit_row["unit"]: requests[unit_row["unit"]]} if unit_row["unit"] in requests else {}
+        persist(out, record)
+        cpg = backend.build(root, language=language)
+        if cpg is None:
+            raise ValueError(backend.fatal or backend.last_failure or ("pin deadline" if backend.timed_out else "CPG build failed"))
+        if proof["cpg_path"] is None:
+            proof["cpg_path"] = str(cpg.cpg_path)
+            proof["cpg_bytes"] = cpg.cpg_path.stat().st_size if cpg.cpg_path.is_file() else 0
+        failure = instrument_failure(proof)
+        if failure:
+            raise ValueError(failure)
+        if cpg.run_batch_once is None:
+            raise ValueError("backend offers no batch operation")
+        answers = cpg.run_batch_once("taint", requests) or {}
+        record["census"] = answers.get("__census__", [])
+        failure = backend.fatal or instrument_failure(proof)
+        if not backend.asked and not backend.timed_out:
+            failure = failure or "query produced no markers"
+        census = record["census"]
+        if census and int(census[0].get("files", 0)) <= 0:
+            failure = failure or "query census read zero files"
+        elif not census and not backend.timed_out:
+            failure = failure or "query census is missing"
+        for index, unit in enumerate(pin["units"]):
+            if not unit["supported"]:
+                continue
+            rid = unit["unit"]
+            extra = {
+                "questions": {rid: requests[rid]},
+                "questions_asked": [rid] if rid in backend.asked else [],
+                "questions_completed": [rid] if rid in answers else [],
+                "instrument": proof,
+            }
+            rows = answers.get(rid)
+            # A batch across backend shards is complete only when every shard answered.
+            # The legacy merger can return a union even when one shard was killed.
+            shards = int(census[0].get("shards", 1)) if census else 1
+            shard_incomplete = shards > 1 and getattr(backend, "completion_counts", {}).get(rid, 0) < shards
+            if rows is not None:
+                raw_flows = [r for r in rows if isinstance(r, dict) and "kind" not in r and "sink" in r]
+                extra.update(witness_rows=raw_flows, traces=[parse_trace(r) for r in raw_flows])
+            if shard_incomplete:
+                extra["questions_completed"] = []
+            names = census[0].get("file_names", []) if census else []
+            file_missing = not any(str(p).removeprefix("./") == unit["file"] or str(p).endswith("/" + unit["file"]) for p in names)
+            dropped = file_missing and rows is not None
+            if failure or dropped:
+                status, reason = "failed", failure or "labelled file was dropped or absent from query census"
+                if dropped:
+                    logs = [line for stage in backend.stages for line in stage.get("labelled_file_logs", {}).get(unit["file"], [])]
+                    reason += "\n" + ("\n".join(dict.fromkeys(logs)) if logs else "frontend emitted no log lines mentioning labelled file")
+            elif rows is None or shard_incomplete:
+                status, reason = ("timeout" if backend.timed_out else "failed"), "question did not complete"
+            elif not rows and rid not in backend.asked:
+                status, reason = "failed", "query produced no markers"
+            else:
+                flows = [r for r in rows if isinstance(r, dict) and "kind" not in r and "sink" in r]
+                if any(not row.get("trace") for row in flows):
+                    raise ValueError("taint witness returned without requested trace")
+                extra.update(witness_rows=flows, traces=[parse_trace(r) for r in flows])
+                status, reason = ("path" if flows else "asked-nothing"), ""
+            record["units"][index] = unit_record(unit, status, reason, **extra)
+    except (OSError, ValueError, RuntimeError) as exc:
+        for index, unit in enumerate(pin["units"]):
+            if unit["supported"]:
+                record["units"][index] = unit_record(
+                    unit,
+                    "timeout" if backend.timed_out and not backend.fatal else "failed",
+                    str(exc),
+                    instrument=proof,
+                    questions={unit["unit"]: record["questions"][unit["unit"]]} if unit["unit"] in record.get("questions", {}) else {},
+                    witness_rows=[r for r in backend.answers.get(unit["unit"], []) if isinstance(r, dict) and "sink" in r],
+                    questions_completed=[unit["unit"]] if unit["unit"] in backend.answers else [],
+                    questions_asked=[unit["unit"]] if unit["unit"] in backend.asked else [],
+                )
+    finally:
+        if hasattr(backend, "close_session"):
+            backend.close_session()
+        if cpg and cpg.cleanup:
+            cpg.cleanup()
+        for stage in backend.stages:
+            if stage["command"] == "joern" and stage.get("exit") and "output_tail" in stage:
+                detail = f"joern query exited {stage['exit']}: {stage['output_tail'] or 'no output'}"
+                for row in record["units"]:
+                    if row["supported"] and row["status"] in {"failed", "timeout"}:
+                        row["reason"] = f"{row['reason']}\n{detail}".strip()
+        record["questions_asked"] = list(dict.fromkeys(backend.asked))
+        record["questions_completed"] = [rid for u in record["units"] for rid in u["questions_completed"]]
+        record["done"] = True
+        persist(out, record)
+    return record
+
+
+def worker(pin, root, out, deadline, question_deadline, heap_profile="vm"):
+    """Run language partitions sequentially under one shared container deadline."""
+    languages = sorted({"javascript" if u["language"] == "typescript" else u["language"] for u in pin["units"] if u["supported"]})
+    if len(languages) <= 1:
+        return worker_language(pin, root, out, deadline, question_deadline, heap_profile=heap_profile)
+    end = time.monotonic() + deadline
+    record = {
+        **pin,
+        "done": False,
+        "units": [
+            unit_record(u, "timeout" if u["supported"] else "unsupported", "pin deadline" if u["supported"] else u["reason"])
+            for u in pin["units"]
+        ],
+        "instruments": {},
+    }
+
+    def persist(_, partial):
+        updates = {u["unit"]: u for u in partial["units"]}
+        record["units"] = [updates.get(u["unit"], u) for u in record["units"]]
+        record["instruments"][language] = partial["instrument"]
+        record["questions_asked"] = [rid for u in record["units"] for rid in u["questions_asked"]]
+        record["questions_completed"] = [rid for u in record["units"] for rid in u["questions_completed"]]
+        write_json(out, record)
+
+    write_json(out, record)
+    for language in languages:
+        remaining = end - time.monotonic()
+        if remaining <= 0:
+            break
+        units = [
+            u for u in pin["units"] if u["supported"] and ("javascript" if u["language"] == "typescript" else u["language"]) == language
+        ]
+        questions = {u["unit"]: pin["questions"][u["unit"]] for u in units}
+        worker_language(
+            {**pin, "units": units, "questions": questions}, root, out, remaining, question_deadline, persist, heap_profile=heap_profile
+        )
+    record["done"] = True
+    write_json(out, record)
+    return record
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--worker", type=Path, required=True)
+    parser.add_argument("--deadline", type=int, default=900)
+    parser.add_argument("--question-deadline", type=int, default=120)
+    parser.add_argument("--heap-profile", choices=("cluster", "vm"), default="vm")
+    parser.add_argument("--root", type=Path, default=Path("/case"))
+    parser.add_argument("--output", type=Path, default=Path("/out/result.json"))
+    args = parser.parse_args()
+    if min(args.deadline, args.question_deadline) <= 0:
+        parser.error("deadlines must be positive")
+    worker(json.loads(args.worker.read_text()), args.root, args.output, args.deadline, args.question_deadline, args.heap_profile)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/learn/engine_worker_service.py

@@ -0,0 +1,315 @@
+"""Image-resident queue workers. Requires strongly consistent S3 GET/PUT/LIST.
+
+Per-task Lamport bakery tickets use only ordinary object operations. A choosing
+marker precedes ticket selection; contenders wait for choosing peers and lower
+(ticket, worker) pairs. Late arrivals cannot displace a running owner. Leases
+assume bounded clock skew (less than MARGIN / 2) and a worker killed by its deadline.
+"""
+
+from __future__ import annotations
+
+import argparse
+import contextlib
+import ctypes
+import io
+import json
+import logging
+import multiprocessing
+import os
+import signal
+import socket
+import tarfile
+import tempfile
+import threading
+import time
+import uuid
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+
+from engine_trace_worker import unit_record, worker
+
+from openultrasast.plane.memory import S3Client, s3_settings
+
+PENDING = "engine-queue/pending/"
+CLAIMED = "engine-queue/claimed/"
+RESULTS = "engine-results/"
+MARGIN = 60
+LOG = logging.getLogger(__name__)
+
+
+def client_from_env():
+    settings = s3_settings(os.environ)
+    bucket = os.environ.get("S3_BUCKET")
+    if not bucket:
+        raise ValueError("S3_BUCKET is required")
+    return S3Client(bucket=bucket, **settings)
+
+
+def read(client, key):
+    value = client.get(key)
+    return json.loads(value[0]) if value else None
+
+
+def put(client, key, value):
+    client.put(key, json.dumps(value).encode(), {})
+
+
+def result_key(task):
+    return f"{RESULTS}{task['run']}/{task['id']}.json"
+
+
+def pack(root):
+    stream = io.BytesIO()
+    with tarfile.open(fileobj=stream, mode="w") as archive:
+        archive.add(root, arcname=".")
+    return stream.getvalue()
+
+
+def unpack(data, root):
+    with tarfile.open(fileobj=io.BytesIO(data)) as archive:
+        # Source snapshots need only regular files and directories, never links/devices.
+        for member in archive.getmembers():
+            target = (root / member.name).resolve()
+            if not target.is_relative_to(root.resolve()) or not (member.isfile() or member.isdir()):
+                raise ValueError("unsafe source archive")
+        archive.extractall(root, filter="data")
+
+
+class Queue:
+    def __init__(self, client, identity=None, *, now=time.time, pause=time.sleep, stop=None):
+        self.client = client
+        self.identity = identity or f"{socket.gethostname()}-{uuid.uuid4().hex}"
+        self.now, self.pause = now, pause
+        self.stop = stop or threading.Event()
+
+    def markers(self, task_id=None):
+        result = []
+        for key in self.client.keys(CLAIMED):
+            if task_id is None or key.rsplit("/", 1)[-1] == task_id:
+                marker = read(self.client, key)
+                if marker:
+                    result.append((key, marker))
+        return result
+
+    def tasks(self):
+        tasks = {}
+        for key in self.client.keys(PENDING):
+            task = read(self.client, key)
+            if task:
+                tasks[task["task_id"]] = task
+        # A dead worker may have removed pending already. Its task remains in the lease.
+        for _, marker in self.markers():
+            if marker["expires"] < self.now():
+                task = marker["task"]
+                tasks.setdefault(task["task_id"], task)
+        return list(tasks.values())
+
+    def claim(self, task):
+        if read(self.client, result_key(task)) is not None or self.stop.is_set():
+            return None
+        key = f"{CLAIMED}{self.identity}/{task['task_id']}"
+        marker = dict(task=task, ticket=0, expires=self.now() + task["deadline"] + MARGIN)
+        put(self.client, key, marker)
+        try:
+            live = [m for _, m in self.markers(task["task_id"]) if m["expires"] > self.now()]
+            marker["ticket"] = 1 + max((m["ticket"] for m in live), default=0)
+            put(self.client, key, marker)
+            while not self.stop.is_set():
+                # Do not start once the execution budget no longer fits within our lease.
+                if self.now() >= marker["expires"] - task["deadline"] - MARGIN / 2:
+                    break
+                peers = [(k, m) for k, m in self.markers(task["task_id"]) if k != key and m["expires"] > self.now()]
+                if any(m["ticket"] == 0 for _, m in peers):
+                    self.pause(0.1)
+                    continue
+                if any((m["ticket"], k) < (marker["ticket"], key) for k, m in peers):
+                    break  # loser backs off and can try another pending task
+                if read(self.client, key) != marker or read(self.client, result_key(task)) is not None:
+                    break
+                self.client.delete(PENDING + task["task_id"])
+                return key
+        except Exception:
+            self.client.delete(key)
+            raise
+        self.client.delete(key)
+        return None
+
+    def process_one(self, execute=None):
+        for task in self.tasks():
+            key = self.claim(task)
+            if key is None:
+                continue
+            try:
+                lease = read(self.client, key)
+                if lease is None:
+                    continue
+                budget = min(task["deadline"], lease["expires"] - self.now() - MARGIN / 2)
+                if budget <= 0:
+                    continue  # Leave the expired lease's task available to recovery.
+                result = (execute or execute_task)(self.client, {**task, "runtime_budget": budget})
+                result.update(executor="queue", worker=self.identity, pod=socket.gethostname(), image=os.environ.get("ENGINE_IMAGE", ""))
+                put(self.client, result_key(task), result)
+                # Publish the result before removing either the input or the claim.
+                self.client.delete(task["input"])
+                for old_key, old in self.markers(task["task_id"]):
+                    if old_key == key or old["expires"] < self.now():
+                        self.client.delete(old_key)
+                return True
+            except Exception:
+                # Keep the lease and task for recovery, without logging credential-bearing errors.
+                LOG.error("task failed; lease retained for recovery")
+                return False
+        return False
+
+
+def _child(pin, root, out, deadline, question_deadline):
+    os.setsid()
+    signal.signal(signal.SIGTERM, signal.SIG_DFL)
+    worker(pin, root, out, deadline, question_deadline)
+
+
+def adopt_orphans():
+    # Linux PR_SET_CHILD_SUBREAPER: escaped Joern sessions reparent here, not
+    # outside the service. There is exactly one task process tree per worker.
+    if ctypes.CDLL(None, use_errno=True).prctl(36, 1, 0, 0, 0) != 0:
+        raise OSError("cannot enable task descendant cleanup")
+
+
+def kill_children():
+    """Freeze parents before walking children, including new-session JVMs."""
+    seen = set()
+
+    def freeze(parent):
+        for entry in Path("/proc").iterdir():
+            if not entry.name.isdigit() or int(entry.name) in seen:
+                continue
+            try:
+                # comm can contain spaces or parentheses. Fields after it begin at state.
+                fields = (entry / "stat").read_text().rsplit(")", 1)[1].split()
+                if int(fields[1]) != parent:
+                    continue
+                pid = int(entry.name)
+                os.kill(pid, signal.SIGSTOP)
+                seen.add(pid)
+                freeze(pid)
+            except (FileNotFoundError, ProcessLookupError, PermissionError):
+                continue
+
+    # A child that exited during discovery can reparent descendants to this
+    # subreaper. Repeat until every remaining descendant is frozen.
+    while True:
+        count = len(seen)
+        freeze(os.getpid())
+        if len(seen) == count:
+            break
+    for pid in seen:
+        with contextlib.suppress(ProcessLookupError):
+            os.kill(pid, signal.SIGKILL)
+
+
+def execute_task(client, task):
+    adopt_orphans()
+    pin = task["pin"]
+    started = time.monotonic()
+    budget = task.get("runtime_budget", task["deadline"])
+    status, reason, code = "failed", "worker produced no result", None
+    with tempfile.TemporaryDirectory(prefix="engine-task-") as scratch:
+        root, out = Path(scratch) / "case", Path(scratch) / "result.json"
+        root.mkdir()
+        try:
+            if task["deadline"] > 900 or task["deadline"] <= 0:
+                raise ValueError("deadline exceeds deployment grace")
+            if task.get("image") != os.environ.get("ENGINE_IMAGE"):
+                raise ValueError("task image differs from deployed image")
+            source = client.get(task["input"])
+            if source is None:
+                raise ValueError("missing source")
+            unpack(source[0], root)
+            # The shipped worker also checks frontend input and graph output for every language.
+            files = [p for p in root.rglob("*") if p.is_file()]
+            if not files:
+                raise ValueError("empty source")
+            LOG.info("input proof: first file bytes=%d", len(files[0].read_bytes()))
+            if time.monotonic() - started >= budget:
+                status, reason = "timeout", "task input deadline"
+                raise TimeoutError(reason)
+            process = multiprocessing.get_context("fork").Process(target=_child, args=(pin, root, out, budget, task["question_deadline"]))
+            process.start()
+            try:
+                process.join(max(0, budget - (time.monotonic() - started)))
+                if process.is_alive():
+                    status, reason = "timeout", "task deadline"
+                else:
+                    reason = f"worker exit {process.exitcode}"
+                code = process.exitcode
+            finally:
+                # Joern creates separate sessions. Kill the whole descendant tree,
+                # not merely the Python wrapper's process group.
+                kill_children()
+                process.join()
+                # Reap adopted JVMs and their launchers after the direct child.
+                while True:
+                    try:
+                        os.waitpid(-1, 0)
+                    except ChildProcessError:
+                        break
+            record = json.loads(out.read_text()) if out.exists() else {**pin, "units": []}
+        except Exception:
+            record = {**pin, "units": []}
+            reason = "worker input or execution failed"
+        if not record.get("done") or code != 0:
+            previous = {u["unit"]: u for u in record["units"]}
+            record["units"] = [
+                previous[u["unit"]]
+                if code is None and previous.get(u["unit"], {}).get("status") in {"path", "asked-nothing"}
+                else unit_record(u, status if u["supported"] else "unsupported", reason if u["supported"] else u["reason"])
+                for u in pin["units"]
+            ]
+        record.update(done=True, container_exit=code, seconds=time.monotonic() - started)
+        return record
+
+
+def status(client):
+    return {
+        "pending": len(client.keys(PENDING)),
+        "claimed": len(client.keys(CLAIMED)),
+        "done": len([k for k in client.keys(RESULTS) if k.endswith(".json")]),
+    }
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--port", type=int, default=8080)
+    args = parser.parse_args()
+    logging.basicConfig(level=logging.INFO)
+    stop = threading.Event()
+    signal.signal(signal.SIGTERM, lambda *_: stop.set())
+    signal.signal(signal.SIGINT, lambda *_: stop.set())
+    client = client_from_env()
+    queue = Queue(client, stop=stop)
+
+    class Health(BaseHTTPRequestHandler):
+        def do_GET(self):
+            self.send_response(200 if self.path == "/healthz" and not stop.is_set() else 503)
+            self.end_headers()
+
+        def log_message(self, *_):
+            pass
+
+    server = ThreadingHTTPServer(("0.0.0.0", args.port), Health)
+    threading.Thread(target=server.serve_forever, daemon=True).start()
+    try:
+        while not stop.is_set():
+            try:
+                if not queue.process_one():
+                    stop.wait(2)
+            except Exception:
+                LOG.error("queue unavailable; retrying")
+                stop.wait(5)
+    finally:
+        server.shutdown()
+        server.server_close()
+
+
+if __name__ == "__main__":
+    main()

benchmarks/learn/joern_leak_audit.py

@@ -0,0 +1,239 @@
+"""Offline audit of completed Joern results; never launches an analyzer or reads corpora.
+
+Missing results stay in coverage, not signal denominators. Observed non-path rows
+score zero for path features. Sink kinds are one-hot indicators, never ordinal
+category IDs. Both signed and best-direction metrics expose reversed shortcuts.
+"""
+
+from __future__ import annotations
+
+import argparse
+import ast
+import hashlib
+import json
+import re
+from collections import Counter
+from pathlib import Path
+
+from slice_leak_audit import THRESHOLD, complete_pairs, metrics
+
+ROOT = Path(__file__).resolve().parents[2]
+UNITS = ROOT / "plane/experiments/exp-005-graph-slice.units.jsonl"
+RESULTS = Path.home() / "ousast-results/plane/engine-trace"
+OUTPUT = ROOT / "benchmarks/measurements/2026-10-04-joern-slice-leak-audit/record.json"
+STATUSES = ("path", "asked-nothing", "failed", "timeout", "unsupported", "missing")
+ASKED = {"path", "asked-nothing"}
+SIGNALS = ("has_path", "shortest_path_steps", "sanitized", "guard_present", "asked")
+ASYMMETRY = ("vulnerable_only", "fixed_only", "both", "neither", "not_both_asked")
+
+
+def materialization_version():
+    # Read the producer's literal without importing its execution dependencies.
+    tree = ast.parse(Path(__file__).with_name("engine_trace.py").read_bytes())
+    for node in tree.body:
+        if isinstance(node, ast.Assign) and any(isinstance(t, ast.Name) and t.id == "MATERIALIZATION_VERSION" for t in node.targets):
+            return ast.literal_eval(node.value)
+    raise ValueError("Producer has no materialization version")
+
+
+def features(row):
+    path = row["status"] == "path"
+    traces = row.get("traces", []) if path else []
+    candidates = [(i, t) for i, t in enumerate(traces) if t.get("steps")]
+    index, trace = min(candidates, key=lambda it: len(it[1]["steps"])) if candidates else (None, {})
+    witnesses = row.get("witness_rows", [])
+    witness = witnesses[index] if index is not None and index < len(witnesses) else {}
+    guard = trace.get("guard", {})
+    sink = witness.get("sinkKind")
+    if not sink:
+        # Retain only the call target: arguments, literals and code are not kinds.
+        match = re.search(r"([\w:$>.\\-]+)\s*\(", str(witness.get("sink", "")))
+        sink = match[1] if match else "unknown"
+    return {
+        "has_path": int(path),
+        "shortest_path_steps": len(trace["steps"]) if trace else None if path else 0,
+        "sanitized": int(bool(trace.get("sanitized", witness.get("sanitized", False)))) if trace else None if path else 0,
+        "guard_present": int(
+            bool(guard.get("text") or guard.get("bounded") or witness.get("bound") or witness.get("bounded"))
+            or any("guard" in s.get("roles", []) for s in trace.get("steps", []))
+        )
+        if trace
+        else None
+        if path
+        else 0,
+        "sink_kind": str(sink) if path else "no-path",
+        "asked": int(row["status"] in ASKED),
+    }
+
+
+def scored(rows, signal):
+    eligible = [r for r in rows if r[signal] is not None]
+    result = metrics(eligible, signal)
+    result["excluded_rows"] = len(rows) - len(eligible)
+    result["flagged"] = (result["pairs"] >= 10 and (result["best_direction_within_pair_accuracy"] or 0) >= THRESHOLD) or (
+        result["rows"] >= 10 and (result["best_direction_pooled_auc"] or 0) >= THRESHOLD
+    )
+    return result
+
+
+def signal_metrics(rows, kinds):
+    return {
+        **{s: scored(rows, s) for s in SIGNALS},
+        "sink_kind": {kind: scored([{**r, "is_kind": int(r["sink_kind"] == kind)} for r in rows], "is_kind") for kind in kinds},
+    }
+
+
+def scope_report(rows, kinds):
+    pairs = complete_pairs(rows)
+    asymmetry = Counter()
+    honest = []
+    for v, f in pairs:
+        if not (v["asked"] and f["asked"]):
+            asymmetry["not_both_asked"] += 1
+        else:
+            honest.extend((v, f))
+            asymmetry[
+                "both"
+                if v["has_path"] and f["has_path"]
+                else "vulnerable_only"
+                if v["has_path"]
+                else "fixed_only"
+                if f["has_path"]
+                else "neither"
+            ] += 1
+    observed = [r for r in rows if r["status"] != "missing"]
+    return {
+        "rows": len(rows),
+        "coverage": {s: sum(r["status"] == s for r in rows) for s in STATUSES},
+        "complete_pairs": len(pairs),
+        "pair_asymmetry": {s: asymmetry[s] for s in ASYMMETRY},
+        "both_asked_pairs": len(honest) // 2,
+        "signals": signal_metrics(observed, kinds),
+        "both_asked_signals": signal_metrics(honest, kinds),
+    }
+
+
+def audit(units, results, *, expected_rows=501, expected_pairs=195):
+    if len(units) != expected_rows or len({u["unit"] for u in units}) != expected_rows:
+        raise ValueError(f"Expected {expected_rows} unique units")
+    if len(complete_pairs(units)) != expected_pairs:
+        raise ValueError(f"Expected {expected_pairs} complete pairs")
+    if not results.is_dir():
+        raise ValueError(f"Results directory is unreadable: {results}")
+    version = materialization_version()
+    unit_index = {u["unit"]: u for u in units}
+    matched = {}
+    files = []
+    counts = Counter()
+    for path in sorted(results.glob("*.json")):
+        raw = path.read_bytes()
+        record = json.loads(raw)  # Corruption fails loudly; producer uses atomic replace.
+        if not isinstance(record, dict) or "repo" not in record or "pin" not in record:
+            continue
+        stale = record.get("materialization_version") != version
+        unfinished = record.get("done") is not True
+        counts["stale"] += stale
+        counts["unfinished"] += unfinished
+        counts["skipped"] += stale or unfinished
+        files.append((path.name, len(raw)))
+        if stale or unfinished:
+            continue
+        counts["accepted_files"] += 1
+        counts["result_rows_read"] += len(record["units"])
+        for row in record["units"]:
+            if row["unit"] not in unit_index:
+                continue
+            unit = unit_index[row["unit"]]
+            if any(row.get(k) != unit.get(k) for k in ("family", "label", "pair")):
+                raise ValueError(f"Result metadata mismatch: {row['unit']}")
+            if row["status"] not in STATUSES[:-1] or not row.get("source"):
+                raise ValueError(f"Invalid result status/source: {row['unit']}")
+            if row["unit"] in matched:
+                raise ValueError(f"Duplicate current result: {row['unit']}")
+            matched[row["unit"]] = {
+                **unit,
+                "source": row["source"],
+                "status": row["status"],
+                "pin_role": row.get("pin_role"),
+                **features(row),
+            }
+    rows = [
+        matched.get(u["unit"], {**u, "source": u.get("source", "unknown"), "status": "missing", **features({"status": "missing"})})
+        for u in units
+    ]
+    kinds = sorted({r["sink_kind"] for r in rows if r["status"] != "missing"})
+    scopes = {"overall": scope_report(rows, kinds)}
+    for field in ("family", "source"):
+        scopes[f"by_{field}"] = {
+            value: scope_report([r for r in rows if r[field] == value], kinds) for value in sorted({r[field] for r in rows})
+        }
+    return {
+        "cost_usd": 0,
+        "materialization_version": version,
+        "threshold": THRESHOLD,
+        "minimum_samples": 10,
+        "unit_rows_read": len(units),
+        "matched_rows": len(matched),
+        "inputs": {
+            **{k: counts[k] for k in ("accepted_files", "result_rows_read", "stale", "unfinished", "skipped")},
+            "files_read": len(files),
+            "bytes_read": sum(size for _, size in files),
+            # Hash a canonical inventory; filenames and paths may identify repositories.
+            "files_sha256": hashlib.sha256(json.dumps(sorted(files), separators=(",", ":")).encode()).hexdigest(),
+        },
+        "semantics": {
+            "signals": "Shortest nonempty trace (first on ties); its sanitizer, guard and corresponding witness sink call target.",
+            "missing": "Coverage only; absent source metadata is unknown. Skipped result units are never consumed.",
+            "non_path": "Observed non-path rows score zero; path rows lacking trace evidence have null trace features.",
+            "asked": "path or asked-nothing=1; failed, timeout, unsupported=0; missing excluded from metrics.",
+            "asymmetry": "Disjoint categories; not_both_asked takes precedence over path asymmetry.",
+            "flags": "Best of both directions, >=0.70 with >=10 pairs for pair accuracy or >=10 rows for AUC.",
+            "sink_kind": "One binary indicator per observed call target, including no-path/unknown; no ordinal encoding.",
+            "limits": "Exploratory associations, not proof of leakage; running input snapshot, no multiple-testing correction.",
+        },
+        **scopes,
+        "units": [{k: r[k] for k in ("unit", "pair", "family", "label", "fold", "source", "status", *SIGNALS, "sink_kind")} for r in rows],
+    }
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--results", type=Path, default=RESULTS)
+    parser.add_argument("--units", type=Path, default=UNITS)
+    parser.add_argument("--output", type=Path, default=OUTPUT)
+    args = parser.parse_args()
+    raw = args.units.read_bytes()
+    units = [json.loads(line) for line in raw.splitlines() if line.strip()]
+    report = audit(units, args.results)
+    report["units_input"] = {"bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest()}
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n")
+    overall, inputs = report["overall"], report["inputs"]
+    lines = [
+        f"Joern leak audit: $0; materialization_version={report['materialization_version']}",
+        f"Units read: {len(units)}; bytes={len(raw)}; file={args.units}",
+        f"Result files read: {inputs['files_read']}; bytes={inputs['bytes_read']}; accepted={inputs['accepted_files']}",
+        f"Skipped: {inputs['skipped']} (stale={inputs['stale']}, unfinished={inputs['unfinished']}; counts overlap)",
+        f"Result rows read: {inputs['result_rows_read']}; matched={report['matched_rows']}",
+        f"Coverage: {overall['coverage']}",
+        f"Pairs: {overall['complete_pairs']}; both asked={overall['both_asked_pairs']}",
+        f"Asymmetry: {overall['pair_asymmetry']}",
+    ]
+    for signal in (*SIGNALS, "sink_kind"):
+
+        def brief(value, signal=signal):
+            if signal == "sink_kind":
+                return {k: brief_metric(m) for k, m in value.items()}
+            return brief_metric(value)
+
+        lines.append(f"{signal}: all={brief(overall['signals'][signal])}; both asked={brief(overall['both_asked_signals'][signal])}")
+    lines.append(f"Family/source metrics and input inventory digest: {args.output}")
+    print("\n".join(lines))
+
+
+def brief_metric(metric):
+    return {k: metric[k] for k in ("rows", "pairs", "within_pair_accuracy", "pooled_auc", "flagged")}
+
+
+if __name__ == "__main__":
+    main()

benchmarks/learn/slice_leak_audit.py

@@ -0,0 +1,177 @@
+"""Read-only, zero-model slice audit. Run: env -u OPENROUTER_API_KEY python benchmarks/learn/slice_leak_audit.py.
+
+Only sources.toml itself is read, never its referenced corpora. Stored source IDs
+are the reporting strata (the store merges advisory pair catalogs into `pairs`).
+Ties get half credit. Both signed and best-direction scores are recorded; like
+learn.leaks, a reversed label shortcut is still a leak. Missing classes/pairs
+produce null metrics, not plausible zeros. Unsupported slices score zero for both
+indicators and remain separately counted in coverage.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+from collections import Counter, defaultdict
+from collections.abc import Mapping, Sequence
+from pathlib import Path
+from typing import Any
+
+from openultrasast.config import load_dotenv
+from openultrasast.learn.examples import load_examples
+from openultrasast.learn.labels import load_sources
+from openultrasast.learn.slice import function_slice
+from openultrasast.plane.memory import MemoryStore, open_store
+
+UNITS = Path("plane/experiments/exp-004-pooled-block-gate.units.jsonl")
+OUTPUT = Path("benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json")
+THRESHOLD = 0.70
+
+
+def complete_pairs(rows: Sequence[Mapping[str, Any]]) -> list[tuple[Mapping[str, Any], Mapping[str, Any]]]:
+    groups: dict[tuple[str, str], list[Mapping[str, Any]]] = defaultdict(list)
+    for row in rows:
+        if row.get("pair"):
+            groups[(row["family"], row["pair"])].append(row)
+    pairs = []
+    for members in groups.values():
+        if len(members) == 2 and {m["label"] for m in members} == {0, 1}:
+            v, f = sorted(members, key=lambda m: -m["label"])
+            if v["group"] != f["group"] or v["fold"] != f["fold"]:
+                raise ValueError("Pair crosses group or fold")
+            pairs.append((v, f))
+        elif len(members) != 1:
+            raise ValueError("Ambiguous pair")
+    return pairs
+
+
+def metrics(rows: Sequence[Mapping[str, Any]], signal: str) -> dict[str, Any]:
+    pairs = complete_pairs(rows)
+    higher = sum(v[signal] > f[signal] for v, f in pairs)
+    lower = sum(v[signal] < f[signal] for v, f in pairs)
+    ties = len(pairs) - higher - lower
+    pair_score = (higher + 0.5 * ties) / len(pairs) if pairs else None
+    positive = [r[signal] for r in rows if r["label"] == 1]
+    negative = [r[signal] for r in rows if r["label"] == 0]
+    auc = (
+        sum((v > f) + 0.5 * (v == f) for v in positive for f in negative) / (len(positive) * len(negative))
+        if positive and negative
+        else None
+    )
+    best_pair = max(pair_score, 1 - pair_score) if pair_score is not None else None
+    best_auc = max(auc, 1 - auc) if auc is not None else None
+    return {
+        "rows": len(rows),
+        "pairs": len(pairs),
+        "positive": len(positive),
+        "negative": len(negative),
+        "higher_on_vulnerable": higher,
+        "higher_on_fixed": lower,
+        "ties": ties,
+        "within_pair_accuracy": pair_score,
+        "pooled_auc": auc,
+        "best_direction_within_pair_accuracy": best_pair,
+        "best_direction_pooled_auc": best_auc,
+        "flagged": any(v is not None and v >= THRESHOLD for v in (best_pair, best_auc)),
+    }
+
+
+def audit(store: MemoryStore, units: Sequence[Mapping[str, Any]], *, expected_rows: int = 501, expected_pairs: int = 195) -> dict[str, Any]:
+    if len(units) != expected_rows or len({u["unit"] for u in units}) != expected_rows:
+        raise ValueError(f"Expected {expected_rows} unique units, read {len(units)}")
+    if len(complete_pairs(units)) != expected_pairs:
+        raise ValueError(f"Expected {expected_pairs} complete pairs")
+    examples = load_examples(store)
+    index = {e.id: e for e in examples}
+    if len(index) != len(examples):
+        raise ValueError("Duplicate example IDs in memory")
+    sources = {s.id for s in load_sources().sources}
+    coverage: dict[str, Counter[str]] = defaultdict(Counter)
+    rows = []
+    excerpt_bytes = 0
+    blobs: dict[str, str] = {}
+    for unit in units:
+        e = index[unit["unit"]]  # missing input is fatal, never counted as no-flow
+        if (e.family, e.label, e.group) != (unit["family"], unit["label"], unit["group"]):
+            raise ValueError(f"Unit metadata differs from store: {e.id}")
+        if e.source not in sources:
+            raise ValueError(f"Source absent from sources.toml: {e.source}")
+        if e.excerpt_sha not in blobs:
+            # MemoryStore.get_blob(verify=True) DELETES corrupt blobs: do not use it.
+            data = store.get_blob("excerpts", e.excerpt_sha, verify=False)
+            if not data or hashlib.sha256(data).hexdigest() != e.excerpt_sha:
+                raise ValueError(f"Missing, empty or corrupt excerpt: {e.id}")
+            excerpt_bytes += len(data)
+            blobs[e.excerpt_sha] = data.decode("utf-8")
+        result = function_slice(blobs[e.excerpt_sha], e.language, e.family)
+        status = "not-analysable" if result is None else "flow" if result.steps else "no-flow"
+        coverage[e.family][status] += 1
+        rows.append(
+            {
+                **unit,
+                "source": e.source,
+                "status": status,
+                "has_flow": int(status == "flow"),
+                "slice_length": len(result.steps) if result else 0,
+            }
+        )
+    pairs = complete_pairs(rows)
+    asymmetry = Counter(
+        "both" if v["has_flow"] and f["has_flow"] else "vulnerable_only" if v["has_flow"] else "fixed_only" if f["has_flow"] else "neither"
+        for v, f in pairs
+    )
+    scopes = {"all": rows, **{f"source:{s}": [r for r in rows if r["source"] == s] for s in sorted({r["source"] for r in rows})}}
+    return {
+        "cost_usd": 0,
+        "threshold": THRESHOLD,
+        "unit_rows_read": len(units),
+        "example_rows_read": len(examples),
+        "matched_rows": len(rows),
+        "excerpts_read": len(blobs),
+        "excerpt_units": len(rows),
+        "excerpt_bytes": excerpt_bytes,
+        "complete_pairs": len(pairs),
+        "coverage": {f: {s: c[s] for s in ("flow", "no-flow", "not-analysable")} for f, c in sorted(coverage.items())},
+        "pair_asymmetry": {k: asymmetry[k] for k in ("vulnerable_only", "fixed_only", "both", "neither")},
+        "signals": {
+            signal: {scope: metrics(members, signal) for scope, members in scopes.items()} for signal in ("has_flow", "slice_length")
+        },
+        "source_strata": "Stored source IDs from sources.toml; advisory pair catalogs are merged into pairs by the example builder.",
+        "limitations": (
+            "Flat IR: no dominance/guards, textual branch order, potential parameter inputs, conservative call returns; no alias analysis."
+        ),
+        "units": [
+            {k: r[k] for k in ("unit", "pair", "family", "label", "fold", "source", "status", "has_flow", "slice_length")} for r in rows
+        ],
+    }
+
+
+def main() -> None:
+    load_dotenv(Path(".env"))
+    os.environ.pop("OPENROUTER_API_KEY", None)  # dotenv must not restore the removed key
+    raw = UNITS.read_bytes()
+    units = [json.loads(line) for line in raw.decode().splitlines() if line.strip()]
+    report = audit(open_store(), units)
+    report["units_bytes"] = len(raw)
+    report["units_sha256"] = hashlib.sha256(raw).hexdigest()
+    OUTPUT.parent.mkdir(parents=True, exist_ok=True)
+    OUTPUT.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n")
+    totals = Counter({s: sum(c[s] for c in report["coverage"].values()) for s in ("flow", "no-flow", "not-analysable")})
+    lines = [
+        f"Slice leak audit: $0; units bytes={len(raw)}",
+        f"Rows read: units={len(units)}, examples={report['example_rows_read']}, matched={report['matched_rows']}",
+        f"Excerpts read: {report['excerpts_read']}; units={report['excerpt_units']}; bytes={report['excerpt_bytes']}",
+        f"Coverage: {dict(totals)}",
+        f"Complete pairs: {report['complete_pairs']}",
+        f"Pair asymmetry: {report['pair_asymmetry']}",
+        f"Has flow: {report['signals']['has_flow']['all']}",
+        f"Slice length: {report['signals']['slice_length']['all']}",
+        f"Source strata: {', '.join(report['signals']['has_flow'])}; threshold={THRESHOLD}",
+        f"Report: {OUTPUT}",
+    ]
+    print("\n".join(lines))
+
+
+if __name__ == "__main__":
+    main()

benchmarks/measurements/2026-09-08-module-audit.json

@@ -1,7 +1,7 @@
 {
-  "modules": 151,
+  "modules": 159,
   "counts": {
-    "load_bearing": 146,
+    "load_bearing": 154,
     "standalone_capability": 5
   },
   "orphaned": [],
@@ -229,6 +229,8 @@
         "model.report",
         "pairs",
         "plane.tasks.alerts",
+        "push.quick",
+        "push.runner",
         "rank",
         "regress.candidate",
         "reports",
@@ -356,12 +358,14 @@
         "learn.excerpt",
         "learn.experiments",
         "learn.features",
+        "learn.fill",
         "learn.folds",
         "learn.labels",
         "learn.leaks",
         "learn.program",
         "learn.retrieve",
         "learn.roles",
+        "learn.slice",
         "plane.harvest",
         "plane.tasks.features",
         "plane.tasks.remember",
@@ -410,7 +414,8 @@
       "importers": [
         "learn.cli",
         "learn.compile",
-        "learn.experiments"
+        "learn.experiments",
+        "learn.fill"
       ]
     },
     {
@@ -422,6 +427,7 @@
         "learn.compile",
         "learn.evaluate",
         "learn.experiments",
+        "learn.fill",
         "learn.folds",
         "learn.program",
         "learn.retrieve"
@@ -435,6 +441,7 @@
         "learn.embeddings",
         "learn.examples",
         "learn.labels",
+        "learn.slice",
         "plane.harvest"
       ]
     },
@@ -443,7 +450,8 @@
       "classification": "load_bearing",
       "reason": "reachable from gate, map_gate, pair_gate, cli",
       "importers": [
-        "learn.cli"
+        "learn.cli",
+        "learn.fill"
       ]
     },
     {
@@ -455,6 +463,14 @@
         "plane.tasks.features"
       ]
     },
+    {
+      "module": "learn.fill",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "learn.cli"
+      ]
+    },
     {
       "module": "learn.folds",
       "classification": "load_bearing",
@@ -500,7 +516,8 @@
         "learn.cli",
         "learn.compile",
         "learn.evaluate",
-        "learn.experiments"
+        "learn.experiments",
+        "learn.fill"
       ]
     },
     {
@@ -539,6 +556,14 @@
         "plane.tasks.roles"
       ]
     },
+    {
+      "module": "learn.slice",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "learn.program"
+      ]
+    },
     {
       "module": "map_gate",
       "classification": "load_bearing",
@@ -560,6 +585,7 @@
         "pairs",
         "plane.tasks.alerts",
         "preprocess",
+        "push.quick",
         "push.runner"
       ]
     },
@@ -579,9 +605,12 @@
         "cli",
         "cpg.backend",
         "cpg.session",
+        "findings",
         "learn.cli",
         "learn.features",
+        "learn.fill",
         "learn.labels",
+        "learn.slice",
         "model.candidates",
         "model.config_value",
         "model.dominance",
@@ -602,6 +631,7 @@
         "push.cache",
         "push.contracts",
         "push.policy",
+        "push.quick",
         "push.report",
         "push.reuse",
         "push.runner",
@@ -649,6 +679,7 @@
         "push.cache",
         "push.contracts",
         "push.policy",
+        "push.quick",
         "push.report",
         "push.reuse",
         "push.runner",
@@ -672,6 +703,7 @@
       "importers": [
         "cli",
         "learn.cli",
+        "learn.fill",
         "plane.budget",
         "plane.generate",
         "push.assistance",
@@ -716,7 +748,9 @@
       "classification": "load_bearing",
       "reason": "reachable from gate, map_gate, pair_gate, cli",
       "importers": [
-        "model.scan"
+        "findings",
+        "model.scan",
+        "push.runner"
       ]
     },
     {
@@ -775,6 +809,7 @@
       "classification": "load_bearing",
       "reason": "model-grounded-detection Req 7.2-7.4: the security vocabularies the CPG queries are parameterised by",
       "importers": [
+        "learn.slice",
         "model.config_value",
         "model.dominance",
         "model.pipeline",
@@ -846,7 +881,10 @@
         "learn.embeddings",
         "learn.examples",
         "learn.experiments",
+        "learn.fill",
         "learn.program",
+        "plane.delivery",
+        "plane.doctor",
         "plane.egress",
         "plane.engine_alerts",
         "plane.generate",
@@ -871,12 +909,21 @@
         "learn.compile",
         "learn.embeddings",
         "learn.experiments",
+        "learn.fill",
         "learn.program",
         "plane.generate",
         "plane.tasks.roles",
         "plane.tasks.verify"
       ]
     },
+    {
+      "module": "plane.delivery",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "plane.reconciler"
+      ]
+    },
     {
       "module": "plane.doctor",
       "classification": "load_bearing",
@@ -924,6 +971,7 @@
       "reason": "ai-service-plane Req 1.1-1.4: ax-shaped Task/Workspace/Model and the Run kind, validated before anything runs",
       "importers": [
         "learn.cli",
+        "plane.delivery",
         "plane.egress",
         "plane.generate",
         "plane.harvest",
@@ -945,12 +993,27 @@
         "learn.embeddings",
         "learn.examples",
         "learn.experiments",
+        "learn.fill",
         "learn.program",
+        "plane.delivery",
+        "plane.doctor",
         "plane.generate",
+        "plane.reconciler",
         "plane.tasks.loop",
         "plane.tasks.remember"
       ]
     },
+    {
+      "module": "plane.profile",
+      "classification": "load_bearing",
+      "reason": "python -m openultrasast.plane.profile",
+      "importers": [
+        "cli",
+        "plane.doctor",
+        "plane.generate",
+        "plane.reconciler"
+      ]
+    },
     {
       "module": "plane.reconciler",
       "classification": "load_bearing",
@@ -1093,6 +1156,7 @@
         "improve.evolve",
         "improve.validator",
         "plane.tasks.loop",
+        "push.quick",
         "scoring.project_score",
         "semantic.overlay"
       ]
@@ -1132,6 +1196,7 @@
         "plane.tasks.alerts",
         "plane.tasks.features",
         "plane.tasks.repo_facts",
+        "push.quick",
         "push.runner",
         "rank",
         "regress.candidate",
@@ -1186,6 +1251,7 @@
       "importers": [
         "cli",
         "push.assistance",
+        "push.background",
         "push.eligibility",
         "push.policy",
         "push.report",
@@ -1203,6 +1269,14 @@
         "push.runner"
       ]
     },
+    {
+      "module": "push.background",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "push.runner"
+      ]
+    },
     {
       "module": "push.cache",
       "classification": "load_bearing",
@@ -1234,6 +1308,23 @@
         "push.runner"
       ]
     },
+    {
+      "module": "push.explain",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "push.report",
+        "push.runner"
+      ]
+    },
+    {
+      "module": "push.install",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "cli"
+      ]
+    },
     {
       "module": "push.policy",
       "classification": "load_bearing",
@@ -1245,12 +1336,21 @@
         "push.vetoes"
       ]
     },
+    {
+      "module": "push.quick",
+      "classification": "load_bearing",
+      "reason": "reachable from gate, map_gate, pair_gate, cli",
+      "importers": [
+        "push.runner"
+      ]
+    },
     {
       "module": "push.report",
       "classification": "load_bearing",
       "reason": "pre-push-safety-net Req 5.1/5.2/5.5/7.3: compact output and complete artifacts; runner integration pending",
       "importers": [
         "push.assistance",
+        "push.background",
         "push.runner"
       ]
     },
@@ -1315,7 +1415,8 @@
         "improve.evolve",
         "map_gate",
         "pairs",
-        "plane.tasks.alerts"
+        "plane.tasks.alerts",
+        "push.quick"
       ]
     },
     {
@@ -1501,6 +1602,7 @@
         "hunter_tools",
         "learn.features",
         "learn.roles",
+        "learn.slice",
         "mapping",
         "model.candidates",
         "model.layout",
@@ -1557,6 +1659,7 @@
         "hunter_tools",
         "learn.features",
         "learn.roles",
+        "learn.slice",
         "mapping",
         "model.candidates",
         "model.layout",
@@ -1590,6 +1693,7 @@
       "importers": [
         "cli",
         "hunter_tools",
+        "learn.slice",
         "model.candidates",
         "pairs",
         "semantic.cst",
@@ -1723,6 +1827,7 @@
       "classification": "load_bearing",
       "reason": "reachable from gate, map_gate, pair_gate, cli",
       "importers": [
+        "learn.slice",
         "semantic.overlay"
       ]
     },
@@ -1748,6 +1853,8 @@
       "reason": "reachable from gate, map_gate, pair_gate, cli",
       "importers": [
         "cli",
+        "learn.cli",
+        "learn.fill",
         "model.endpoint",
         "plane.budget",
         "plane.tasks.roles",

benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-k3s.txt

@@ -0,0 +1,5 @@
+[ok] profile k3s: exec=remote kube_context=ousast-k3s registry=ghcr.io/norandom images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=ghcr-pull
+[FAIL] cluster ousast-k3s: context ousast-k3s unreachable (error: context "ousast-k3s" does not exist); docs/deployment.md lists the prerequisites
+[FAIL] runner image in ghcr.io/norandom: images file /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json: No such file or directory; docs/deployment.md lists the prerequisites
+[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
+exit=1 seconds=1.1

benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-kind.txt

@@ -0,0 +1,7 @@
+[ok] profile kind: exec=local kube_context=kind-ousast registry=localhost:5001 images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)
+[ok] cluster kind-ousast: reachable
+[ok] agent substrate (ate-system): 9 pods ready
+[ok] ax controller (ax-system): 4 pods ready
+[ok] runner image in localhost:5001: ousast-runner@sha256:95edc7ed5621... present in localhost:5001
+[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
+exit=0 seconds=1.4

benchmarks/measurements/2026-10-02-k8s-01-profile/literal-reinserted-fail.txt

@@ -0,0 +1,19 @@
+F..                                                                      [100%]
+=================================== FAILURES ===================================
+________ test_no_host_literal_outside_the_profile_and_the_kind_bring_up ________
+
+    def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None:
+        kind = load_profile("kind", environ={})
+        found = offenders(_files(), kind.registry)
+>       assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found)
+E       AssertionError: host-only literals (Req 1.3):
+E         src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast"  # reinserted for the record 01 demonstration
+E       assert ['src/openult...emonstration'] == []
+E         
+E         Left contains one more item: 'src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast"  # reinserted for the record 01 demonstration'
+E         Use -v to get more diff
+
+tests/test_plane_literals.py:81: AssertionError
+=========================== short test summary info ============================
+FAILED tests/test_plane_literals.py::test_no_host_literal_outside_the_profile_and_the_kind_bring_up
+1 failed, 2 passed in 0.26s

benchmarks/measurements/2026-10-02-k8s-01-profile/literal-restored-pass.txt

@@ -0,0 +1,2 @@
+...                                                                      [100%]
+3 passed in 0.23s

benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-big.yaml

@@ -0,0 +1,16 @@
+apiVersion: ax.io/v1alpha1
+kind: Workspace
+metadata: {name: ousast-probe-big-ws, atespace: default}
+spec:
+  files:
+    - {path: "hello.py", content: "def hello():\n    return greet()\n"}
+---
+apiVersion: ax.io/v1alpha1
+kind: Task
+metadata: {name: ousast-probe-big, atespace: default}
+spec:
+  image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
+  command: ["repo-facts"]
+  env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
+  resources: {requests: {cpu: "250m", memory: "2560Mi"}, limits: {cpu: "1", memory: "3Gi"}}
+  workspaces: [{name: ousast-probe-big-ws, path: "/workspace"}]

benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-small.yaml

@@ -0,0 +1,16 @@
+apiVersion: ax.io/v1alpha1
+kind: Workspace
+metadata: {name: ousast-probe-small-ws, atespace: default}
+spec:
+  files:
+    - {path: "hello.py", content: "def hello():\n    return greet()\n"}
+---
+apiVersion: ax.io/v1alpha1
+kind: Task
+metadata: {name: ousast-probe-small, atespace: default}
+spec:
+  image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
+  command: ["repo-facts"]
+  env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
+  resources: {requests: {cpu: "250m", memory: "256Mi"}, limits: {cpu: "1", memory: "512Mi"}}
+  workspaces: [{name: ousast-probe-small-ws, path: "/workspace"}]

benchmarks/measurements/2026-10-02-k8s-01-profile/probe/probe-pool.yaml

@@ -0,0 +1,26 @@
+# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
+# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
+# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
+# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
+# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
+# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
+apiVersion: ate.dev/v1alpha1
+kind: WorkerPool
+metadata:
+  name: probe-pool
+  namespace: ax-system
+  labels:
+    workload: ousast
+spec:
+  replicas: 1
+  workerImage: localhost:5001/ateom-gvisor-715889664656de67e44382a8d6ab981d@sha256:220f1cf13ea485b73a2e4270fa544901a18ae826d8f22bd54d42761944a41c5c
+  template:
+    nodeSelector:
+      ate.dev/substrate-version: "4f9c293"
+    resources:
+      limits:
+        cpu: "2"
+        memory: 3Gi
+      requests:
+        cpu: 250m
+        memory: 3Gi

benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-before.json

@@ -0,0 +1,78 @@
+{
+  "workers": [
+    {
+      "ip": "10.244.0.22",
+      "labels": {
+        "workload": "ousast"
+      },
+      "metadata": {
+        "createTime": "2026-09-29T16:32:08.594663703Z",
+        "name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
+        "uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
+        "updateTime": "2026-10-02T05:24:59.069304031Z",
+        "version": "1758"
+      },
+      "nodeName": "ousast-control-plane",
+      "sandboxClass": "gvisor",
+      "status": {
+        "capacity": {
+          "actors": 1000,
+          "resources": {
+            "limits": [
+              {
+                "name": "cpu",
+                "quantity": "1"
+              },
+              {
+                "name": "memory",
+                "quantity": "1536Mi"
+              }
+            ]
+          }
+        },
+        "state": "WORKER_STATE_ACTIVE"
+      },
+      "workerNamespace": "ax-system",
+      "workerPod": "ousast-pool-6656b55f5b-9wbrw",
+      "workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
+      "workerPool": "ousast-pool"
+    },
+    {
+      "ip": "10.244.0.28",
+      "labels": {
+        "workload": "ousast"
+      },
+      "metadata": {
+        "createTime": "2026-10-02T18:29:30.486026311Z",
+        "name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
+        "uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
+        "updateTime": "2026-10-02T18:29:30.551668391Z",
+        "version": "2"
+      },
+      "nodeName": "ousast-control-plane",
+      "sandboxClass": "gvisor",
+      "status": {
+        "capacity": {
+          "actors": 1000,
+          "resources": {
+            "limits": [
+              {
+                "name": "cpu",
+                "quantity": "2"
+              },
+              {
+                "name": "memory",
+                "quantity": "3Gi"
+              }
+            ]
+          }
+        },
+        "state": "WORKER_STATE_ACTIVE"
+      },
+      "workerNamespace": "ax-system",
+      "workerPod": "probe-pool-5dffd74bdc-dqlww",
+      "workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
+      "workerPool": "probe-pool"
+    }
+  ]
+}

benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-big.json

@@ -0,0 +1,81 @@
+{
+  "workers": [
+    {
+      "ip": "10.244.0.22",
+      "labels": {
+        "workload": "ousast"
+      },
+      "metadata": {
+        "createTime": "2026-09-29T16:32:08.594663703Z",
+        "name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
+        "uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
+        "updateTime": "2026-10-02T18:29:45.281958347Z",
+        "version": "1761"
+      },
+      "nodeName": "ousast-control-plane",
+      "sandboxClass": "gvisor",
+      "status": {
+        "allocated": {
+          "actors": 1
+        },
+        "capacity": {
+          "actors": 1000,
+          "resources": {
+            "limits": [
+              {
+                "name": "cpu",
+                "quantity": "1"
+              },
+              {
+                "name": "memory",
+                "quantity": "1536Mi"
+              }
+            ]
+          }
+        },
+        "state": "WORKER_STATE_ACTIVE"
+      },
+      "workerNamespace": "ax-system",
+      "workerPod": "ousast-pool-6656b55f5b-9wbrw",
+      "workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
+      "workerPool": "ousast-pool"
+    },
+    {
+      "ip": "10.244.0.28",
+      "labels": {
+        "workload": "ousast"
+      },
+      "metadata": {
+        "createTime": "2026-10-02T18:29:30.486026311Z",
+        "name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
+        "uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
+        "updateTime": "2026-10-02T18:29:30.551668391Z",
+        "version": "2"
+      },
+      "nodeName": "ousast-control-plane",
+      "sandboxClass": "gvisor",
+      "status": {
+        "capacity": {
+          "actors": 1000,
+          "resources": {
+            "limits": [
+              {
+                "name": "cpu",
+                "quantity": "2"
+              },
+              {
+                "name": "memory",
+                "quantity": "3Gi"
+              }
+            ]
+          }
+        },
+        "state": "WORKER_STATE_ACTIVE"
+      },
+      "workerNamespace": "ax-system",
+      "workerPod": "probe-pool-5dffd74bdc-dqlww",
+      "workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
+      "workerPool": "probe-pool"
+    }
+  ]
+}

benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-small.json

@@ -0,0 +1,81 @@
+{
+  "workers": [
+    {
+      "ip": "10.244.0.22",
+      "labels": {
+        "workload": "ousast"
+      },
+      "metadata": {
+        "createTime": "2026-09-29T16:32:08.594663703Z",
+        "name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
+        "uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
+        "updateTime": "2026-10-02T18:29:53.001457323Z",
+        "version": "1763"
+      },
+      "nodeName": "ousast-control-plane",
+      "sandboxClass": "gvisor",
+      "status": {
+        "allocated": {
+          "actors": 1
+        },
+        "capacity": {
+          "actors": 1000,
+          "resources": {
+            "limits": [
+              {
+                "name": "cpu",
+                "quantity": "1"
+              },
+              {
+                "name": "memory",
+                "quantity": "1536Mi"
+              }
+            ]
+          }
+        },
+        "state": "WORKER_STATE_ACTIVE"
+      },
+      "workerNamespace": "ax-system",
+      "workerPod": "ousast-pool-6656b55f5b-9wbrw",
+      "workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
+      "workerPool": "ousast-pool"
+    },
+    {
+      "ip": "10.244.0.28",
+      "labels": {
+        "workload": "ousast"
+      },
+      "metadata": {
+        "createTime": "2026-10-02T18:29:30.486026311Z",
+        "name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
+        "uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
+        "updateTime": "2026-10-02T18:30:04.825058843Z",
+        "version": "4"
+      },
+      "nodeName": "ousast-control-plane",
+      "sandboxClass": "gvisor",
+      "status": {
+        "capacity": {
+          "actors": 1000,
+          "resources": {
+            "limits": [
+              {
+                "name": "cpu",
+                "quantity": "2"
+              },
+              {
+                "name": "memory",
+                "quantity": "3Gi"
+              }
+            ]
+          }
+        },
+        "state": "WORKER_STATE_ACTIVE"
+      },
+      "workerNamespace": "ax-system",
+      "workerPod": "probe-pool-5dffd74bdc-dqlww",
+      "workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
+      "workerPool": "probe-pool"
+    }
+  ]
+}

benchmarks/measurements/2026-10-02-k8s-01-profile/record.json

@@ -0,0 +1,132 @@
+{
+  "record": "k8s-01-profile",
+  "spec": "plane-on-kubernetes",
+  "tasks": ["1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8"],
+  "date": "2026-10-02",
+  "host": "the maintainer's laptop: 7.9 GiB, single-node kind cluster, RustFS store reachable over https",
+  "commits": {
+    "1.1": "8eac1b2",
+    "1.2": "77a03ba",
+    "1.3": "36f5927",
+    "1.4": "8099b29",
+    "1.5": "86ec237",
+    "1.6": "2afa20e",
+    "1.7": "2cca03c",
+    "1.8": "the commit that adds this line (git log: plane(1.8))"
+  },
+  "images": {
+    "runner_before": "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884",
+    "runner_after_up_sh": "localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb",
+    "pin_file": "ops/k8s/profiles/kind-images.json (written by ops/ax/up.sh; the templates re-pinned to it, 128 image lines)"
+  },
+  "instrument_checks": {
+    "code_under_test": "PYTHONPATH=<worktree>/src first; `python -c 'import openultrasast; print(openultrasast.__file__)'` printed the worktree path before every gate",
+    "store_reached": {
+      "how": "scratch store_probe.py: open_store('s3://sast-memory') through the worktree code (verify_bucket runs), then read the probe object",
+      "open_and_verify_seconds": 0.62,
+      "probe_key": "_probe/select.jsonl",
+      "probe_bytes": 24,
+      "probe_version_id_present": true
+    },
+    "smoke_delivery_sizes_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291, "task.yaml": 1148},
+    "registry_resolution": "doctor HEADs /v2/ousast-runner/manifests/<digest> on the kind registry: HTTP 2xx for the new digest"
+  },
+  "doctor": {
+    "kind": {
+      "command": "ousast plane doctor --profile kind",
+      "exit": 0,
+      "seconds": 1.4,
+      "lines": [
+        "[ok] profile kind: exec=local kube_context=... registry=... images=... router_url=(tunnel) ax_server=(tunnel) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)",
+        "[ok] cluster <kind context>: reachable",
+        "[ok] agent substrate (ate-system): 9 pods ready",
+        "[ok] ax controller (ax-system): 4 pods ready",
+        "[ok] runner image in <kind registry>: ousast-runner@sha256:95edc7ed5621... present",
+        "[ok] memory store s3://sast-memory: opens (an S3 bucket is verified)"
+      ],
+      "file": "doctor-kind.txt"
+    },
+    "k3s": {
+      "command": "ousast plane doctor --profile k3s",
+      "exit": 1,
+      "seconds": 1.1,
+      "failing_lines": [
+        "cluster ousast-k3s: context does not exist (expected: the cluster does not exist yet)",
+        "runner image in ghcr.io/norandom: images file ops/k8s/profiles/images.json missing (expected: the release asset is published by task 4.1)"
+      ],
+      "passing_lines": ["profile k3s: every address reported", "memory store s3://sast-memory: opens"],
+      "note": "two expected failures, not one: the images file is the release asset and does not exist before group 4; the namespaces are not checked behind an unreachable context, so no third line fails for the same cause",
+      "file": "doctor-k3s.txt"
+    }
+  },
+  "literal_scan": {
+    "test": "tests/test_plane_literals.py",
+    "scope": "src/, plane/, ops/ (minus ops/ax/up.sh, ops/k8s/profiles/), docs/",
+    "patterns": ["kind context", "kind registry", "kind gateway 172.19.", "loopback with port", "port-forward"],
+    "exemptions": [
+      "router.py for loopback-with-port and port-forward",
+      "lines tagged ax-tunnel or loopback (two: the runner's own health URL, the Joern session's query URL)",
+      "digest-pinned image: lines under plane/tasks/ whose registry equals the kind profile's (a third test asserts every one does)"
+    ],
+    "offenders_at_1_3": 0,
+    "offenders_before_1_3": 23,
+    "reinsertion_demo": {
+      "inserted": "a kind-context literal appended to src/openultrasast/plane/workspaces.py",
+      "pytest_exit_with_literal": 1,
+      "named": "src/openultrasast/plane/workspaces.py:37: [kind context]",
+      "restored": "byte for byte (cmp)",
+      "pytest_exit_restored": 0,
+      "files": ["literal-reinserted-fail.txt", "literal-restored-pass.txt"]
+    }
+  },
+  "up_sh_rerun": {
+    "command": "OUSAST_PYTHON=<main venv python> ops/ax/up.sh",
+    "exit": 0,
+    "seconds": 38,
+    "kept": ["cluster", "substrate", "egress gateway", "ax control plane", "worker pool ousast-pool (2/2 ready)", "ax CLI (newer than the checkout)"],
+    "done": ["receiver Service re-applied", "runner image rebuilt and pushed (new digest above)", "kind-images.json written", "smoke Task applied and resumed"],
+    "first_attempt_failure": "`profile --print images` answered `no image named ''`; fixed in profile._lookup (an images path with no sub-key is the file path) and covered by test_plane_profile",
+    "disk_gb_free_before_after": [1.7, 1.5],
+    "file": "up-sh-rerun.txt"
+  },
+  "smoke_run_1_4": {
+    "command": "ops/ax/smoke-run.sh (receiver delivery: the path group 2 replaces)",
+    "exit": 0,
+    "seconds": 24,
+    "status": "done",
+    "delivered": "facts.json 451 B, summary.json 238 B, units.jsonl 291 B; summary.image = the new digest, units 2/2",
+    "file": "smoke-run-1.4.txt"
+  },
+  "placement_probe_1_5": {
+    "script": "ops/ax/probe-placement.sh",
+    "pools": {"ousast-pool": "scaled to 1 worker, 1 CPU / 1536Mi", "probe-pool": "1 worker, 2 CPU / 3Gi, the already-built ateom-gvisor image (no ko build)"},
+    "big_actor": {"requests_memory": "2560Mi", "limits_memory": "3Gi", "landed_on": "ousast-pool (1536Mi worker): status.allocated.actors=1; probe-pool 0"},
+    "small_actor": {"requests_memory": "256Mi", "limits_memory": "512Mi", "landed_on": "ousast-pool: allocated.actors=1; probe-pool 0"},
+    "placement": "any",
+    "consequence": "design.md section 4: group 5 takes the fallback (an engine atespace with engine-pool as its only pool if Substrate scopes pools by atespace; else a plain Kubernetes Job on k3s and alerts-engine on kind)",
+    "resume_latency_seconds": "first resume of each new template succeeded at once (golden snapshot built within the resume deadline); whole probe 46 s",
+    "cleanup": "probe tasks deleted, probe-pool deleted, ousast-pool back to 2/2 (verified by kubectl get workerpool)",
+    "files": ["probe/probe.log", "probe/workers-before.json", "probe/workers-ousast-probe-big.json", "probe/workers-ousast-probe-small.json", "probe/ousast-probe-big.yaml", "probe/ousast-probe-small.yaml", "probe/probe-pool.yaml"]
+  },
+  "reconciler_bound_1_6": {
+    "lines_at_spec_commit_6bd44ca": 477,
+    "lines_after_1_2": 489,
+    "added_by_1_2": "the profile parameter of run() and its docstring (+12)",
+    "estimate_2_4": "Delivery wiring: open the store, build Delivery, pass the delivery body to the starter, delivered() and collect() in the await/execute path, state mirroring, the OUSAST_DELIVERY=receiver switch: about +25 while the receiver is kept (group 2 keeps both paths), about -20 when 2.6 removes the receiver wiring",
+    "projected_during_group_2": 514,
+    "projected_after_2_6": 494,
+    "decision": "the estimate crosses 500 while both paths coexist: task 2.4 moves the bound in tests/test_plane_reconciler.py to 550 with the reason in the assertion message, and 2.6 records the final count"
+  },
+  "gvisor_tarball_1_7": {
+    "status": "planned, manual",
+    "where": "docs/deployment.md B.1",
+    "url": "gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd (sha256-pinned in substrate/manifests/ate-install/sandboxconfig-gvisor.yaml)",
+    "check": "the atelet's fetch line in `kubectl -n ate-system logs` of a worker pod on the k3s node; labelled planned until read there"
+  },
+  "gates": {
+    "1.1": "pytest 2002 passed + module-audit manifest regenerated (test_module_audit 6 passed), ruff 0, format 0, mypy 0",
+    "1.2": "pytest 2006 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
+    "1.3": "pytest 2009 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0; mkdocs --strict 0",
+    "1.4": "see the commit message"
+  }
+}

benchmarks/measurements/2026-10-02-k8s-01-profile/smoke-run-1.4.txt

@@ -0,0 +1,9 @@
+run finished: done
+memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
+run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
+
+TASK                         MODEL                  CALLS     PROMPT  CACHE-HIT    OUTPUT        USD
+facts                        -                          0        n/a        n/a       n/a        n/a  [done]
+----------------------------------------------------------------------------------------------------
+total                        -                        n/a        n/a        n/a       n/a        n/a
+smoke exit=0 seconds=24

benchmarks/measurements/2026-10-02-k8s-01-profile/up-sh-rerun.txt

@@ -0,0 +1,58 @@
+
+== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (20:35:35)
+cluster exists; kept
+
+== agent substrate (ate-system) (20:35:35)
+substrate ready; kept
+NAME                              READY   STATUS      RESTARTS       AGE
+ate-api-server-647bb6f8b6-c9k5k   1/1     Running     2 (12h ago)    3d
+ate-api-server-647bb6f8b6-dznjn   1/1     Running     1 (12h ago)    3d
+ate-controller-c95586684-g7nvn    1/1     Running     0              3d2h
+atelet-4f9c293-9zpfg              1/1     Running     0              3d2h
+atenet-egress-d74b8f6bf-8zmxj     1/1     Running     0              3d
+atenet-router-54887bc8f9-rqs6g    2/2     Running     2 (12h ago)    3d2h
+postgres-0                        2/2     Running     3 (3d2h ago)   3d2h
+rustfs-6c4c677cb6-cqvhc           1/1     Running     3 (3d2h ago)   3d2h
+rustfs-bucket-init-nvfr6          0/1     Completed   0              3d2h
+
+== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (20:35:35)
+gateway present; kept
+
+== ax control plane (ax-system) (20:35:35)
+ax ready; kept
+NAME                           READY   STATUS    RESTARTS   AGE
+ax-redis-86df786c87-pz5tp      1/1     Running   0          3d2h
+ax-server-b479bdc76-hwkqb      1/1     Running   0          3d2h
+ousast-pool-6656b55f5b-5brkn   1/1     Running   0          5m23s
+ousast-pool-6656b55f5b-9wbrw   1/1     Running   0          3d2h
+
+== artifact receiver Service (ousast-receiver.ax-system -> this host) (20:35:35)
+service/ousast-receiver unchanged
+endpointslice.discovery.k8s.io/ousast-receiver-host configured
+
+== worker pool (gVisor): pools.default of the profile (20:35:36)
+pool ousast-pool exists (2/2 ready); kept
+pod/ousast-pool-6656b55f5b-5brkn condition met
+pod/ousast-pool-6656b55f5b-9wbrw condition met
+
+== runner image localhost:5001/ousast-runner (20:35:37)
+sha256:f004a5ef8f71b1cb8a5880c509389c0569181af7b19170b574958cc3ef75cb1c
+localhost:5001/ousast-runner:dev
+runner image localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
+re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
+
+== ax CLI from the deployed checkout (a release CLI skews from the server) (20:36:05)
+ax CLI newer than the checkout; kept
+
+== smoke task (20:36:05)
+task.ax.io/ousast-smoke created
+workspace.ax.io/ousast-smoke-ws unchanged
+task.ax.io/ousast-smoke resumed
+NAME           ATESPACE   PHASE     ACTOR          WORKER-IP     AGE
+ousast-smoke   default    Running   ousast-smoke   10.244.0.29   5s
+
+== footprint (20:36:11)
+               total        used        free      shared  buff/cache   available
+Mem:            7877        4797         225          63        3464        3079
+kind-registry 53.77MiB / 7.693GiB 0.00%
+up.sh exit 0

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/egress-policy-read-back.json

@@ -0,0 +1,24 @@
+{
+  "metadata": {
+    "atespace": "default",
+    "createTime": "2026-10-02T19:39:19.428045847Z",
+    "name": "default",
+    "uid": "741e52b7-aaf1-4a91-b4bf-fc3b20ba4a36",
+    "updateTime": "2026-10-02T19:39:19.428045847Z",
+    "version": "1"
+  },
+  "rules": [
+    {
+      "tlsPassthrough": {
+        "hostnames": [
+          "files.because-security.com"
+        ],
+        "ports": {
+          "numbers": [
+            443
+          ]
+        }
+      }
+    }
+  ]
+}

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json

@@ -0,0 +1,83 @@
+{
+  "record": "k8s-02-store-delivery",
+  "spec": "plane-on-kubernetes",
+  "tasks": ["2.1", "2.2", "2.3", "2.4", "2.5", "2.6"],
+  "date": "2026-10-02",
+  "commits": {
+    "2.1": "35d8032",
+    "2.2": "9506a18",
+    "2.3": "c49eae4",
+    "2.4": "605b7af",
+    "2.5": "the commit that adds this file (git log: plane(2.5))",
+    "receiver_removed": "pending: task 2.6 (paused 2026-10-03 by the maintainer's reprioritisation; the receiver path still exists behind OUSAST_DELIVERY=receiver)"
+  },
+  "images": {
+    "runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019",
+    "built_by": "ops/ax/up.sh rerun (file up-sh-rebuild.txt): every step kept but the runner build; the image carries the 2.2 runner",
+    "disk_gb_free_after_build": 1.3
+  },
+  "instrument_checks": {
+    "code_under_test": "the worktree's src first on PYTHONPATH; `ousast` run as `python -m openultrasast.cli` from the worktree (ops/ax/smoke-run.sh with OUSAST_PYTHON)",
+    "store_read_back": {
+      "how": "scratch store_tar_check.py opens the store through the worktree code (verify_bucket runs), lists runs/ax-e2e-smoke/, downloads output.tar, compares its members with the files the reconciler extracted (file store-check.json)",
+      "tar_bytes": 10240,
+      "tar_members_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
+      "extracted_files_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
+      "members_equal_extracted": true,
+      "member_bytes_total": 980,
+      "note": "a tar is 512-byte blocks: 10240 B of tar carry 980 B of files; done.json records the tar's byte size"
+    },
+    "no_receiver_in_the_task": "the rendered task.yaml is 996 B against 1148 B on the receiver path: no OUSAST_ARTIFACT_URL/DIAL in the Task's env; the start body carries the URLs"
+  },
+  "smoke_runs": {
+    "store_1": {
+      "command": "ops/ax/smoke-run.sh (store delivery, the default after 2.4; OUSAST_DELIVERY unset)",
+      "exit": 0,
+      "seconds_submit_to_done": 26,
+      "tar_put_at": "2026-10-02T19:37:24+00:00 (the object's LastModified)",
+      "collected_at": "2026-10-02T19:37:26+00:00 (done.json finished)",
+      "put_to_collect_seconds": 2,
+      "poll_interval_seconds": 3,
+      "file": "smoke-run-store-1.txt"
+    },
+    "seeded_rerun": {
+      "command": "the same, with the local results root deleted and the store's runs/ax-e2e-smoke/ intact",
+      "exit": 0,
+      "seconds": 2,
+      "what_happened": "state.json seeded from the store: every task done, nothing submitted to ax (plane-on-kubernetes 2.1, design section 2: a rerun from another machine resumes from the store)",
+      "file": "smoke-run-seeded-rerun.txt"
+    },
+    "store_2": {
+      "command": "the same, after deleting runs/ax-e2e-smoke/ from the store (5 objects) so the task executes again, with the egress policy polled",
+      "exit": 0,
+      "seconds_submit_to_done": 24,
+      "tar_put_at": "2026-10-02T19:39:38+00:00",
+      "collected_at": "2026-10-02T19:39:40+00:00",
+      "put_to_collect_seconds": 2,
+      "file": "smoke-run-store-2.txt"
+    }
+  },
+  "objects_delivered_to_the_store": {
+    "runs/ax-e2e-smoke/facts/output.tar": 10240,
+    "runs/ax-e2e-smoke/facts/facts.json": 451,
+    "runs/ax-e2e-smoke/facts/summary.json": 238,
+    "runs/ax-e2e-smoke/facts/done.json": 149,
+    "runs/ax-e2e-smoke/state.json": 350,
+    "note": "units.jsonl is extracted locally but not a declared output of the smoke Run, so it is not re-put"
+  },
+  "egress_policy_read_back": {
+    "how": "kubectl ate get egress-policy ax-e2e-smoke-facts -a default -o json, polled while the actor lived (file egress-policy-read-back.json)",
+    "rules": [{"tlsPassthrough": {"hostnames": ["files.because-security.com"], "ports": {"numbers": [443]}}}],
+    "reading": "one TLS rule naming the store host, no http rule; the model-free smoke task names no Git host and no Model host",
+    "also": "Egress.apply reads every policy back and raises on a mismatch, so each completed Run is itself the equality check"
+  },
+  "status_table": "ousast plane status ax-e2e-smoke printed the attribution table (facts: done, calls 0) in every run; the table is in the smoke-run-*.txt files",
+  "reconciler_lines": {"after_2_4": 549, "bound": 550, "note": "both paths coexist until 2.6; the receiver wiring goes then"},
+  "gates": {
+    "2.1": "pytest 2020 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
+    "2.2": "pytest 2024 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
+    "2.3": "pytest 2027 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
+    "2.4": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
+    "2.5": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0 (first attempt 2026-10-02: 11 failed + 5 errors, all the FileStore 1 GiB free-space refusal at 0.97 GiB free; rerun after the disk was freed)"
+  }
+}

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-seeded-rerun.txt

@@ -0,0 +1,9 @@
+run finished: done
+memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
+run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
+
+TASK                         MODEL                  CALLS     PROMPT  CACHE-HIT    OUTPUT        USD
+facts                        -                        n/a        n/a        n/a       n/a        n/a  [done]
+----------------------------------------------------------------------------------------------------
+total                        -                        n/a        n/a        n/a       n/a        n/a
+smoke exit=0 seconds=2

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-1.txt

@@ -0,0 +1,9 @@
+run finished: done
+memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
+run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
+
+TASK                         MODEL                  CALLS     PROMPT  CACHE-HIT    OUTPUT        USD
+facts                        -                          0        n/a        n/a       n/a        n/a  [done]
+----------------------------------------------------------------------------------------------------
+total                        -                        n/a        n/a        n/a       n/a        n/a
+smoke exit=0 seconds=26

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-2.txt

@@ -0,0 +1,9 @@
+run finished: done
+memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
+run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
+
+TASK                         MODEL                  CALLS     PROMPT  CACHE-HIT    OUTPUT        USD
+facts                        -                          0        n/a        n/a       n/a        n/a  [done]
+----------------------------------------------------------------------------------------------------
+total                        -                        n/a        n/a        n/a       n/a        n/a
+smoke exit=0 seconds=24

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/store-check.json

@@ -0,0 +1,36 @@
+{
+ "store": "s3://sast-memory",
+ "objects_under_run": {
+  "runs/ax-e2e-smoke/facts/done.json": 149,
+  "runs/ax-e2e-smoke/facts/facts.json": 451,
+  "runs/ax-e2e-smoke/facts/output.tar": 10240,
+  "runs/ax-e2e-smoke/facts/summary.json": 238,
+  "runs/ax-e2e-smoke/state.json": 350
+ },
+ "tar_bytes": 10240,
+ "tar_members": {
+  "facts.json": 451,
+  "summary.json": 238,
+  "units.jsonl": 291
+ },
+ "extracted_files": {
+  "facts.json": 451,
+  "units.jsonl": 291,
+  "summary.json": 238
+ },
+ "members_equal_extracted": true,
+ "member_bytes_total": 980,
+ "extracted_bytes_total": 980,
+ "done_json": {
+  "bytes": 10240,
+  "files": 3,
+  "finished": "2026-10-02T19:39:40+00:00",
+  "published": [
+   "facts.json",
+   "summary.json"
+  ],
+  "status": "done",
+  "task": "facts"
+ },
+ "tar_put_at_last_modified": "2026-10-02T19:39:38+00:00"
+}

benchmarks/measurements/2026-10-02-k8s-02-store-delivery/up-sh-rebuild.txt

@@ -0,0 +1,58 @@
+
+== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (21:36:18)
+cluster exists; kept
+
+== agent substrate (ate-system) (21:36:18)
+substrate ready; kept
+NAME                              READY   STATUS      RESTARTS       AGE
+ate-api-server-647bb6f8b6-c9k5k   1/1     Running     2 (13h ago)    3d1h
+ate-api-server-647bb6f8b6-dznjn   1/1     Running     1 (13h ago)    3d1h
+ate-controller-c95586684-g7nvn    1/1     Running     0              3d3h
+atelet-4f9c293-9zpfg              1/1     Running     0              3d3h
+atenet-egress-d74b8f6bf-8zmxj     1/1     Running     0              3d1h
+atenet-router-54887bc8f9-rqs6g    2/2     Running     2 (13h ago)    3d3h
+postgres-0                        2/2     Running     3 (3d3h ago)   3d3h
+rustfs-6c4c677cb6-cqvhc           1/1     Running     3 (3d3h ago)   3d3h
+rustfs-bucket-init-nvfr6          0/1     Completed   0              3d3h
+
+== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (21:36:18)
+gateway present; kept
+
+== ax control plane (ax-system) (21:36:19)
+ax ready; kept
+NAME                           READY   STATUS    RESTARTS   AGE
+ax-redis-86df786c87-pz5tp      1/1     Running   0          3d3h
+ax-server-b479bdc76-hwkqb      1/1     Running   0          3d3h
+ousast-pool-6656b55f5b-5brkn   1/1     Running   0          66m
+ousast-pool-6656b55f5b-9wbrw   1/1     Running   0          3d3h
+
+== artifact receiver Service (ousast-receiver.ax-system -> this host) (21:36:19)
+service/ousast-receiver unchanged
+endpointslice.discovery.k8s.io/ousast-receiver-host configured
+
+== worker pool (gVisor): pools.default of the profile (21:36:20)
+pool ousast-pool exists (2/2 ready); kept
+pod/ousast-pool-6656b55f5b-5brkn condition met
+pod/ousast-pool-6656b55f5b-9wbrw condition met
+
+== runner image localhost:5001/ousast-runner (21:36:21)
+sha256:0501baf180b29f7aa7bbb8888bebabfc5d62429b7bdaa7f5a62b5b188f8ad5f6
+localhost:5001/ousast-runner:dev
+runner image localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
+re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
+
+== ax CLI from the deployed checkout (a release CLI skews from the server) (21:36:32)
+ax CLI newer than the checkout; kept
+
+== smoke task (21:36:32)
+task.ax.io/ousast-smoke created
+workspace.ax.io/ousast-smoke-ws unchanged
+task.ax.io/ousast-smoke resumed
+NAME           ATESPACE   PHASE     ACTOR          WORKER-IP     AGE
+ousast-smoke   default    Running   ousast-smoke   10.244.0.29   4s
+
+== footprint (21:36:38)
+               total        used        free      shared  buff/cache   available
+Mem:            7877        5202         382          63        2903        2674
+kind-registry 32.45MiB / 7.693GiB 0.00%
+up.sh exit 0

benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json

@@ -0,0 +1,792 @@
+{
+  "id": "2026-10-02-pre-push-operational-1",
+  "task": "pre-push-safety-net 17.7",
+  "requirements": [
+    "9.1",
+    "9.2",
+    "9.3",
+    "9.4",
+    "9.5",
+    "9.6",
+    "6.3"
+  ],
+  "question": "Does the hook installed on any repository give honest, advisory-only results, and never block a push?",
+  "subjects": "The audit's three shallow clones (kennethreitz/records, expressjs/serve-static, unrolled/secure), each with the same planted commit c2 on c1, copied fresh from the audit's scratch clones.",
+  "instrument_proof": "Every replay and hook push records snapshots[].bytes_read and quick_tier files/bytes_read. The head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head (records 43,577; serve-static 61,525; secure 103,023), and the quick tier reads exactly the changed source files.",
+  "host": "Joern is not on the host, so host runs cannot run the engine; that is the case a first-time pip user hits, and the output must say so. The Docker engine run uses the 2026-09-14 openultrasast:dev image with the worktree src mounted read-only, as the audit did; the image was not rebuilt (1.1 GB disk free, below the 4 GB threshold).",
+  "before": {
+    "source": "/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready/gap-report.md (audit at main 49e11e0)",
+    "replay_host": {
+      "records": {
+        "exit": 0,
+        "wall_seconds": 1.06,
+        "files_bytes_read": [
+          21,
+          43577
+        ],
+        "findings_shown": 0,
+        "terminal": "generic: Some checks could not be completed. Review the comparison details...",
+        "skip_reasons_named_in_terminal": 0
+      },
+      "serve-static": {
+        "exit": 0,
+        "wall_seconds": 1.1,
+        "files_bytes_read": [
+          26,
+          61525
+        ],
+        "findings_shown": 0,
+        "terminal": "same generic line",
+        "skip_reasons_named_in_terminal": 0
+      },
+      "secure": {
+        "exit": 0,
+        "wall_seconds": 1.04,
+        "files_bytes_read": [
+          19,
+          103023
+        ],
+        "findings_shown": 0,
+        "terminal": "same generic line; Go not named",
+        "skip_reasons_named_in_terminal": 0
+      }
+    },
+    "docker_engine_serve_static": {
+      "deadline_30": {
+        "exit": 0,
+        "wall_seconds": 31.3,
+        "terminal": "Analysis did not finish. Rerun with a longer analysis deadline."
+      },
+      "deadline_300": {
+        "exit": 0,
+        "wall_seconds": 261.7,
+        "raw_engine_findings": 2,
+        "admitted": 0,
+        "dispositions": "injection and output_encoding at debug.js:2: comparison_unknown/base_incomplete + capability_unavailable",
+        "terminal": "No actionable defects reported; Details: /out/... (container path)"
+      }
+    },
+    "quick_scan": {
+      "serve-static": {
+        "exit": 0,
+        "wall_seconds": 0.46,
+        "findings": "js-eval, js-xss-response at debug.js:2"
+      },
+      "records": {
+        "exit": 0,
+        "wall_seconds": 0.49,
+        "findings": "2 pre-existing (setup.py os-command); planted %-formatted SQL missed"
+      },
+      "secure": {
+        "exit": 0,
+        "wall_seconds": 0.48,
+        "findings": 0,
+        "terminal": "No quick-mode findings were emitted (silent zero for Go)"
+      }
+    },
+    "hook": {
+      "ousast_not_on_path": {
+        "push_exit": "rejected",
+        "terminal": ".git/hooks/pre-push: 13: exec: ousast: not found"
+      },
+      "first_push_new_branch": {
+        "push_exit": 0,
+        "wall_seconds": 0.58,
+        "terminal": "No analysis result is available. (does not say new branch)"
+      },
+      "update": {
+        "push_exit": 0,
+        "wall_seconds": 1.36,
+        "terminal": "No actionable defects reported; analysis is incomplete."
+      },
+      "bad_OUSAST_PUSH_DEADLINE_or_cache_inside_repo": {
+        "exit": 2,
+        "effect": "would block the push"
+      },
+      "install": "ops/install-pre-push only (checkout); not in the wheel"
+    }
+  },
+  "verdict": {
+    "never_blocks": "PASS in this sample: 10 hook pushes across the three repositories all exited 0, including `ousast` absent from PATH (rejected before) and OUSAST_PUSH_DEADLINE=abc (exit 2 before). Blocking stays an explicit opt-in (unit-tested, not exercised here).",
+    "never_hides": "PASS in this sample: every incomplete run printed one plain `Skipped:` line per reason (2-5 per run), with no hex path or sha256 token. Go is named as not covered (`language not covered: go (1 changed file(s), 10 in the repository)`, and `notice: language not covered: go (10 files)` in `scan --mode quick`). The first push of a new branch says there is no base and what it checked instead.",
+    "quick_tier_within_deadline": "On the host, every replay and hook push finished in 0.69-1.21 s against the 30 s deadline. The quick tier read exactly the changed source file (records 1 file / 17,449 B; serve-static 1 / 87 B; secure 1 / 193 B).",
+    "planted_js_eval": "SURFACES. Quick tier: js-eval (CWE-95) and js-xss-response (CWE-79) at debug.js:2 on the host, 0.9 s, in replay and in the hook push. Engine (Docker, deadline 300): injection and output_encoding at debug.js:2, now novelty `new` / `file_added_in_head` (before: `unknown` / `base_incomplete`). The only veto is `capability_unavailable`, so it is shown as an advisory engine finding, not an alert.",
+    "planted_python_sql": "STILL MISSED. No quick rule matches the %-formatted db.query string in records.py, and the host has no Joern, so nothing in the pre-push path reports it. Recorded as a coverage gap, with no rule edit.",
+    "planted_go_command": "NOT DETECTED, NOW NAMED: Go has no engine frontend and no quick rules.",
+    "engine_deadline": "Not solved. The Docker engine took 190.9 s wall time (head 107.6 s, base 80.7 s) against a 300 s deadline. The audit measured 261.7 s on a different image/src combination. A 30 s hook deadline still cannot hold it. --engine background starts it after the push, and the next push showed the earlier result. On the host that result says 'the engine did not run (Joern unavailable)', which is honest."
+  },
+  "coverage_gaps": [
+    "records: the planted Python SQL injection (`db.query(\"... '%s'\" % name)`) is matched by no quick rule, before or after. Recorded as a quick-rule coverage gap; no rule was edited for a planted sample (that would be hand-tuning). On the host the engine cannot run, so nothing in the pre-push path reports it.",
+    "secure: Go has no engine frontend and no quick rules. It is now named in the terminal (language not covered: go), not detected.",
+    "The engine still cannot finish inside the 30 s hook deadline (the audit measured 260 s on 26 files); --engine background moves it after the push, graph reuse remains the later speed task."
+  ],
+  "after": {
+    "replay_host": {
+      "records": {
+        "pre-push --engine inline": {
+          "exit": 0,
+          "wall_seconds": 1.206878771,
+          "deadline_seconds": 30,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "changed lines could not be matched one-to-one to base lines in records.py",
+            "the changed code has no counterpart in the base revision to compare against",
+            "engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
+            "No actionable defects reported; analysis is incomplete.",
+            "Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
+          ],
+          "snapshots_files_bytes": [
+            [
+              21,
+              43577
+            ],
+            [
+              21,
+              43482
+            ]
+          ],
+          "quick_tier_files_bytes": [
+            [
+              1,
+              17449
+            ]
+          ],
+          "quick_tier_seconds": [
+            0.041
+          ],
+          "coverage_reasons_raw": [
+            "7265636f7264732e7079:ambiguous_line_correspondence",
+            "base_counterpart_unresolved",
+            "cpg_build_failed",
+            "analysis_incomplete",
+            "base_comparison_unavailable"
+          ],
+          "admitted_alerts": 0,
+          "push_disposition": "allow",
+          "total_seconds_internal": 0.483
+        },
+        "pre-push --engine off": {
+          "exit": 0,
+          "wall_seconds": 0.954010589,
+          "deadline_seconds": 30,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "changed lines could not be matched one-to-one to base lines in records.py",
+            "engine checks were switched off for this run (--engine off)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
+          ],
+          "snapshots_files_bytes": [
+            [
+              21,
+              43577
+            ],
+            [
+              21,
+              43482
+            ]
+          ],
+          "quick_tier_files_bytes": [
+            [
+              1,
+              17449
+            ]
+          ],
+          "quick_tier_seconds": [
+            0.042
+          ],
+          "coverage_reasons_raw": [
+            "7265636f7264732e7079:ambiguous_line_correspondence",
+            "engine_off",
+            "analysis_incomplete",
+            "base_comparison_unavailable"
+          ],
+          "admitted_alerts": 0,
+          "push_disposition": "allow",
+          "total_seconds_internal": 0.498
+        },
+        "scan --mode quick": {
+          "exit": 0,
+          "wall_seconds": 0.407089313,
+          "terminal": [
+            "file_targets=9",
+            "findings=2"
+          ]
+        }
+      },
+      "serve-static": {
+        "pre-push --engine inline": {
+          "exit": 0,
+          "wall_seconds": 0.892543629,
+          "deadline_seconds": 30,
+          "quick_findings_shown": [
+            "js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
+            "js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
+          ],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "the changed code has no counterpart in the base revision to compare against",
+            "engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
+            "No actionable defects reported; analysis is incomplete.",
+            "Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
+          ],
+          "snapshots_files_bytes": [
+            [
+              26,
+              61525
+            ],
+            [
+              25,
+              61438
+            ]
+          ],
+          "quick_tier_files_bytes": [
+            [
+              1,
+              87
+            ]
+          ],
+          "quick_tier_seconds": [
+            0.017
+          ],
+          "coverage_reasons_raw": [
+            "base_counterpart_unresolved",
+            "cpg_build_failed",
+            "analysis_incomplete",
+            "base_comparison_unavailable"
+          ],
+          "admitted_alerts": 0,
+          "push_disposition": "allow",
+          "total_seconds_internal": 0.496
+        },
+        "pre-push --engine off": {
+          "exit": 0,
+          "wall_seconds": 0.878791036,
+          "deadline_seconds": 30,
+          "quick_findings_shown": [
+            "js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
+            "js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
+          ],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "engine checks were switched off for this run (--engine off)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
+          ],
+          "snapshots_files_bytes": [
+            [
+              26,
+              61525
+            ],
+            [
+              25,
+              61438
+            ]
+          ],
+          "quick_tier_files_bytes": [
+            [
+              1,
+              87
+            ]
+          ],
+          "quick_tier_seconds": [
+            0.016
+          ],
+          "coverage_reasons_raw": [
+            "engine_off",
+            "analysis_incomplete",
+            "base_comparison_unavailable"
+          ],
+          "admitted_alerts": 0,
+          "push_disposition": "allow",
+          "total_seconds_internal": 0.471
+        },
+        "scan --mode quick": {
+          "exit": 0,
+          "wall_seconds": 0.394269229,
+          "terminal": [
+            "file_targets=4",
+            "findings=2"
+          ]
+        }
+      },
+      "secure": {
+        "pre-push --engine inline": {
+          "exit": 0,
+          "wall_seconds": 0.762186729,
+          "deadline_seconds": 30,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
+            "the changed code has no counterpart in the base revision to compare against",
+            "engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
+            "engine has no frontend for at least one language in this push",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
+            "No actionable defects reported; analysis is incomplete.",
+            "Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
+          ],
+          "snapshots_files_bytes": [
+            [
+              19,
+              103023
+            ],
+            [
+              18,
+              102830
+            ]
+          ],
+          "quick_tier_files_bytes": [
+            [
+              1,
+              193
+            ]
+          ],
+          "quick_tier_seconds": [
+            0.015
+          ],
+          "coverage_reasons_raw": [
+            "language_not_covered:go:1:10",
+            "base_counterpart_unresolved",
+            "cpg_build_failed",
+            "frontend_unsupported",
+            "analysis_incomplete",
+            "base_comparison_unavailable"
+          ],
+          "admitted_alerts": 0,
+          "push_disposition": "allow",
+          "total_seconds_internal": 0.352
+        },
+        "pre-push --engine off": {
+          "exit": 0,
+          "wall_seconds": 0.694374205,
+          "deadline_seconds": 30,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
+            "engine checks were switched off for this run (--engine off)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
+          ],
+          "snapshots_files_bytes": [
+            [
+              19,
+              103023
+            ],
+            [
+              18,
+              102830
+            ]
+          ],
+          "quick_tier_files_bytes": [
+            [
+              1,
+              193
+            ]
+          ],
+          "quick_tier_seconds": [
+            0.016
+          ],
+          "coverage_reasons_raw": [
+            "language_not_covered:go:1:10",
+            "engine_off",
+            "analysis_incomplete",
+            "base_comparison_unavailable"
+          ],
+          "admitted_alerts": 0,
+          "push_disposition": "allow",
+          "total_seconds_internal": 0.311
+        },
+        "scan --mode quick": {
+          "exit": 0,
+          "wall_seconds": 0.372512766,
+          "terminal": [
+            "file_targets=10",
+            "findings=0",
+            "notice: language not covered: go (10 files)"
+          ]
+        }
+      }
+    },
+    "hook_host": {
+      "records": {
+        "new-branch": {
+          "push_exit": 0,
+          "wall_seconds": 0.873912782,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "New branch: instead, quick rules checked the files its last commit changed (compared with c925575).",
+            "Quick rules: no pattern match on changed code (2 file(s), 18416 bytes read).",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
+            " * [new branch]      HEAD~1 -> main"
+          ]
+        },
+        "update": {
+          "push_exit": 0,
+          "wall_seconds": 0.890046656,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "changed lines could not be matched one-to-one to base lines in records.py",
+            "the changed code has no counterpart in the base revision to compare against",
+            "engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
+            "No actionable defects reported; analysis is incomplete.",
+            "Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
+            "   ea42736..ad3d7db  HEAD -> main"
+          ]
+        },
+        "install_uninstall": [
+          "install rc=0",
+          "uninstall rc=0"
+        ]
+      },
+      "serve-static": {
+        "new-branch": {
+          "push_exit": 0,
+          "wall_seconds": 0.904302643,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "New branch: instead, quick rules checked the files its last commit changed (compared with 76312e1).",
+            "Quick rules: no pattern match on changed code (0 file(s), 0 bytes read).",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
+            " * [new branch]      HEAD~1 -> main"
+          ]
+        },
+        "update": {
+          "push_exit": 0,
+          "wall_seconds": 0.92386555,
+          "quick_findings_shown": [
+            "js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
+            "js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
+          ],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "the changed code has no counterpart in the base revision to compare against",
+            "engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
+            "No actionable defects reported; analysis is incomplete.",
+            "Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
+            "   74be78a..468213d  HEAD -> main"
+          ]
+        },
+        "no-ousast": {
+          "push_exit": 0,
+          "wall_seconds": 0.017937758,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [],
+          "other_lines": [
+            "ousast: pre-push check did not run: 'ousast' is not on PATH. Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
+            "   468213d..aad3dfe  HEAD -> main"
+          ]
+        },
+        "bad-deadline": {
+          "push_exit": 0,
+          "wall_seconds": 0.300776,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [],
+          "other_lines": [
+            "usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]",
+            "                       [--comparison-base COMPARISON_BASE]",
+            "                       [--model-config MODEL_CONFIG] [--prior-hook PRIOR_HOOK]",
+            "                       --artifact ARTIFACT [--cache-dir CACHE_DIR]",
+            "                       [--deadline DEADLINE]",
+            "                       [--cancellation-allowance CANCELLATION_ALLOWANCE]",
+            "                       [--max-regions MAX_REGIONS]",
+            "                       [--mode {advisory,blocking}]",
+            "                       [--incomplete-coverage {allow,block}]",
+            "                       [--engine {inline,background,off}]",
+            "                       [--background-deadline BACKGROUND_DEADLINE]",
+            "                       [--experimental-declarations EXPERIMENTAL_DECLARATIONS]",
+            "                       [--experimental-record-vetoes]",
+            "                       path",
+            "ousast pre-push: error: argument --deadline: invalid float value: 'abc'",
+            "ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
+            "   aad3dfe..6fcfec7  HEAD -> main"
+          ]
+        },
+        "background-1": {
+          "push_exit": 0,
+          "wall_seconds": 0.915950565,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [
+            "Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result."
+          ],
+          "skip_lines": [
+            "engine checks run in the background after this push; the next push shows their result",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 93 bytes read).",
+            "No analysis result is available.",
+            "Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
+            "   6fcfec7..c5b6c4d  HEAD -> main"
+          ]
+        },
+        "background-2": {
+          "push_exit": 0,
+          "wall_seconds": 0.879868436,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [
+            "Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable)."
+          ],
+          "skip_lines": [
+            "engine checks were switched off for this run (--engine off)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable).",
+            "Quick rules: no pattern match on changed code (1 file(s), 99 bytes read).",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
+            "   c5b6c4d..3eee5ef  HEAD -> main"
+          ]
+        },
+        "install_uninstall": [
+          "install rc=0",
+          "uninstall rc=0"
+        ]
+      },
+      "secure": {
+        "new-branch": {
+          "push_exit": 0,
+          "wall_seconds": 0.96295861,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "New branch: instead, quick rules checked the files its last commit changed (compared with ba42bc3).",
+            "Quick rules: no pattern match on changed code (1 file(s), 41228 bytes read).",
+            "No analysis result is available.",
+            "Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
+            " * [new branch]      HEAD~1 -> main"
+          ]
+        },
+        "update": {
+          "push_exit": 0,
+          "wall_seconds": 0.747049238,
+          "quick_findings_shown": [],
+          "engine_findings_shown": [],
+          "skip_lines": [
+            "language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
+            "the changed code has no counterpart in the base revision to compare against",
+            "engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
+            "engine has no frontend for at least one language in this push",
+            "no base revision was analysed, so no engine finding can be called new or worsened"
+          ],
+          "other_lines": [
+            "Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
+            "No actionable defects reported; analysis is incomplete.",
+            "Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
+            "To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
+            "   740abc8..64739c4  HEAD -> main"
+          ]
+        },
+        "install_uninstall": [
+          "install rc=0",
+          "uninstall rc=0"
+        ]
+      }
+    },
+    "docker_engine_serve_static": {
+      "log": [
+        "MemAvailable=3283MB disk_free=1159MiB",
+        "base=74be78a8ffad679edfe135c457ce141114d96fcc head=468213d8a737ad03a1f712c792c1db8476e154df",
+        "exit=0 elapsed=190.895648822s"
+      ],
+      "exit": 0,
+      "wall_seconds": 190.895648822,
+      "deadline_seconds": 300,
+      "quick_findings_shown": [
+        "js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
+        "js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
+      ],
+      "engine_findings_shown": [
+        "Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert."
+      ],
+      "skip_lines": [
+        "engine findings are not qualified as alerts yet: no capability has passed independent evaluation",
+        "the capability registry does not match this build (eligibility provenance mismatch), so no capability can be used",
+        "20 engine check(s) depend on dynamic or external code the engine cannot follow"
+      ],
+      "other_lines": [
+        "Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert.",
+        "Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
+        "No actionable defects reported; analysis is incomplete.",
+        "Notice: Some checks are not enabled for normal alerts. Review capability coverage in the details."
+      ],
+      "snapshots_files_bytes": [
+        [
+          26,
+          61525
+        ],
+        [
+          25,
+          61438
+        ]
+      ],
+      "quick_tier_files_bytes": [
+        [
+          1,
+          87
+        ]
+      ],
+      "quick_tier_seconds": [
+        0.038
+      ],
+      "coverage_reasons_raw": [
+        "capability_unavailable",
+        "eligibility:eligibility_provenance_mismatch",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:7ba72698b15c888c0835cad443fc1f177f1d2e13ed2658b17e816cc0b81415de",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:392beff95d0e245cbaef2676813d7f69d35599fb32977c8f197e6df46738fc75",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:ec16a67e984a2f82c3ce9a04dbc44eae1e950393b1c1c4fae88485446f1e2f95",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:0a3bd3f145260792e9a5d4af2546020689cfd08d0b1e82264d7fe31dda7210e7",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:1551c000ba35ba4e5488013f51171b27ab10df8b80f29509610670192cff8a30",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:5b687fa7647cefd89d064b3e1b5e5a62842f8fd23104900e8af8d5b14fc800fd",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:f122da538137e0a3a3e598c24e6af57c5e0b90ff48dacb4fd1b9d82ac20c35ef",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:8ab9ddcb1f4431fa3a3c1cce4cc3d289bcb5d363d47aac1bcf35bc7db86ce866",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:19f1031c2c010d91fb41e23955cd5f18eaa5ede355e68fab9e18c8249c21c2ee",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:d31a861124274a09e3cb8f3398831ca299cc0cfad72ea385083c1fd88b7a2c45",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:c8a1c8042befd3ff6f296921666949850a1c54d63c77dd44296e7be96e621127",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:3bb50c50f77027f1e0347f490e35fdd0a75ae586fdf92775c753877833a9d099",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:c32cb93cad379fee7b814900e6673aa5ae470c27acd55d33f855987bccb1ab8e",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:300fba1acc39cb0ab32ebbd66a8674daecf496b3d8c675d27f7f71cf9f788c9e",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:40c99dfe6d923602dc3aa6353f3b5bc017c0d1a67595c19ddf8206ef431db23d",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:0e99666c9be7c4704f6c6f858844d721ed8f0b4e3978d53073bc2c1b64d3f075",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:da903214d018e0b1bd7a3a3b635b3e0fb2a23fd44527f41a9483c0bf3913f2df",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:79d89b41b86aadcc0423baa786e0518ccff2a5b7853ff05986682ecbd636af52",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:7d3331ca3c06d1013c758d3648bfebb1223385955e77df977266fc621f62e733",
+        "dynamic_external_or_depth_context_unresolved:contributor-scan:54fd93450c8db7e6daef06fe795babb26726402211ca8f6d0a8679a375786041",
+        "analysis_incomplete"
+      ],
+      "admitted_alerts": 0,
+      "push_disposition": "allow",
+      "total_seconds_internal": 189.327,
+      "dispositions": [
+        {
+          "family": "injection",
+          "site": "debug.js:2:debug",
+          "novelty": "new",
+          "reason": "file_added_in_head",
+          "vetoes": [
+            "capability_unavailable"
+          ]
+        },
+        {
+          "family": "output_encoding",
+          "site": "debug.js:2:debug",
+          "novelty": "new",
+          "reason": "file_added_in_head",
+          "vetoes": [
+            "capability_unavailable"
+          ]
+        }
+      ],
+      "timings": {
+        "deadline_seconds": 300.0,
+        "cancellation_allowance_seconds": 2.0,
+        "resolution_seconds": 0.0,
+        "provenance_seconds": 0.3,
+        "snapshot_seconds": 0.3,
+        "discovery_seconds": 0.3,
+        "preparation_seconds": 0.6,
+        "quick_seconds": 0.0,
+        "head_seconds": 107.6,
+        "base_seconds": 80.7,
+        "admission_seconds": 0.0,
+        "cleanup_seconds": 0.0,
+        "total_seconds": 189.3
+      }
+    }
+  },
+  "build": {
+    "worktree_head": "591818d + hook install-hint fix (this commit)",
+    "wheel_sha256": "5b725adb052239c2afaec237fecf582f69fc5ea61799dc67f0c07e09e10908c9",
+    "installed_with": "uv pip install <wheel> into a scratch venv"
+  },
+  "inputs": {
+    "records": {
+      "language": "python",
+      "tracked_files": 21,
+      "tracked_bytes": 43577,
+      "planted": "records.py: db.query(\"... '%s'\" % name)"
+    },
+    "serve-static": {
+      "language": "javascript",
+      "tracked_files": 26,
+      "tracked_bytes": 61525,
+      "planted": "new debug.js: res.end(String(eval(req.query.code)))"
+    },
+    "secure": {
+      "language": "go",
+      "tracked_files": 19,
+      "tracked_bytes": 103023,
+      "planted": "new run.go: exec.Command(\"sh\", \"-c\", r.URL.Query().Get(\"cmd\"))"
+    }
+  },
+  "notes": [
+    "The Docker engine run used the worktree src at 591818d mounted over the stale 2026-09-14 openultrasast:dev image (not rebuilt: 1.1 GB disk free, below the 4 GB threshold). `eligibility_provenance_mismatch` still appears; the audit already suspected src/image skew, and it is not investigated here.",
+    "The run shows host paths after `Details:` because ops/ousast-docker mounts the artifact directory at its own path. Before, it showed the container path /out/...",
+    "The hook's 'not on PATH' hint first said 'pip install openultrasast', which is not on PyPI. The first proof run caught it. It was corrected to the git URL, and the replay and hook proofs were rerun on the rebuilt wheel recorded under build. The Docker engine run predates the hint fix and does not involve the hook script.",
+    "Host runs read their input. Each head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head, and the quick tier lists the files it read."
+  ]
+}

benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest/record.json

@@ -0,0 +1,351 @@
+{
+ "created": "2026-10-03",
+ "note": "COUNTS ONLY. Real (vulnerable function, fixed function) pairs where the function survives the fix, mined at zero model cost from reviewed GitHub advisories for the six compiled families, added as pointer-pair corpus benchmarks/pairs/advisory-fixes-2 (98 pairs, source `advisory-fixes-2` in learn/sources.toml), labelled, given static feature records and built into the memory store; their excerpts embedded. No chat-model call. Candidate, path, function and repository names stay in the catalog; the two pairs the reserved-population test flagged are withheld (never printed: found by bisection with the test's output discarded).",
+ "method": {
+  "enumeration": "gh api /advisories?type=reviewed&ecosystem=<eco>&cwes=<family CWEs>&published=>2025-06-01, per ecosystem (pip, npm, maven, composer, go, rubygems) x family; newest first",
+  "advisories_enumerated": 6594,
+  "advisories_published_after_2026_06_01": 2400,
+  "automated_screen": "single family by CWE; a linked fix commit (at most 3, one repository); repository not named anywhere under benchmarks/ (populations v1/v2, every pair corpus; excluded v3 file names never opened), no WordPress plugin, one pair per repository; SPDX licence permissive; one-parent commit with <= 4 non-test source files and <= 120 changed source lines; the function holding most changed CODE lines (comments and blanks excluded) inside its own matcher span, declared exactly once on both sides, body changed, >= 25% of the file's changed code lines (second pass; the first pass counted lines by nearest declaration above)",
+  "file_reads": "the two versions of each changed source file through the contents API (metadata + two raw files per file; no clone, no scanner, engine, plane task or model on any of these repositories)",
+  "manual_review": "every automated candidate's function diff was read (agent, no model call) and kept only when the named function itself holds the fix",
+  "reserved_population_guard": "tests/test_independent_population.py::test_no_reserved_repository_is_referenced_by_the_development_tree, run with its output discarded; bisection over catalog subsets, 23 runs",
+  "quotas": "20 per family on the second pass (14 on the first); advisories past a met quota were not examined",
+  "mechanism": "family default (injection/output_encoding/deserialization source_reaches_sink, path path_join_user_input, access_control missing_auth_guard, untrusted_destination validation_strength for CWE-601 else source_reaches_sink); provenance = agent"
+ },
+ "screened": {
+  "advisories_examined_second_pass": 2983,
+  "outcomes_second_pass": {
+   "no fix commit linked": 1247,
+   "repository already in benchmarks/ (populations v1/v2 or a pair corpus)": 520,
+   "language outside the feature vocabulary (go/ruby)": 332,
+   "CWEs span several families": 253,
+   "repository already kept (one pair per repository)": 140,
+   "licence not permissive or unrecorded": 135,
+   "large commit (refactor or release, not the fix)": 125,
+   "kept": 110,
+   "no function declared on both sides with a changed body": 51,
+   "fix spread over several commits or repositories": 25,
+   "manual review: the chosen function does not hold the fix (comment-only, misattributed, partial or family mismatch)": 13,
+   "no source file changed": 11,
+   "merge or root commit": 11,
+   "language outside the feature vocabulary (go/ruby/c#/...)": 9,
+   "WordPress plugin (population v3 safety)": 1
+  },
+  "candidates_manually_reviewed": 145,
+  "kept_after_review": 100,
+  "review_drops": {
+   "total": 45,
+   "function misattributed (matcher span overrun or an undetected enclosing declaration)": 13,
+   "change in the function unrelated to the advisory (refactor, rename, formatting, another advisory)": 11,
+   "partial: the function only calls or configures the real fix": 7,
+   "comment or documentation change only": 6,
+   "one pair per repository (a reviewed pair of the same repository kept)": 4,
+   "second pass chose another function of an advisory already reviewed": 2,
+   "family mismatch": 1,
+   "unsure": 1
+  },
+  "reserved_population_drops": 2,
+  "kept": 98
+ },
+ "kept": {
+  "total": 98,
+  "by_family": {
+   "access_control": 17,
+   "deserialization": 11,
+   "injection": 20,
+   "output_encoding": 16,
+   "path": 17,
+   "untrusted_destination": 17
+  },
+  "post_cutoff_by_family": {
+   "access_control": 13,
+   "deserialization": 2,
+   "injection": 20,
+   "output_encoding": 16,
+   "path": 17,
+   "untrusted_destination": 11
+  },
+  "post_cutoff_total": 79,
+  "by_language": {
+   "typescript": 34,
+   "python": 30,
+   "javascript": 15,
+   "java": 10,
+   "php": 9
+  },
+  "by_ecosystem": {
+   "npm": 48,
+   "pip": 32,
+   "maven": 9,
+   "composer": 9
+  },
+  "licences": {
+   "MIT": 67,
+   "Apache-2.0": 24,
+   "BSD-3-Clause": 5,
+   "ISC": 1,
+   "Unlicense": 1
+  },
+  "split": {
+   "train": 51,
+   "holdout": 47
+  },
+  "published_range": [
+   "2025-06-06",
+   "2026-10-02"
+  ],
+  "verification": {
+   "verified": 98,
+   "excerpt_bytes_read": 526134
+  },
+  "known_limit": "excerpts inherit the label builder's matcher: some spans run past the function's end into the next declaration; the label is path::function and the changed lines were checked inside the span"
+ },
+ "labels": {
+  "command": "ousast learn labels --root . --out <scratch> --snapshot <scratch>",
+  "rows": [
+   6589,
+   6981
+  ],
+  "new_pair_rows": 392,
+  "new_pin_rows_by_label": {
+   "positive": 98,
+   "negative": 98
+  },
+  "fixed_side_negative": "98 of 98 pairs (absent_fixed_side 0)"
+ },
+ "features": {
+  "command": "benchmarks/learn/build_features.py --labels <the 392 new rows> --units <units.json of `ousast plane harvest --catalog benchmarks/pairs/advisory-fixes-2/catalog.toml` (manifests written, no Run started)> --kinds pairs",
+  "records": 392,
+  "instrument_states_plane_all_families": {
+   "agree:missing": 74,
+   "agree:none": 122,
+   "delta:not_applicable": 196,
+   "engine:missing": 128,
+   "engine:none": 68,
+   "entry_points:not_applicable": 196,
+   "facts:not_applicable": 196,
+   "language:ran": 196,
+   "model_sinks:missing": 196,
+   "quick:ran": 196,
+   "roles:ran": 196,
+   "source:ran": 196,
+   "verify:missing": 74,
+   "verify:none": 122
+  },
+  "note": "quick rules and deterministic roles ran on each side; engine, verify, agree and model sinks never ran for these pairs (missing/none: null features, never zeros)"
+ },
+ "memory_build": {
+  "command": "ousast learn memory build --labels <labels> --units <merged units.json> --profile static --root . (exit 2: adjudications 9 of 43 anonymous functions, unchanged since 2026-10-01; rows kept)",
+  "examples": [
+   937,
+   1133
+  ],
+  "pairs_examples": [
+   772,
+   968
+  ],
+  "pairs_counts_after": {
+   "absent_fixed_side": 0,
+   "examples": 968,
+   "labels": 1936,
+   "no_excerpt": 0,
+   "no_record": 968,
+   "recorded": 968,
+   "unread": 0
+  }
+ },
+ "positives_only_groups": {
+  "how": "the 2026-10-02 gap record's count: static example rows of the store, per family, repository groups with a positive and no negative",
+  "before": 99,
+  "after": 99,
+  "before_note": "the store before this harvest held 937 examples (the 10-02 gap record counted 904 and 98; the 10-02 rebuild added one population-v2 positive without a negative)",
+  "why_unchanged": "the 99 are data gaps of other corpora (94 vibe-py pairs whose fixed side lost the function, 3 dev-php, 2 population-v2); a new repository group arrives with both labels, so it adds paired groups and leaves those counts as they were",
+  "paired_groups_total": [
+   151,
+   249
+  ],
+  "positives_only_share_of_positive_groups_six_families": [
+   0.4034,
+   0.2857
+  ],
+  "by_family_before_after": {
+   "access_control": {
+    "groups_pos_only": [
+     12,
+     12
+    ],
+    "groups_pos": [
+     32,
+     49
+    ],
+    "groups_both": [
+     20,
+     37
+    ],
+    "examples_neg": [
+     20,
+     37
+    ]
+   },
+   "config_secrets": {
+    "groups_pos_only": [
+     3,
+     3
+    ],
+    "groups_pos": [
+     5,
+     5
+    ],
+    "groups_both": [
+     2,
+     2
+    ],
+    "examples_neg": [
+     13,
+     13
+    ]
+   },
+   "deserialization": {
+    "groups_pos_only": [
+     20,
+     20
+    ],
+    "groups_pos": [
+     41,
+     52
+    ],
+    "groups_both": [
+     21,
+     32
+    ],
+    "examples_neg": [
+     21,
+     32
+    ]
+   },
+   "injection": {
+    "groups_pos_only": [
+     11,
+     11
+    ],
+    "groups_pos": [
+     44,
+     64
+    ],
+    "groups_both": [
+     33,
+     53
+    ],
+    "examples_neg": [
+     70,
+     90
+    ]
+   },
+   "memory": {
+    "groups_pos_only": [
+     0,
+     0
+    ],
+    "groups_pos": [
+     5,
+     5
+    ],
+    "groups_both": [
+     5,
+     5
+    ],
+    "examples_neg": [
+     173,
+     173
+    ]
+   },
+   "output_encoding": {
+    "groups_pos_only": [
+     14,
+     14
+    ],
+    "groups_pos": [
+     36,
+     52
+    ],
+    "groups_both": [
+     22,
+     38
+    ],
+    "examples_neg": [
+     61,
+     77
+    ]
+   },
+   "path": {
+    "groups_pos_only": [
+     12,
+     12
+    ],
+    "groups_pos": [
+     33,
+     50
+    ],
+    "groups_both": [
+     21,
+     38
+    ],
+    "examples_neg": [
+     21,
+     38
+    ]
+   },
+   "unknown": {
+    "groups_pos_only": [
+     0,
+     0
+    ],
+    "groups_pos": [
+     2,
+     2
+    ],
+    "groups_both": [
+     2,
+     2
+    ],
+    "examples_neg": [
+     2,
+     2
+    ]
+   },
+   "untrusted_destination": {
+    "groups_pos_only": [
+     27,
+     27
+    ],
+    "groups_pos": [
+     52,
+     69
+    ],
+    "groups_both": [
+     25,
+     42
+    ],
+    "examples_neg": [
+     36,
+     53
+    ]
+   }
+  }
+ },
+ "embeddings": {
+  "command": "ousast learn memory embed --budget-usd 0.20 (dry run first: 66,005 estimated tokens, $0.0013)",
+  "requested": 1004,
+  "cached": 781,
+  "embedded": 223,
+  "calls": 7,
+  "tokens": 71163,
+  "usd": 0.001423,
+  "note": "embedded = the new pairs' excerpts plus the excerpts the 10-02 rebuild left without a vector; OpenRouter, the embedding model .env names"
+ },
+ "spend": {
+  "chat_model_calls": 0,
+  "embedding_usd": 0.001423,
+  "github_api": "metadata and raw contents only"
+ }
+}

benchmarks/measurements/2026-10-04-decision-engine-harvest-3/record.json

@@ -0,0 +1,326 @@
+{
+ "created": "2026-10-04",
+ "note": "COUNTS ONLY. Batch 3 of real (vulnerable function, fixed function) pairs whose function survives the fix, mined at zero model cost from reviewed GitHub advisories (pip, npm, maven, composer) and added as pointer-pair corpus benchmarks/pairs/advisory-fixes-3 (source `advisory-fixes-3` in learn/sources.toml); labelled, given static feature records, built into the memory store and embedded. No chat-model call, no Docker, no Joern. Candidate, path, function and repository names stay in the catalog; the pairs the reserved-population guard flagged are withheld (found by bisection with the test output discarded; names never printed).",
+ "method": {
+  "enumeration": "gh api /advisories?type=reviewed&ecosystem=<eco>&cwes=<family CWEs>&published=>2023-01-01 per ecosystem (pip, npm, maven, composer) x six compiled families; Go, Ruby and C# not enumerated; newest first",
+  "advisories_enumerated": 8531,
+  "advisories_published_after_2026_06_01": 1855,
+  "automated_screen": "as batch 2 (single family by CWE; linked fix commit, at most 3, one repository; repository not named anywhere under benchmarks/ incl. batches 1-2; excluded v3 file names never opened; no WordPress plugin; one pair per repository; SPDX licence permissive; one-parent commit; the function holding most changed code lines inside its own matcher span, declared once on both sides, body changed, >= 25% of the file's changed code lines) plus three widenings: (a) an advisory linking only a merged pull request uses its single commit or its squash/rebase merge commit; (b) commits of up to 6 non-test source files / 240 changed lines (batch 2: 4 / 120); (c) a second pass over advisories whose CWEs span two families, the family then chosen from the function's diff on review",
+  "file_reads": "commit metadata and the two versions of each changed source file through the contents API into scratch (no clone, no scanner, engine, plane Run or model on these repositories); scratch deleted after",
+  "manual_review": "every automated candidate's in-function diff read (agent, no model call); kept only when the named function itself holds the fix; a weak inline check replaced by a stronger helper call counts as the fix, a function that only forwards a flag or calls a fix made elsewhere does not",
+  "reserved_population_guard": "pytest -q tests/ -k 'reserved or v3' with output discarded, bisected over catalog subsets (18 runs; baseline with an empty batch green); final catalog green",
+  "rename_check": "canonical full_name of every kept repository from the API: no two kept pairs share a repository, none renamed into a name the benchmarks use",
+  "mechanism": "family default as batch 2 (injection/output_encoding/deserialization source_reaches_sink, path path_join_user_input, access_control missing_auth_guard, untrusted_destination validation_strength for CWE-601 else source_reaches_sink); provenance = agent"
+ },
+ "screened": {
+  "advisories_examined": 8805,
+  "outcomes": {
+   "no fix commit linked": 3480,
+   "repository already in benchmarks/ (populations v1/v2 or a pair corpus)": 1993,
+   "licence not permissive or unrecorded": 1249,
+   "repository already kept (one pair per repository)": 657,
+   "kept": 398,
+   "CWEs span several families": 274,
+   "no function declared on both sides with a changed body": 191,
+   "large commit (refactor or release, not the fix)": 173,
+   "fix spread over several commits or repositories": 167,
+   "merge or root commit": 115,
+   "no source file changed": 42,
+   "language outside the feature vocabulary (go/ruby/c#/...)": 32,
+   "manual review: the chosen function does not hold the fix (comment-only, misattributed, partial or family mismatch)": 19,
+   "repository unavailable": 9,
+   "WordPress plugin (population v3 safety)": 4,
+   "fix commit unavailable": 2
+  },
+  "outcomes_note": "single-family pass (screen 2, every advisory examined, no quota met) plus the two-family pass; `kept` = automated candidates; `manual review: the chosen function does not hold the fix (comment-only, misattributed, partial or family mismatch)` = the screen re-meeting a function the 2026-10-03 review had dropped",
+  "candidates_manually_reviewed": 412,
+  "kept_after_review": 255,
+  "review_drops": {
+   "total": 157,
+   "change in the function unrelated to the advisory (refactor, rename, formatting, another advisory)": 33,
+   "function misattributed (matcher span overrun or an undetected enclosing declaration)": 33,
+   "partial: the function only calls or configures the real fix": 29,
+   "function diff not small (rewrite or refactor around the fix)": 28,
+   "unsure": 17,
+   "family mismatch": 11,
+   "same fix as a kept pair in a fork or sibling repository (would leak across repository folds)": 3,
+   "WordPress plugin (population v3 safety; missed by the automated screen)": 2,
+   "reserved-population guard (flagged by the 2026-10-03 guard run; name not printed)": 1
+  },
+  "one_pair_per_repository_drops": 3,
+  "reserved_population_drops": 2,
+  "kept": 251
+ },
+ "kept": {
+  "total": 251,
+  "by_family": {
+   "access_control": 20,
+   "deserialization": 9,
+   "injection": 56,
+   "output_encoding": 69,
+   "path": 71,
+   "untrusted_destination": 26
+  },
+  "post_cutoff_by_family": {
+   "injection": 5,
+   "output_encoding": 5,
+   "path": 2
+  },
+  "post_cutoff_total": 12,
+  "by_language": {
+   "java": 37,
+   "javascript": 37,
+   "php": 46,
+   "python": 83,
+   "typescript": 48
+  },
+  "post_cutoff_by_language": {
+   "python": 6,
+   "typescript": 6
+  },
+  "by_family_language": {
+   "access_control:java": 5,
+   "access_control:javascript": 3,
+   "access_control:php": 7,
+   "access_control:python": 2,
+   "access_control:typescript": 3,
+   "deserialization:java": 2,
+   "deserialization:php": 1,
+   "deserialization:python": 6,
+   "injection:java": 5,
+   "injection:javascript": 7,
+   "injection:php": 9,
+   "injection:python": 24,
+   "injection:typescript": 11,
+   "output_encoding:java": 7,
+   "output_encoding:javascript": 20,
+   "output_encoding:php": 12,
+   "output_encoding:python": 14,
+   "output_encoding:typescript": 16,
+   "path:java": 10,
+   "path:javascript": 4,
+   "path:php": 10,
+   "path:python": 34,
+   "path:typescript": 13,
+   "untrusted_destination:java": 8,
+   "untrusted_destination:javascript": 3,
+   "untrusted_destination:php": 7,
+   "untrusted_destination:python": 3,
+   "untrusted_destination:typescript": 5
+  },
+  "by_ecosystem": {
+   "composer": 46,
+   "maven": 37,
+   "npm": 81,
+   "pip": 87
+  },
+  "licences": {
+   "Apache-2.0": 80,
+   "BSD-2-Clause": 3,
+   "BSD-3-Clause": 18,
+   "ISC": 1,
+   "MIT": 147,
+   "MPL-2.0": 1,
+   "Unlicense": 1
+  },
+  "split": {
+   "holdout": 124,
+   "train": 127
+  },
+  "published_by_year": {
+   "2023": 50,
+   "2024": 52,
+   "2025": 66,
+   "2026": 83
+  },
+  "published_range": [
+   "2023-01-04",
+   "2026-10-01"
+  ],
+  "via_pull_request": 13,
+  "commit_over_120_source_lines": 10,
+  "two_family_advisory_family_chosen_on_review": 3,
+  "verification": {
+   "verified": 251,
+   "excerpt_bytes_read": 1297516
+  },
+  "targets": {
+   "asked": "injection and untrusted_destination >= 60 each; path, access_control, output_encoding, deserialization >= 30 where advisories allow; >= 250 total",
+   "met": {
+    "access_control": false,
+    "deserialization": false,
+    "injection": false,
+    "output_encoding": true,
+    "path": true,
+    "untrusted_destination": false
+   },
+   "limit": "the reviewed-advisory pool, not the review: every advisory since 2023-01-01 in the four ecosystems was examined; the post-cutoff pool was largely spent by batch 2 (its repositories are now excluded), and untrusted_destination/access_control/deserialization advisories mostly lack a linked single fix commit or sit in repositories the benchmarks already use"
+  }
+ },
+ "labels": {
+  "command": "ousast learn labels --root . --out <scratch> --snapshot <scratch>",
+  "rows": [
+   6981,
+   7985
+  ],
+  "pairs_rows": [
+   1936,
+   2940
+  ],
+  "new_pair_rows": 1004
+ },
+ "features": {
+  "command": "benchmarks/learn/build_features.py --labels <the 1004 new rows> --units <merged units.json of `ousast plane harvest --catalog benchmarks/pairs/advisory-fixes-3/catalog.toml` (manifests written, no Run started)> --catalog benchmarks/pairs/advisory-fixes-3/catalog.toml --kinds pairs",
+  "records": {
+   "plane:access_control": 40,
+   "plane:deserialization": 18,
+   "plane:injection": 112,
+   "plane:output_encoding": 138,
+   "plane:path": 142,
+   "plane:untrusted_destination": 52,
+   "static:access_control": 40,
+   "static:deserialization": 18,
+   "static:injection": 112,
+   "static:output_encoding": 138,
+   "static:path": 142,
+   "static:untrusted_destination": 52
+  },
+  "instrument_states_plane_all_families": {
+   "agree:missing": 164,
+   "agree:none": 338,
+   "delta:not_applicable": 502,
+   "engine:missing": 406,
+   "engine:none": 96,
+   "entry_points:not_applicable": 502,
+   "facts:not_applicable": 502,
+   "language:ran": 502,
+   "model_sinks:missing": 502,
+   "quick:ran": 502,
+   "roles:ran": 502,
+   "source:ran": 502,
+   "verify:missing": 164,
+   "verify:none": 338
+  },
+  "note": "quick rules and deterministic roles ran on each side; engine, verify, agree and model sinks never ran for these pairs (missing/none: null features, never zeros)"
+ },
+ "memory_build": {
+  "command": "ousast learn memory build --labels <labels> --units <merged units.json> --profile static --root . (exit 2: adjudications 9 of 43 anonymous functions, unchanged since 2026-10-01; rows kept)",
+  "examples": [
+   1133,
+   1635
+  ],
+  "pairs_examples": [
+   968,
+   1470
+  ],
+  "pairs_counts_after": {
+   "absent_fixed_side": 0,
+   "examples": 1470,
+   "labels": 2940,
+   "no_excerpt": 0,
+   "no_record": 1470,
+   "recorded": 1470,
+   "unread": 0
+  }
+ },
+ "complete_pairs": {
+  "how": "store example rows (profile static) keyed as learn.experiments.freeze_units pairs them: (repo, candidate, family) holding both labels; whole memory, every family",
+  "before": 459,
+  "after": 710,
+  "by_family": {
+   "access_control": [
+    37,
+    57
+   ],
+   "config_secrets": [
+    12,
+    12
+   ],
+   "deserialization": [
+    32,
+    41
+   ],
+   "injection": [
+    68,
+    124
+   ],
+   "memory": [
+    149,
+    149
+   ],
+   "output_encoding": [
+    77,
+    146
+   ],
+   "path": [
+    38,
+    109
+   ],
+   "unknown": [
+    2,
+    2
+   ],
+   "untrusted_destination": [
+    44,
+    70
+   ]
+  },
+  "exp_004_005_units": "195 complete pairs inside exp-004/005's frozen 501 units (six families, evaluation folds); those units are frozen and were not regenerated here"
+ },
+ "positives_only_groups": {
+  "before": 99,
+  "after": 99,
+  "groups_both_by_family": {
+   "access_control": [
+    37,
+    57
+   ],
+   "config_secrets": [
+    2,
+    2
+   ],
+   "deserialization": [
+    32,
+    41
+   ],
+   "injection": [
+    53,
+    109
+   ],
+   "memory": [
+    5,
+    5
+   ],
+   "output_encoding": [
+    38,
+    107
+   ],
+   "path": [
+    38,
+    109
+   ],
+   "unknown": [
+    2,
+    2
+   ],
+   "untrusted_destination": [
+    42,
+    68
+   ]
+  }
+ },
+ "embeddings": {
+  "command": "ousast learn memory embed --budget-usd 0.20 (dry run first: 144,806 estimated tokens, $0.0029)",
+  "requested": 1479,
+  "cached": 1004,
+  "embedded": 475,
+  "calls": 15,
+  "tokens": 150339,
+  "usd": 0.003007
+ },
+ "spend": {
+  "chat_model_calls": 0,
+  "embedding_usd": 0.003007,
+  "github_api": "metadata and raw contents only"
+ }
+}

benchmarks/measurements/2026-10-04-unseen-01-used-set/record.json

@@ -0,0 +1,86 @@
+{
+  "count": 1029,
+  "guard": "green",
+  "sha256": "5a644a1e93c68a0a35c794e653286d0532b7eb8d757123ccce5f08b11bf9a4a2",
+  "sources": {
+    "harvest": {
+      "bytes": 501031,
+      "count": 467,
+      "files": 3,
+      "known_empty": false,
+      "rows": 446,
+      "skipped_values": 446
+    },
+    "measurements": {
+      "bytes": 21169903,
+      "count": 278,
+      "files": 221,
+      "known_empty": false,
+      "rows": 376,
+      "skipped_values": 681
+    },
+    "memory_local": {
+      "bytes": 8084523,
+      "count": 192,
+      "files": 776,
+      "known_empty": false,
+      "rows": 759,
+      "skipped_values": 2
+    },
+    "memory_s3": {
+      "bytes": 15928676,
+      "count": 572,
+      "files": 1499,
+      "known_empty": false,
+      "rows": 1490,
+      "skipped_values": 2
+    },
+    "pairs": {
+      "bytes": 1680480,
+      "count": 728,
+      "files": 28,
+      "known_empty": false,
+      "rows": 900,
+      "skipped_values": 1260
+    },
+    "pointers": {
+      "bytes": 892448,
+      "count": 537,
+      "files": 7,
+      "known_empty": false,
+      "rows": 586,
+      "skipped_values": 666
+    },
+    "populations": {
+      "bytes": 32431,
+      "count": 28,
+      "files": 2,
+      "known_empty": false,
+      "rows": 28,
+      "skipped_values": 0
+    },
+    "sweep": {
+      "bytes": 34006439,
+      "count": 1029,
+      "files": 1619,
+      "known_empty": false,
+      "rows": 2155,
+      "skipped_values": 0
+    },
+    "units": {
+      "bytes": 319641,
+      "count": 229,
+      "files": 9,
+      "known_empty": false,
+      "rows": 642,
+      "skipped_values": 0
+    }
+  },
+  "test_exit_code": 0,
+  "tests": {
+    "errors": 0,
+    "failures": 0,
+    "skipped": 1,
+    "tests": 41
+  }
+}

benchmarks/measurements/2026-10-04-unseen-06-scorer/record.json

@@ -0,0 +1,41 @@
+{
+  "commit": "48ae918dec25119142ae781607ba99306ae5ec17",
+  "fixture_bytes": 15374,
+  "fixture_sha256": "dd3c774a47d553105ed85b9cea8810736c233f870e5a015559a1524d5db5a14d",
+  "implementation_sha256": {
+    "benchmarks/ax/batch.py": "37aecac93beb25c02826e81eacaf064fbebb447a998fe139f6d94a07035b8685",
+    "benchmarks/unseen/replay.py": "32efb3e0b9e9d247f2780b5f3ceeb01b4c2cbda3656c75a741f9925621cfdfbc",
+    "benchmarks/unseen/replay_entry.py": "994d8e239eb505dc2c5bad4cdb8b5d45967a8a2aa9ef48fb4c2f20ba45736280",
+    "benchmarks/unseen/score.py": "56e0a7b34482b5d84df39824b8ee447c283e35bd932f38cfe692cf84cb52cef7"
+  },
+  "network_proofs": {
+    "4.4": "coordinator_pending",
+    "5.4": "coordinator_pending"
+  },
+  "scope": "offline synthetic scorer only",
+  "score_command_exit": 0,
+  "scorer_output_sha256": "77f154eedb6a4acd098ab63a0b17434291f0d226dabe8ac66f2e0b362154d002",
+  "scorer_test_exit": 0,
+  "scorer_tests": 14,
+  "synthetic_records": 20,
+  "uncommitted": true,
+  "validation": {
+    "generic_dispatcher_tests": {
+      "exit_code": 0,
+      "passed": 5
+    },
+    "mypy": 0,
+    "proof_and_scorer_tests": {
+      "exit_code": 0,
+      "passed": 16
+    },
+    "requested_pytest": {
+      "exit_code": 0,
+      "passed": 279,
+      "skipped": 8
+    },
+    "reserved_repository_guard": 0,
+    "ruff_check": 0,
+    "ruff_format_check": 0
+  }
+}

benchmarks/pairs/advisory-fixes-2/verification.json

@@ -0,0 +1,496 @@
+{
+  "pairs": [
+    {
+      "name": "prefect-token-validation-e216171",
+      "ok": true,
+      "bytes_read": 3697
+    },
+    {
+      "name": "arcadedb-getdatabasegroupsconfiguration-04110c0",
+      "ok": true,
+      "bytes_read": 3896
+    },
+    {
+      "name": "langflow-get-flow-by-id-or-endpoint-name-2c9f498",
+      "ok": true,
+      "bytes_read": 3948
+    },
+    {
+      "name": "grav-plugin-api-update-26f529c",
+      "ok": true,
+      "bytes_read": 5598
+    },
+    {
+      "name": "google-maps-mcp-main-00d8725",
+      "ok": true,
+      "bytes_read": 3371
+    },
+    {
+      "name": "camofox-mcp-createmcphttpapp-599f56e",
+      "ok": true,
+      "bytes_read": 5587
+    },
+    {
+      "name": "spring-graphql-isoverridefor-63a89ed",
+      "ok": true,
+      "bytes_read": 2045
+    },
+    {
+      "name": "jenkins-dofilltimezonenameitems-0586de4",
+      "ok": true,
+      "bytes_read": 2093
+    },
+    {
+      "name": "netlicensing-mcp-dispatch-fbbb1d5",
+      "ok": true,
+      "bytes_read": 2610
+    },
+    {
+      "name": "starlette-admin-validate-where-af05b45",
+      "ok": true,
+      "bytes_read": 2061
+    },
+    {
+      "name": "next-auth-initauth-d008b9b",
+      "ok": true,
+      "bytes_read": 8193
+    },
+    {
+      "name": "deepseek-mcp-server-starthttptransport-dab07ed",
+      "ok": true,
+      "bytes_read": 2394
+    },
+    {
+      "name": "network-ai-routerequest-a59c13a",
+      "ok": true,
+      "bytes_read": 5945
+    },
+    {
+      "name": "device-builder-parse-args-9e294f7",
+      "ok": true,
+      "bytes_read": 15243
+    },
+    {
+      "name": "typespec-start-30d6f65",
+      "ok": true,
+      "bytes_read": 2109
+    },
+    {
+      "name": "tomcat-findsecurityconstraints-8bafd79",
+      "ok": true,
+      "bytes_read": 18199
+    },
+    {
+      "name": "vm2-makebuiltinsfromlegacyoptions-b0f5066",
+      "ok": true,
+      "bytes_read": 3539
+    },
+    {
+      "name": "inlong-filtersensitive-ebcd6d3",
+      "ok": true,
+      "bytes_read": 6474
+    },
+    {
+      "name": "h2o-3-validatejdbcurl-0298ee3",
+      "ok": true,
+      "bytes_read": 4919
+    },
+    {
+      "name": "skops-load-model-29d61ea",
+      "ok": true,
+      "bytes_read": 1847
+    },
+    {
+      "name": "picklescan-list-globals-58983e1",
+      "ok": true,
+      "bytes_read": 6975
+    },
+    {
+      "name": "sglang-deserialize-49afb3d",
+      "ok": true,
+      "bytes_read": 4334
+    },
+    {
+      "name": "pdfminer-six-load-data-b808ee0",
+      "ok": true,
+      "bytes_read": 2399
+    },
+    {
+      "name": "reverb-handle-9ec26f8",
+      "ok": true,
+      "bytes_read": 3342
+    },
+    {
+      "name": "transformers-is-kernel-a7f8e7f",
+      "ok": true,
+      "bytes_read": 1921
+    },
+    {
+      "name": "opentelemetry-java-instrumentation-read-9cf4fba",
+      "ok": true,
+      "bytes_read": 2075
+    },
+    {
+      "name": "keras-from-config-55888d3",
+      "ok": true,
+      "bytes_read": 3154
+    },
+    {
+      "name": "airflow-handle-event-submit-a1ab065",
+      "ok": true,
+      "bytes_read": 4828
+    },
+    {
+      "name": "sequelize-partialescape-5deadd2",
+      "ok": true,
+      "bytes_read": 6826
+    },
+    {
+      "name": "electerm-mv-aa77881",
+      "ok": true,
+      "bytes_read": 1258
+    },
+    {
+      "name": "systeminformation-checklinuxdchpinterfaces-bbfddde",
+      "ok": true,
+      "bytes_read": 2208
+    },
+    {
+      "name": "prompty-buildagent-c27402d",
+      "ok": true,
+      "bytes_read": 3198
+    },
+    {
+      "name": "datamodel-code-generator-get-field-extras-17fc235",
+      "ok": true,
+      "bytes_read": 2471
+    },
+    {
+      "name": "spring-data-mongodb-bindablevaluefor-24d5afc",
+      "ok": true,
+      "bytes_read": 9895
+    },
+    {
+      "name": "spring-ai-emitlucenestring-2b20cfc",
+      "ok": true,
+      "bytes_read": 4037
+    },
+    {
+      "name": "aws-cdk-write-baa9e1d",
+      "ok": true,
+      "bytes_read": 1461
+    },
+    {
+      "name": "danger-js-localgetfileatsha-087a729",
+      "ok": true,
+      "bytes_read": 1724
+    },
+    {
+      "name": "argos-javascript-gitfetch-8355f3a",
+      "ok": true,
+      "bytes_read": 1319
+    },
+    {
+      "name": "cakephp-transformfunctionexpression-138f2f6",
+      "ok": true,
+      "bytes_read": 6754
+    },
+    {
+      "name": "orval-getroutepath-8ef1bfd",
+      "ok": true,
+      "bytes_read": 2607
+    },
+    {
+      "name": "typeorm-handler-41d1c62",
+      "ok": true,
+      "bytes_read": 11977
+    },
+    {
+      "name": "velocity-js-isblockedpathkey-f8e47a6",
+      "ok": true,
+      "bytes_read": 1539
+    },
+    {
+      "name": "jsonata-evaluategroupexpression-4b217d5",
+      "ok": true,
+      "bytes_read": 7399
+    },
+    {
+      "name": "nuxt-getislandcontext-5b60017",
+      "ok": true,
+      "bytes_read": 6805
+    },
+    {
+      "name": "gitpython-option-candidates-96a888f",
+      "ok": true,
+      "bytes_read": 3421
+    },
+    {
+      "name": "mermaid-addnamespace-12d472c",
+      "ok": true,
+      "bytes_read": 9722
+    },
+    {
+      "name": "php-codesniffer-getblamecontent-7a3a6bb",
+      "ok": true,
+      "bytes_read": 2040
+    },
+    {
+      "name": "xmldom-serializetostring-1cde3e3",
+      "ok": true,
+      "bytes_read": 23297
+    },
+    {
+      "name": "trix-fromjson-9c0a993",
+      "ok": true,
+      "bytes_read": 1295
+    },
+    {
+      "name": "oauthlib-prepare-token-revocation-request-c888359",
+      "ok": true,
+      "bytes_read": 4800
+    },
+    {
+      "name": "suneditor-submitsrc-9d43a5e",
+      "ok": true,
+      "bytes_read": 2319
+    },
+    {
+      "name": "mathlive-atomtomathml-5fe1c46",
+      "ok": true,
+      "bytes_read": 32529
+    },
+    {
+      "name": "genql-comment-2dbc80f",
+      "ok": true,
+      "bytes_read": 1633
+    },
+    {
+      "name": "angular-visitelementlike-417a407",
+      "ok": true,
+      "bytes_read": 8480
+    },
+    {
+      "name": "cyberchef-objtotable-85db3be",
+      "ok": true,
+      "bytes_read": 2556
+    },
+    {
+      "name": "mdc-isanchorlinkallowed-61d636c",
+      "ok": true,
+      "bytes_read": 1795
+    },
+    {
+      "name": "node-xhtml-purifier-node-21d461a",
+      "ok": true,
+      "bytes_read": 7486
+    },
+    {
+      "name": "orpc-init-daabded",
+      "ok": true,
+      "bytes_read": 6240
+    },
+    {
+      "name": "mkdocs-material-mountsearchsuggest-52fb6be",
+      "ok": true,
+      "bytes_read": 4493
+    },
+    {
+      "name": "jodit-isremovablenode-49a31f4",
+      "ok": true,
+      "bytes_read": 3921
+    },
+    {
+      "name": "prowler-transform-6db407e",
+      "ok": true,
+      "bytes_read": 5481
+    },
+    {
+      "name": "commonmark-parseattributes-dfcdf45",
+      "ok": true,
+      "bytes_read": 6437
+    },
+    {
+      "name": "svgo-fn-3db3ef3",
+      "ok": true,
+      "bytes_read": 6895
+    },
+    {
+      "name": "tiptap-mergeattributes-01d7af8",
+      "ok": true,
+      "bytes_read": 3690
+    },
+    {
+      "name": "nogginlessdom-matchfilesnapshot-785e6ac",
+      "ok": true,
+      "bytes_read": 3023
+    },
+    {
+      "name": "gradio-preprocess-97d541f",
+      "ok": true,
+      "bytes_read": 2581
+    },
+    {
+      "name": "gemini-bridge-resolve-path-8f3b85a",
+      "ok": true,
+      "bytes_read": 2006
+    },
+    {
+      "name": "libreoffice-convert-convertwithoptions-b78f17d",
+      "ok": true,
+      "bytes_read": 8912
+    },
+    {
+      "name": "atomic-agents-stack-do-get-ec474f4",
+      "ok": true,
+      "bytes_read": 3493
+    },
+    {
+      "name": "browse-mcp-downloadurl-5352a4a",
+      "ok": true,
+      "bytes_read": 5302
+    },
+    {
+      "name": "notebooklm-mcp-makeslug-13828d9",
+      "ok": true,
+      "bytes_read": 2103
+    },
+    {
+      "name": "langchain-nvidia-url-to-b64-string-b483c7c",
+      "ok": true,
+      "bytes_read": 2318
+    },
+    {
+      "name": "redocly-cli-iteratepathitems-5041204",
+      "ok": true,
+      "bytes_read": 4412
+    },
+    {
+      "name": "pip-filename-10dfb6b",
+      "ok": true,
+      "bytes_read": 1419
+    },
+    {
+      "name": "lib-get-db-path-d665042",
+      "ok": true,
+      "bytes_read": 3879
+    },
+    {
+      "name": "blender-mcp-download-polyhaven-asset-30a3308",
+      "ok": true,
+      "bytes_read": 35668
+    },
+    {
+      "name": "datasets-set-feature-f989ef9",
+      "ok": true,
+      "bytes_read": 4882
+    },
+    {
+      "name": "postcss-loadfile-7beca13",
+      "ok": true,
+      "bytes_read": 1712
+    },
+    {
+      "name": "vitest-configureserver-51edf2b",
+      "ok": true,
+      "bytes_read": 3287
+    },
+    {
+      "name": "next-js-getfilepath-968b9fc",
+      "ok": true,
+      "bytes_read": 2462
+    },
+    {
+      "name": "composer-install-53b8bb4",
+      "ok": true,
+      "bytes_read": 4127
+    },
+    {
+      "name": "vue-cli-mayproxy-72ba750",
+      "ok": true,
+      "bytes_read": 1203
+    },
+    {
+      "name": "h3-redirectback-459a1c6",
+      "ok": true,
+      "bytes_read": 2750
+    },
+    {
+      "name": "axios-setproxy-fb3befb",
+      "ok": true,
+      "bytes_read": 4403
+    },
+    {
+      "name": "rembg-s-command-07ad0d4",
+      "ok": true,
+      "bytes_read": 21204
+    },
+    {
+      "name": "http-server-getpreviousurl-2008fb6",
+      "ok": true,
+      "bytes_read": 4395
+    },
+    {
+      "name": "authlib-get-authorization-grant-3be0846",
+      "ok": true,
+      "bytes_read": 2631
+    },
+    {
+      "name": "java-client-overrideserverurl-2b9cd44",
+      "ok": true,
+      "bytes_read": 1382
+    },
+    {
+      "name": "n8n-mcp-validatewebhookurl-9639f75",
+      "ok": true,
+      "bytes_read": 13458
+    },
+    {
+      "name": "ufo-withoutbase-5cd9e67",
+      "ok": true,
+      "bytes_read": 2154
+    },
+    {
+      "name": "symfony-dogenerate-4b63c3a",
+      "ok": true,
+      "bytes_read": 16981
+    },
+    {
+      "name": "php-weasyprint-isoptionurl-9582dcf",
+      "ok": true,
+      "bytes_read": 1090
+    },
+    {
+      "name": "compliance-trestle-check-metadata-endpoints-d107cd1",
+      "ok": true,
+      "bytes_read": 2412
+    },
+    {
+      "name": "vllm-preprocess-audio-54503ec",
+      "ok": true,
+      "bytes_read": 4629
+    },
+    {
+      "name": "phpspreadsheet-webservice-7ef7b25",
+      "ok": true,
+      "bytes_read": 4441
+    },
+    {
+      "name": "frontmcp-dofetch-077201e",
+      "ok": true,
+      "bytes_read": 3737
+    },
+    {
+      "name": "ip-address-islinklocal-d03e960",
+      "ok": true,
+      "bytes_read": 1489
+    },
+    {
+      "name": "pyjwt-fetch-data-0a795b8",
+      "ok": true,
+      "bytes_read": 3385
+    }
+  ],
+  "verified": 98,
+  "bytes_read": 526134
+}

benchmarks/unseen/README.md

@@ -0,0 +1,312 @@
+# Unseen evaluation, group 1
+
+The coordinator runs task 1.5 from the repository root:
+
+```sh
+export PATH=/home/mc/Source/OpenUltraSAST/.venv/bin:$PATH
+export PYTHONPATH=$PWD/src
+python -m benchmarks.unseen.eligibility --s3-store s3:// --verify-tests --output benchmarks/measurements/2026-10-04-unseen-01-used-set/record.json
+```
+
+This is the networked command; implementation tests use fakes. It loads `.env` with
+`openultrasast.config.load_dotenv` (exported values win), opens the local FileStore and
+S3 MemoryStore through `openultrasast.plane.memory.open_store(..., read_only=True)`,
+and uses `GH_TOKEN` for GitHub GET resolution. `s3://` selects `S3_BUCKET`; pass
+`--s3-store s3://BUCKET/PREFIX` for a prefixed store and `--local-store file:///PATH`
+for a different local store. Read-only S3 opening skips the normal write/Select probe;
+collection uses LIST/GET only. Existing normal store opening is unchanged.
+
+Pair catalogs/recipes include local pointer rows (`vendored = false`, including
+`fix_repo`). Additional external manifests can be supplied with repeatable
+`--pointer-manifest PATH`. Registered units in memory and experiment YAML are followed;
+missing units with a registered digest fail. Plans without frozen units are not used data.
+Experiment units identify repositories in `group`. Binary measurement artifacts are
+read and counted; structured JSON, TOML and multi-document YAML are parsed strictly.
+
+Each source records files/objects read, bytes read, repository-bearing rows (distinct
+names per input), and distinct repository counts. The digest is SHA-256 of sorted,
+normalized names joined with newlines. Source counts overlap. An empty source fails;
+only after establishing it is empty may the coordinator pass `--known-empty LABEL`.
+The output records that declaration. Missing/unreadable inputs still fail.
+
+GitHub redirects add the requested and canonical names, plus parent/source fork
+identities. Used names are also resolved so an old used name excludes a renamed
+candidate. GitHub cannot enumerate all historical names: injected clients may supply
+`former_names`; actual historical aliases come from used references and redirects.
+A 404 for a historical/synthetic used identity retains its literal identity; other
+API failures stop collection. Candidate resolution always fails closed on a 404.
+`resolve(candidate, api)` and `rejection(resolution, used, selected_networks)` are
+available to the later sourcing task. Names stay in memory; output contains only
+counts, labels and SHA-256 digests. Output files are created exclusively, never overwritten.
+
+`--verify-tests` runs the three group-1 test modules with both output streams discarded,
+including the existing population guard. Only aggregate test counts and the exit code
+enter the record; a failed test prevents the record. V3 paths are excluded before
+eligibility reads; only the existing guard handles v3 inside its black-box test process.
+
+Population reservations retain their existing sweep over `benchmarks/`, `src/`, and
+`tests/`. Pool reservations also cover `plane/`. Each reservation exempts only its own
+directory; population checks include all of `benchmarks/unseen/private/`. This preserves
+the existing population behavior while adding pools (the hard rule for this increment).
+
+All `pool-pN.toml` files require a matching `freeze-pN.json`; drafts use `draft-pN.toml`.
+The freeze digest is SHA-256 of parsed public TOML encoded as UTF-8 JSON with sorted
+keys, compact separators and `ensure_ascii=False`. Private entries must be represented
+by opaque digests in that public manifest, as task 3.2 requires.
+
+The ledger exposes `append(path, row)` and `read(path)`. Rows contain `slice`, `decision`
+(an opaque decision or experiment label), `purpose`, ISO `date`, `result_digest`, and
+`freeze_digest`. Appends lock and validate the existing chain, reject duplicates or a
+changed freeze, and reject qualification of an informed slice for the same decision.
+Keep the published final digest to authenticate history: the chain detects row edits,
+but cannot independently prove that an entire history was not replaced or truncated.
+
+## Sourcing and extraction (tasks 2.1–2.4)
+
+The coordinator's networked task **2.5** is now:
+
+```sh
+export PATH=/home/mc/Source/OpenUltraSAST/.venv/bin:$PATH
+export PYTHONPATH=$PWD/src
+python -m benchmarks.unseen.draft --root "$PWD" --s3-store s3:// --seed 20260601 \
+  --cache /home/mc/ousast-results/plane/unseen-draft --max-hours 8
+```
+
+Export `GH_TOKEN` first, or supply it through the root `.env` (exported values win).
+The same read-only local/S3 stores, `--pointer-manifest`, and explicit `--known-empty`
+options as task 1.5 apply. Missing or unreadable sources fail. The used set is rebuilt
+and canonicalized before candidate selection, including redirects and fork networks.
+No population v3 file is opened and no guard/bisection is run by this command; that is
+reserved for group 3. An existing draft is never overwritten.
+
+The command writes `--cache/journal/draft-p1.toml` and
+`--cache/journal/private-draft-p1.toml` only after all sourcing stages complete.
+Both stay outside the repository, with mode 0600. The public-format file contains permissive entries
+and, for every private entry, only an opaque ID, slice, private marker, and SHA-256
+of canonical JSON containing its URL and ordered `[base, head]` pairs. Private entries
+carry the complete records, including licences, in the private-format file.
+Licensing is re-read through GitHub's licence endpoint **at the actual extracted
+vulnerability replay head**; the candidate's earlier fix-parent classification is
+provisional. An explicit SPDX declaration in the pinned file takes precedence over
+the API's classification. Missing/unknown/compound licences remain private.
+
+Three seeded slices each contain 100 repositories, one vulnerability range and twelve
+ordinary changes per repository. Each ecosystem/post-cutoff stratum differs from its
+proportional per-slice target by less than one repository. Post-cutoff candidates are
+attempted first within each ecosystem. Below 300 eligible repositories the command
+exits 1 with counts and does not write a partial draft or relax any selection bar.
+The group-2 measured exit and label confirmation remain coordinator tasks 2.5/2.6;
+passing implementation tests does not assert that 300 eligible repositories exist.
+
+Clones and checkpoints default to `results_root()/unseen-draft` (normally
+`~/ousast-results/plane/unseen-draft`), overridable by `--cache /outside/path`.
+The cache cannot lie inside `--root`. One opaque, disposable clone exists at a time;
+it is removed on success, rejection, or failure. `--filter=blob:none --no-checkout`
+keeps the commit graph needed for temporal thirds, blame, and release-tag ancestry,
+and fetches source blobs on demand. This is a **partial clone**, not a shallow history.
+The 500 MiB disk cap is checked after Git commands, including lazy blob fetches;
+one in-flight command can transiently exceed it before rejection and cleanup.
+Git output is captured, never forwarded. Each repository must read a nonempty source
+blob before extraction. Summary instrumentation includes source read operations and
+actual bytes, clone count, cumulative/peak disk bytes, wall time, and logical API calls
+and HTTP attempts. Zero source bytes or a failed Git read is an instrument failure,
+not an eligibility drop. Stdout is a JSON summary of counts/digests only; no names or
+URLs enter stdout or a measurement record. Raw per-change data stays in the external journal and manifests.
+
+Extraction uses Git and syntax boundaries, with no scanner, engine, or model. Python
+uses stdlib AST; Java/PHP/JS/TS use conservative named-function brace matching. Ambiguous
+or unparseable functions can reject a candidate. The first removed code line per
+fix-touched function is the SZZ sink proxy; pure-addition fixes use its declaration.
+Normalized text absence, non-root/non-merge ancestry, file/line limits, and the OSV
+release bracket are checked before accepting `introducing`. Matching tags are `VERSION`
+or `vVERSION`; an absent/ambiguous tag or missing prior release cannot certify the
+bracket. After at most five walk-backs, `last_touch` uses `git log -L` with the resolved
+numeric function extent (also works where Git lacks a language-specific function driver).
+Method, bias, fallback reasons, head function/range, files/hunks, family, advisory, and
+cutoff flag are recorded. Task 2.6 must review every third `introducing` label.
+
+All reviewed advisories are enumerated to exclude **all** known repository fixes from
+ordinary changes, including old advisories and advisories outside the four candidate
+ecosystems. Fix URLs are canonicalized before indexing so renames cannot hide them.
+The ordinary filter uses only the design's narrowed security pattern. It counts
+rejections (including oversized commits and repositories failing the 40-commit floor),
+checks both sides for touches to known vulnerable functions, and inspects generated
+headers and inherited `.gitattributes`. Sampling allocates twelve draws proportionally
+across three history-time thirds and four source-size buckets, with one per third.
+
+For offline tests, inject GitHub/OSV clients, the clone context, and `UsedSet` into
+`draft.run`; HTTP transports and Git subprocess runners also accept fakes. Tests use
+sanitized REST-shaped data and a temporary synthetic local Git history. No implementation
+test contacts GitHub, OSV, a model, or a cluster. HTTP retries are bounded to five attempts;
+rate-limit reset/Retry-After waits and transient 5xx backoff are tested with fake clocks
+and sleepers. A stale candidate 404 is counted; other service failures fail the run.
+
+### Checkpoint and resume
+
+Replay reads bounded JSONL records into a disposable SQLite working store, with a 2 MiB
+page cache. Advisory descriptions, non-fix references and vulnerable-version payloads
+are discarded; package identities, CWEs, status, publication dates and every fix link
+are retained. Repository metadata keeps canonical/fork identities and eligibility fields.
+Advisory ordering, fix indexes and checkpoint lookups stay on disk, independent of the
+number of journal records. The original seeded ordering and ordinary draws are preserved.
+Per-repository history and exclusion sets also spill to private temporary storage, which
+is removed on success or failure. Budget disk space for the journal, working store and
+spooled Git output as well as the clone.
+
+To bound individual reads, journal records are limited to 16 MiB, decoded Git command
+output to 8 MiB, history messages to 1 MiB and source blobs to 512 KiB. Exceeding a limit
+fails the instrument explicitly; it never counts as an empty history or a clean result.
+The synthetic 20,000-advisory regression checks a 64 MiB traced-memory ceiling across
+replay, enumeration, ordering and fix indexing; this is not a live coordinator RSS measurement.
+
+The memory fix can migrate the exact preceding design fingerprints automatically,
+keeping `inputs-before-memory-fix.json` and leaving the JSONL journal untouched. All
+other input/design mismatches still refuse resume. This compatibility exception must
+be removed if a later change alters selection or sampling semantics.
+
+The command above uses journal directory
+`/home/mc/ousast-results/plane/unseen-draft/journal`. Run the same command to resume.
+`--max-hours` is a per-invocation budget; changing it does not invalidate checkpoints.
+Create `journal/STOP` to stop between units, then remove it before resuming. A page,
+repository resolution, candidate decision, or repository extraction already running
+finishes first, so the time budget is not a hard timeout on a large repository.
+Clean stops exit 0 with `status: stopped`; failures and shortfalls exit 1. Completion
+exits 0 with `status: draft`. No partial manifest is published.
+
+`inputs.json` binds the seed, source-labelled used-set digest, known-empty declarations,
+clone cap, and design fingerprints (sourcing/extraction/eligibility/publication code and
+family taxonomy). A mismatch refuses resume and prints only the changed parameter
+labels. Use a new external cache for a deliberately changed design or population.
+The used set is rebuilt each invocation; repository resolutions, including 404s, are
+reused from the journal. Live GitHub metadata for unfinished units can still change;
+completed pages and decisions are the durable sourcing snapshot.
+
+Each completed unit is appended to its stage JSONL and fsynced before work advances:
+`advisories.jsonl` (pages), `resolution.jsonl` (alias/fork metadata),
+`eligibility.jsonl` (candidate decisions), `extraction.jsonl` (repository/advisory
+results, vulnerable range, ordinary draws, pinned licence, method and counts), and
+`completion.jsonl` (final manifest digest). A lock permits only one writer.
+`progress.json` is atomically rewritten after every unit with stage counts, last unit,
+last observed GitHub rate-limit headers, and cumulative active elapsed seconds.
+These files may contain identities and must stay outside version control. Stdout and
+repository records contain only counts, digests and fixed labels.
+
+Replaying completed units makes no GitHub/OSV or clone calls. Final manifests are
+reconstructed deterministically from journal results; an interrupted publication can
+be retried, but a different existing manifest is refused. Completed JSONL rows are
+never rewritten. Malformed or torn records fail closed instead of being interpreted
+as completed work. Tests interrupt each stage with fakes, then compare resumed and
+uninterrupted manifests byte for byte.
+
+### Planning estimate for task 2.5 (not measured)
+
+Let A be all reviewed advisories, R distinct fix-linked repository identities, U used
+identities not already cached, C candidate advisories passing early filters, and K
+repositories surviving extraction. GitHub requests are approximately
+`ceil(A/100) + R + U + 2*C + K` (enumeration, canonical repositories, fix metadata,
+provisional licence, final licence), plus retries. OSV requests are at most C. Git
+fetches and lazy blob transfers are additional network operations, not GitHub API calls.
+
+For a planning scenario of A=40,000, R=15,000, U≈1,000, C=600–1,500, K=300:
+about **18,000–20,000 GitHub calls** and **600–1,500 OSV calls**. The all-advisory index
+makes this substantially more than 300 repository lookups. At an assumed 5,000-request
+hourly allowance, allow several rate-limit windows. Actual token limits can differ.
+For 300–1,000 attempted clones averaging 20–100 MB after source hydration, plan roughly
+**6–100 GB transferred**, with one clone occupying about 500 MiB at most between commands
+plus the retained journal and manifests after completion. Transfer volume is an estimate; disk bytes
+are only a proxy. A working planning range is **8–48 hours**, potentially longer on
+large histories: every first-parent commit is inspected and lazy blobs can dominate.
+These are assumptions, not evidence from a network run. The command reports actual
+counts and wall time; task 2.5 establishes the measured estimate.
+
+## Dispatcher, replay and scoring (groups 4–6)
+
+`benchmarks.ax.batch` owns Task transport and the shared scheduler. New workloads supply
+`Item` input bytes/commands and a `Workload` image, URL allowlist, deadline and result
+validator. Two AX slots and one VM slot are the default; sandbox death gets one fresh
+Task before VM fallback, OOM goes straight to VM, and instrument failures get one rerun.
+Cleanup failure prevents further launches. Create `STOP` in the output directory to stop
+launching; remove it and repeat the command to resume matching input digests. The legacy
+engine adapter retains its `questions.json` object name for compatibility; replay inputs
+and outputs use `.dat`. Both use the same object polling and cleanup implementation.
+
+Replay reads `pool-pN.toml`, verifies its adjacent `freeze-pN.json` canonical digest,
+selects `--slice`, and resolves private pointers through `private/pool-pN.toml` (or
+`--private-manifest`). It never reads a draft. `--dry-run` prints only counts and digests.
+The five-change proof uses a separate frozen JSON manifest from this project's own
+source-changing history. Raw output must be outside the repository.
+
+Each output root contains `inputs.json` and digest-keyed records and artifact directories.
+The scorer accepts that root directly, joins change labels, and includes missing or stale
+results as instrument failures. It refuses a slice above 5% failure after reruns. Example:
+
+```sh
+python -m benchmarks.unseen.score "$HOME/ousast-results/unseen/p1/1/default" \
+  --out "$HOME/ousast-results/unseen/p1/1/default-score.json"
+```
+
+For paired binary arms add `--against BASELINE_ROOT --slice-id p1/2`; `--informed` forces
+`unseen.gate=not_run`. The later experiment integration owns ledger verification; callers
+of `gate` must pass that ledger decision. Score-bearing arms use `budgets` with fixed
+thresholds and `gate(..., binary=False, threshold=..., threshold_fixed=True)`. In-slice
+thresholds are exploratory and cannot qualify adoption. JVM wall time is explicitly the
+sum of the hook artifact's scan build/query times, not a separately observed process timer.
+
+### Coordinator task 4.4
+
+Run from the merged repository root. Set `ENGINE_IMAGE` to the **engine** digest printed
+by Engine image CI, `ENGINE_UNITS` and `ENGINE_EXAMPLES` to the frozen input files for the
+two stored pins, `ENGINE_BASELINE` to their stored record directory, and `AX_KUBECONFIG`
+to the operator's kube-ax kubeconfig. Host store credentials stay on the VM; use the public
+HTTPS `S3_ENDPOINT=https://files.because-security.com` and the operator's `OUSAST_MEMORY`.
+Use a fresh output directory, so the smoke executes instead of resuming prior records.
+
+```sh
+export PATH=/home/mc/Source/OpenUltraSAST/.venv/bin:$PATH
+export PYTHONPATH=$PWD/src
+export S3_ENDPOINT=https://files.because-security.com
+python benchmarks/learn/engine_trace.py \
+  --units "$ENGINE_UNITS" --examples "$ENGINE_EXAMPLES" \
+  --executor ax --parallel 2 --limit 2 --image "$ENGINE_IMAGE" \
+  --kubeconfig "$AX_KUBECONFIG" --out "$HOME/ousast-results/unseen/proof-04"
+python -m benchmarks.unseen.proof engine --image "$ENGINE_IMAGE" \
+  --records "$HOME/ousast-results/unseen/proof-04" --baseline "$ENGINE_BASELINE" \
+  --out "benchmarks/measurements/$(date +%F)-unseen-04-dispatcher/record.json"
+```
+
+### Coordinator task 5.4
+
+Merge before running this proof. `.github/workflows/engine-image.yml` builds the `engine`
+and `replay` targets on push to main; it publishes `ousast-engine-task` and
+`ousast-replay-task` separately and prints their digests. Set `REPLAY_IMAGE` to the latter
+full `ghcr.io/...@sha256:...` reference. No manual image build is needed. Set
+`KIND_KUBECONFIG` and `AX_KUBECONFIG` to the corresponding AX kubeconfigs; the kind lane
+uses the identical Task API/manifest, never a Kubernetes Job.
+
+```sh
+export PATH=/home/mc/Source/OpenUltraSAST/.venv/bin:$PATH
+export PYTHONPATH=$PWD/src
+export S3_ENDPOINT=https://files.because-security.com
+python -m benchmarks.unseen.replay --proof-history \
+  --manifest "$HOME/ousast-results/unseen/proof-05-inputs.json"
+python -m benchmarks.unseen.replay --manifest "$HOME/ousast-results/unseen/proof-05-inputs.json" \
+  --image "$REPLAY_IMAGE" --lane kind --kubeconfig "$KIND_KUBECONFIG" \
+  --out "$HOME/ousast-results/unseen/proof-05-kind"
+python -m benchmarks.unseen.replay --manifest "$HOME/ousast-results/unseen/proof-05-inputs.json" \
+  --image "$REPLAY_IMAGE" --lane ax --kubeconfig "$AX_KUBECONFIG" \
+  --out "$HOME/ousast-results/unseen/proof-05-ax"
+python -m benchmarks.unseen.replay --manifest "$HOME/ousast-results/unseen/proof-05-inputs.json" \
+  --image "$REPLAY_IMAGE" --lane docker \
+  --out "$HOME/ousast-results/unseen/proof-05-vm"
+python -m benchmarks.unseen.proof replay --image "$REPLAY_IMAGE" \
+  --kind-records "$HOME/ousast-results/unseen/proof-05-kind" \
+  --ax-records "$HOME/ousast-results/unseen/proof-05-ax" \
+  --vm-records "$HOME/ousast-results/unseen/proof-05-vm" \
+  --out "benchmarks/measurements/$(date +%F)-unseen-05-replay-task/record.json"
+```
+
+The record command fails if fewer than five results exist, identities differ, bytes/JVM
+proof is missing, or quick-tier findings differ. It does not convert a quick-only run
+without a JVM into an engine proof. Pool replays use `--lane mixed` (default) for the two
+AX slots and single VM overflow/fallback lane; `--arm long_deadline` changes only the
+hook deadline to 300 seconds and is a separate input digest.

benchmarks/unseen/draft.py

@@ -0,0 +1,493 @@
+"""Coordinator-only sourcing command: python -m benchmarks.unseen.draft --help.
+
+All network and git operations are injected at the run() boundary. Only counts,
+digests and fixed failure codes go to stdout; identity-bearing output stays in the
+private journal outside the repository. No population guard is run here.
+"""
+
+from __future__ import annotations
+
+import argparse
+import copy
+import hashlib
+import json
+import os
+import random
+import subprocess
+import tempfile
+import time
+from collections import Counter, defaultdict
+from contextlib import contextmanager
+from dataclasses import asdict
+from pathlib import Path
+
+from . import eligibility, extract, source
+from .extract import Git, InstrumentFailure
+from .journal import Journal, ResolvedGitHub, ResumeMismatch, Stopped, atomic, digest, external
+from .storage import AdvisoryRows, DiskSet, FixIndex, working_store
+
+
+def canonical(value) -> bytes:
+    return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
+
+
+def private_digest(row: dict) -> str:
+    return hashlib.sha256(canonical({"url": row["url"], "commits": [[c["base"], c["head"]] for c in row["changes"]]})).hexdigest()
+
+
+def assign_slices(rows: list[dict], *, seed: int) -> list[dict]:
+    if len(rows) != 300 or len({r["repository"] for r in rows}) != 300:
+        raise ValueError("pool_size")
+    for row in rows:
+        kinds = Counter(c["kind"] for c in row["changes"])
+        if kinds != {"vulnerability": 1, "ordinary": 12}:
+            raise ValueError("change_count")
+        if row["license_class"] not in {"permissive", "private"}:
+            raise ValueError("license_class")
+        if row["license_class"] == "permissive" and row["license_spdx"] not in source.PERMISSIVE:
+            raise ValueError("license_class")
+    rng = random.Random(seed)
+    groups = defaultdict(list)
+    for row in sorted(rows, key=lambda r: r["repository"]):
+        groups[row["ecosystem"], row["post_cutoff"]].append(row)
+    sizes = Counter({1: 0, 2: 0, 3: 0})
+    assigned = []
+    for key in sorted(groups):
+        group = groups[key]
+        rng.shuffle(group)
+        quotient, remainder = divmod(len(group), 3)
+        # Give each stratum its floor on all slices, then fill globally least-full slices.
+        quotas = {i: quotient for i in sizes}
+        order = list(sizes)
+        rng.shuffle(order)
+        order.sort(key=lambda i: sizes[i])
+        for i in order[:remainder]:
+            quotas[i] += 1
+        offset = 0
+        for slice_id in sorted(quotas):
+            for row in group[offset : offset + quotas[slice_id]]:
+                digest = private_digest(row)
+                assigned.append({**row, "id": "r-" + digest[:24], "slice": slice_id})
+            sizes[slice_id] += quotas[slice_id]
+            offset += quotas[slice_id]
+    if set(sizes.values()) != {100}:
+        raise ValueError("slice_size")
+    return sorted(assigned, key=lambda r: (r["slice"], r["id"]))
+
+
+def toml(value) -> str:
+    """Inline TOML values keep nested change records structured without a new dependency."""
+    if isinstance(value, bool):
+        return "true" if value else "false"
+    if isinstance(value, str):
+        return json.dumps(value, ensure_ascii=False)
+    if isinstance(value, int):
+        return str(value)
+    if isinstance(value, list):
+        return "[" + ", ".join(toml(v) for v in value) + "]"
+    if isinstance(value, dict):
+        return "{" + ", ".join(json.dumps(k) + " = " + toml(v) for k, v in sorted(value.items()) if v is not None) + "}"
+    raise ValueError("unsupported_manifest_type")
+
+
+def manifest(rows: list[dict], seed: int) -> str:
+    lines = ['status = "draft"', 'pool = "p1"', f"seed = {seed}"]
+    for row in rows:
+        lines.append("\n[[repository]]")
+        lines.extend(f"{key} = {toml(value)}" for key, value in sorted(row.items()) if value is not None)
+    return "\n".join(lines) + "\n"
+
+
+def write_draft(root: Path, assigned: list[dict], *, seed: int) -> tuple[Path, Path]:
+    public_path = root / "benchmarks/unseen/draft-p1.toml"
+    private_path = root / "benchmarks/unseen/private/draft-p1.toml"
+    if public_path.exists() or private_path.exists():
+        raise FileExistsError("draft_exists")
+    public, private = [], []
+    for row in assigned:
+        if row["license_class"] == "permissive":
+            public.append(row)
+        else:
+            private.append(row)
+            public.append({"id": row["id"], "sha256": private_digest(row), "slice": row["slice"], "private": True})
+    private_path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
+    # Exclusive creates protect drafts awaiting human label review. Private first: never
+    # publish a pointer whose private manifest failed to write.
+    with private_path.open("x") as handle:
+        os.chmod(private_path, 0o600)
+        handle.write(manifest(private, seed))
+    with public_path.open("x") as handle:
+        handle.write(manifest(public, seed))
+    return public_path, private_path
+
+
+class Clones:
+    """One disposable blobless clone at a time; no name in cache paths or subprocess output."""
+
+    def __init__(self, root: Path, cache: Path, *, runner=None, limit_bytes=500 * 1024 * 1024):
+        self.cache = external(root, cache)
+        self.runner = runner or subprocess.run
+        self.limit_bytes = limit_bytes
+        self.counts = Counter()
+
+    @contextmanager
+    def open(self, url: str):
+        self.cache.mkdir(parents=True, exist_ok=True)
+        started = time.monotonic()
+        with tempfile.TemporaryDirectory(prefix="clone-", dir=self.cache) as directory:
+            path = Path(directory)
+            git = Git(path, runner=self.runner, limit_bytes=self.limit_bytes)
+            try:
+                # Full commit graph is needed for temporal thirds and OSV brackets; blobs
+                # are fetched lazily. This is a partial clone, not a truncated history.
+                done = self.runner(
+                    ["git", "-c", "credential.helper=", "clone", "--quiet", "--filter=blob:none", "--no-checkout", url, str(path)],
+                    stdin=subprocess.DEVNULL,
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.PIPE,
+                    check=False,
+                    timeout=180,
+                    env={**os.environ, "GIT_TERMINAL_PROMPT": "0"},
+                )
+                if done.returncode:
+                    raise InstrumentFailure("clone_failed")
+                self.counts["clones"] += 1
+                if git.disk_bytes() > self.limit_bytes:
+                    raise InstrumentFailure("clone_size_cap")
+                yield git
+            finally:
+                size = git.disk_bytes()
+                self.counts["clone_bytes"] += size
+                self.counts["peak_clone_bytes"] = max(self.counts["peak_clone_bytes"], size)
+                self.counts["source_bytes_read"] += git.bytes_read
+                self.counts["source_files_read"] += git.files_read
+                self.counts["clone_wall_seconds"] += round(time.monotonic() - started, 3)
+
+
+def prove_source(git, revision: str) -> dict:
+    files = git.run("ls-tree", "-r", "--name-only", revision).splitlines()
+    for path in files:
+        if Path(path).suffix in extract.LANGUAGES and not extract.generated_path(path):
+            text = git.blob(revision, path)
+            if text:
+                return {"files": 1, "bytes": getattr(git, "last_blob_bytes", len(text.encode()))}
+    raise InstrumentFailure("zero_source_bytes")
+
+
+def advisory_index(rows, github, *, db):
+    """Include ALL enumerated fixes, even old, unmapped, multi-fix and other ecosystems.
+
+    Canonicalize link names as well, so an old advisory URL survives repository renames.
+    A 404 is not treated as proof that an advisory is unrelated to a selected repository.
+    """
+    index = FixIndex(db)
+    for row in rows:
+        for name, sha in source.fix_links(row):
+            try:
+                resolved = eligibility.resolve(name, github)
+            except eligibility.RepositoryNotFound:
+                index.add(name, sha)
+                continue
+            index.add(resolved.full_name, sha)
+    return index
+
+
+def repository_changes(git, candidate, fixes: set[str], *, seed: int):
+    proof = prove_source(git, candidate.parent)
+    change = extract.introducing(git, candidate)
+    with working_store() as db:
+        known = DiskSet(db, "known_functions")
+        expanded = DiskSet(db, "expanded_fixes")
+        pending = DiskSet(db, "pending_fixes")
+        pending.update(fixes)
+        pending.add(candidate.fix)
+        for fix in pending:
+            # Explicitly fetch detached advisory commits unavailable on default-branch history.
+            # All git including this possible network call still goes through the injected client.
+            git.run("fetch", "--quiet", "--filter=blob:none", "origin", fix)
+            full = git.run("rev-parse", fix).strip()
+            expanded.add(full)
+            parents = git.parents(full)
+            for parent in parents:
+                sites = extract.fix_sites(git, full, parent, global_ok=True)
+                known.update((site["path"], site["function"]) for site in sites)
+        rows = extract.ordinary_history(git, "HEAD")
+        ordinary, exclusions = extract.draw_ordinary(rows, expanded, known, seed=seed)
+    return [change, *ordinary], exclusions, proof
+
+
+def inputs(used, seed, known_empty, clones):
+    from openultrasast.model.taxonomy import DEFAULT_FAMILIES_PATH
+
+    parameters = {
+        "seed": seed,
+        "used_set_digest": digest({label: sorted(names) for label, names in used.sources.items()}),
+        "known_empty": sorted(known_empty),
+        "clone_limit_bytes": getattr(clones, "limit_bytes", 500 * 1024 * 1024),
+        "journal_version": 1,
+    }
+    # Hardcoded design thresholds and algorithms are covered as well as taxonomy data.
+    for name in ("draft", "source", "extract", "eligibility", "journal", "storage"):
+        parameters["design_" + name] = hashlib.sha256(Path(__file__).with_name(name + ".py").read_bytes()).hexdigest()
+    parameters["design_taxonomy"] = hashlib.sha256(DEFAULT_FAMILIES_PATH.read_bytes()).hexdigest()
+    return parameters
+
+
+# The pre-memory-fix implementation has identical ordering/selection semantics. Only
+# this exact fingerprint set may migrate; changed seeds, taxonomy or used sets still fail.
+LEGACY_DESIGNS = {
+    "design_draft": "906bf36de11761931b2c1b62a2ea8da3c8124f9d754455f1a0acf29a6db576a3",
+    "design_source": "01c896bd0418faf0d073dc2d5cb6dbc5d7903b95b5e8216c9022bc9985326a9c",
+    "design_extract": "593d954d9c22b835e1d349df6b575707a39098dca2eff4425f67a0de14c6525e",
+    "design_journal": "c97ad0761ea5bbfe65db816b13dfa79d93608ea03bcd5848f70686ed9157bdf7",
+}
+
+
+def legacy_inputs(parameters):
+    return {key: LEGACY_DESIGNS.get(key, value) for key, value in parameters.items() if key != "design_storage"}
+
+
+def enumerate_advisories(github, journal):
+    page, records = 1, AdvisoryRows(journal.db, "advisory_records")
+    while True:
+        key = str(page)
+        if key not in journal.rows["advisories"]:
+            journal.boundary()
+            result = github.advisory_page(page)
+            journal.put("advisories", key, result)
+        result = journal.rows["advisories"][key]
+        for row in result["rows"]:
+            records[row["ghsa_id"]] = row
+        if result["done"]:
+            break
+        page += 1
+    if not records:
+        raise InstrumentFailure("empty_advisories")
+    return records
+
+
+def decide(row, github, osv, used, selected_networks):
+    try:
+        links = source.fix_links(row)
+        if len(links) != 1:
+            raise source.Rejected("fix_links")
+        name, _ = next(iter(links))
+        resolved = eligibility.resolve(name, github)
+        rejected_by = eligibility.rejection(resolved, used, set(selected_networks))
+        if rejected_by:
+            raise source.Rejected("used_" + rejected_by)
+        candidate = source.candidate(row, github, osv)
+        return {"candidate": asdict(candidate), "network": resolved.network}
+    except source.Rejected as exc:
+        return {"rejected": str(exc)}
+    except eligibility.RepositoryNotFound:
+        return {"rejected": "repository_or_commit_not_found"}
+
+
+def extract_repository(candidate, github, clones, fixes, seed):
+    before = dict(clones.counts)
+    try:
+        with clones.open(candidate.url) as git:
+            git.run("fetch", "--quiet", "--filter=blob:none", "origin", candidate.fix)
+            changes, exclusions, proof = repository_changes(git, candidate, fixes, seed=seed)
+            pinned = changes[0]["head"]
+            license_record = github.get_license(candidate.repository, pinned)
+            spdx, license_class = source.license_info(license_record)
+            entry = {
+                **asdict(candidate),
+                "license_spdx": spdx,
+                "license_class": license_class,
+                "license_ref": pinned,
+                "license_path": license_record.get("path", ""),
+                "changes": changes,
+                "instrument": proof,
+            }
+        result = {"entry": entry, "ordinary_exclusions": exclusions}
+    except source.Rejected as exc:
+        result = {"rejected": str(exc), "ordinary_exclusions": getattr(exc, "counts", {})}
+    except eligibility.RepositoryNotFound:
+        result = {"rejected": "repository_or_commit_not_found", "ordinary_exclusions": {}}
+    result["counts"] = {key: value - before.get(key, 0) for key, value in clones.counts.items()}
+    return result
+
+
+def publish(journal, assigned, seed):
+    """Idempotent publication from completed journal data, never inside the checkout."""
+    public, private = [], []
+    for row in assigned:
+        if row["license_class"] == "permissive":
+            public.append(row)
+        else:
+            private.append(row)
+            public.append({"id": row["id"], "sha256": private_digest(row), "slice": row["slice"], "private": True})
+    for name, rows in (("private-draft-p1.toml", private), ("draft-p1.toml", public)):
+        path = journal.path / name
+        data = manifest(rows, seed).encode()
+        if path.exists():
+            if path.read_bytes() != data:
+                raise ValueError("draft_exists")
+        else:
+            atomic(path, data)
+
+
+def run(
+    root: Path,
+    *,
+    github,
+    osv,
+    clones,
+    used: eligibility.UsedSet,
+    seed: int,
+    known_empty=None,
+    journal_path=None,
+    max_hours=None,
+    clock=None,
+) -> dict:
+    started = time.monotonic()
+    report = {"status": "sourcing", "seed": seed, "candidates": {}, "rejections": {}, "ordinary_exclusions": {}}
+    counts, rejected, ordinary_counts = Counter(), Counter(), Counter()
+    checkpoint = None
+    try:
+        known_empty = set(known_empty or ())
+        used.report(known_empty)
+        cache = getattr(clones, "cache", root.parent / (root.name + "-cache"))
+        path = external(root, journal_path or cache / "journal")
+        checkpoint = Journal(
+            path, max_hours=max_hours, clock=clock, rate_limit=lambda: getattr(getattr(github, "transport", None), "rate_limit", {})
+        )
+        parameters = inputs(used, seed, known_empty, clones)
+        with checkpoint.open(parameters, compatible_parameters=legacy_inputs(parameters)):
+            checkpoint.boundary()
+            api = ResolvedGitHub(github, checkpoint)
+            used = copy.deepcopy(used)  # canonicalization must not mutate the resume input
+            eligibility.canonicalize_used(used, api)
+            report["used_set"] = used.report(known_empty)
+            rows = enumerate_advisories(api, checkpoint)
+            report["advisories"] = len(rows)
+            fixes_by_repo = advisory_index(rows, api, db=checkpoint.db)
+            selected_networks, entries = set(), []
+            for row in source.ordered_advisories(rows, seed):
+                checkpoint.boundary()
+                key = row["ghsa_id"]
+                counts[source.ecosystem(row)] += 1
+                if key not in checkpoint.rows["eligibility"]:
+                    decision = decide(row, api, osv, used, selected_networks)
+                    checkpoint.put("eligibility", key, decision)
+                decision = checkpoint.rows["eligibility"][key]
+                if "rejected" in decision:
+                    rejected[decision["rejected"]] += 1
+                    continue
+                candidate = source.Candidate(**decision["candidate"])
+                # Multiple advisories for one repository can have different fix/range inputs.
+                extraction_key = digest({"repository": candidate.repository, "advisory": key, "fix": candidate.fix})
+                if extraction_key not in checkpoint.rows["extraction"]:
+                    checkpoint.boundary()
+                    result = extract_repository(candidate, api, clones, fixes_by_repo.get(candidate.repository, set()), seed)
+                    checkpoint.put("extraction", extraction_key, result)
+                result = checkpoint.rows["extraction"][extraction_key]
+                ordinary_counts.update(result["ordinary_exclusions"])
+                if "rejected" in result:
+                    rejected[result["rejected"]] += 1
+                    continue
+                selected_networks.add(decision["network"])
+                entries.append(result["entry"])
+                if len(entries) == 300:
+                    break
+            report.update(
+                {
+                    "repositories": len(entries),
+                    "candidates": dict(counts),
+                    "rejections": dict(rejected),
+                    "ordinary_exclusions": dict(ordinary_counts),
+                }
+            )
+            checkpoint.boundary()
+            if len(entries) != 300:
+                report["status"] = "insufficient_repositories"
+                checkpoint.progress(report["status"])
+                return report
+            assigned = assign_slices(entries, seed=seed)
+            completion = {"manifest_sha256": digest(assigned), "repositories": len(entries)}
+            if "all" not in checkpoint.rows["completion"]:
+                checkpoint.put("completion", "all", completion)
+            elif checkpoint.rows["completion"]["all"] != completion:
+                raise ValueError("journal_completion_mismatch")
+            checkpoint.boundary()
+            publish(checkpoint, assigned, seed)
+            report.update(
+                {
+                    "status": "draft",
+                    "changes": 3900,
+                    "slices": 3,
+                    "license_split": dict(Counter(e["license_class"] for e in entries)),
+                    "methods": dict(Counter(e["changes"][0]["method"] for e in entries)),
+                    "post_cutoff": sum(e["post_cutoff"] for e in entries),
+                    **completion,
+                }
+            )
+            checkpoint.progress("draft")
+            return report
+    except ResumeMismatch as exc:
+        report.update({"status": "resume_refused", "changed_parameters": exc.parameters})
+        return report
+    except Stopped as exc:
+        report.update({"status": "stopped", "reason": str(exc)})
+        return report
+    except Exception as exc:
+        report.update({"status": "instrument_failure", "error_type": type(exc).__name__})
+        return report
+    finally:
+        report.update({"candidates": dict(counts), "rejections": dict(rejected), "ordinary_exclusions": dict(ordinary_counts)})
+        report["instrument"] = dict(clones.counts)
+        report["wall_seconds"] = round(time.monotonic() - started, 3)
+        report["github_calls"] = github.calls
+        report["osv_calls"] = osv.calls
+        report["github_http_attempts"] = getattr(getattr(github, "transport", None), "attempts", github.calls)
+        report["osv_http_attempts"] = getattr(getattr(osv, "transport", None), "attempts", osv.calls)
+
+
+def main(argv: list[str] | None = None) -> int:
+    from openultrasast import config
+    from openultrasast.plane import memory
+    from openultrasast.plane.reconciler import results_root
+
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--root", type=Path, default=Path.cwd())
+    parser.add_argument("--cache", type=Path)
+    parser.add_argument("--max-hours", type=float, help="Stop between units after this many hours per invocation")
+    parser.add_argument("--seed", type=int, default=20260601)
+    parser.add_argument("--local-store")
+    parser.add_argument("--s3-store", default="s3://")
+    parser.add_argument("--pointer-manifest", type=Path, action="append", default=[])
+    parser.add_argument("--known-empty", choices=(*eligibility.SOURCES, "memory_local", "memory_s3"), action="append", default=[])
+    args = parser.parse_args(argv)
+    try:
+        config.load_dotenv(args.root / ".env")
+        # Refuse an existing draft before doing any network work.
+        if any((args.root / p).exists() for p in ("benchmarks/unseen/draft-p1.toml", "benchmarks/unseen/private/draft-p1.toml")):
+            raise FileExistsError("draft_exists")
+        clones = Clones(args.root, args.cache or results_root() / "unseen-draft")
+        stores = {
+            "local": memory.open_store(args.local_store or "file://" + str(results_root() / "memory"), read_only=True),
+            "s3": memory.open_store(args.s3_store, read_only=True),
+        }
+        used = eligibility.build_used(args.root, stores=stores, pointer_manifests=args.pointer_manifest)
+        report = run(
+            args.root,
+            github=source.GitHub(os.environ.get("GH_TOKEN", "")),
+            osv=source.OSV(),
+            clones=clones,
+            used=used,
+            seed=args.seed,
+            known_empty=set(args.known_empty),
+            journal_path=clones.cache / "journal",
+            max_hours=args.max_hours,
+        )
+    except Exception as exc:
+        report = {"status": "instrument_failure", "error_type": type(exc).__name__}
+    print(json.dumps(report, sort_keys=True))
+    return 0 if report["status"] in {"draft", "stopped"} else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/unseen/eligibility.py

@@ -0,0 +1,379 @@
+"""Read-only used-set collection. Only counts, source labels and digests leave this module's CLI.
+
+Run from the repository root: python -m benchmarks.unseen.eligibility --help.
+Population v3 is exclusively the existing guard's responsibility.
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import re
+import tomllib
+from collections.abc import Iterable, Mapping
+from dataclasses import dataclass, field
+from pathlib import Path
+from urllib.parse import urlsplit
+
+import yaml
+
+from openultrasast.plane.memory import MemoryStore
+
+URL = re.compile(r"github\.com/([\w.-]+/[\w.-]+)", re.I)
+NAME = re.compile(r"[\w.-]+/[\w.-]+")
+SOURCES = ("pairs", "pointers", "populations", "harvest", "units", "measurements", "sweep")
+
+
+def normalize(value: str) -> str:
+    text = value.strip().lower().rstrip("/")
+    if NAME.fullmatch(text):
+        name = text.removesuffix(".git")
+    else:
+        # Structured fields must be identities, not prose containing a URL.
+        if any(char.isspace() for char in text):
+            raise ValueError("invalid_repository")
+        url = urlsplit(text if "://" in text else "https://" + text)
+        if url.scheme not in {"http", "https", "ssh", "git"} or not url.hostname or "." not in url.hostname:
+            raise ValueError("invalid_repository")
+        parts = url.path.strip("/").split("/")
+        if len(parts) < 2:
+            raise ValueError("invalid_repository")
+        name = "/".join(parts[:2]).removesuffix(".git")
+        if not NAME.fullmatch(name):
+            raise ValueError("invalid_repository")
+        if url.hostname != "github.com":
+            return url.hostname + "/" + name
+    if not NAME.fullmatch(name) or any(part in {".", ".."} for part in name.split("/")):
+        raise ValueError("invalid_repository")
+    return name
+
+
+def excluded(path: Path) -> bool:
+    return any("v3" in part.lower() or part in {".git", "__pycache__", ".venv"} for part in path.parts)
+
+
+def repositories(value: object, counts: dict[str, int] | None = None) -> set[str]:
+    """Structured repo fields include bare owner/name pointers; URLs also occur in prose."""
+    found = set()
+    if isinstance(value, dict):
+        for key, item in value.items():
+            if key in {"repo", "fix_repo", "repository", "full_name", "origin"} or (key == "group" and "unit" in value):
+                try:
+                    if not isinstance(item, str):
+                        raise ValueError("invalid_repository")
+                    found.add(normalize(item))
+                except ValueError:
+                    if counts is not None:
+                        counts["skipped_values"] = counts.get("skipped_values", 0) + 1
+            else:
+                found.update(repositories(item, counts))
+    elif isinstance(value, list):
+        for item in value:
+            found.update(repositories(item, counts))
+    elif isinstance(value, str):
+        found.update(normalize(m[1]) for m in URL.finditer(value))
+    return found
+
+
+def pointer_repositories(value: object, counts: dict[str, int] | None = None) -> set[str]:
+    if isinstance(value, dict):
+        if value.get("vendored") is False:
+            return repositories(value, counts)
+        return set().union(*(pointer_repositories(v, counts) for v in value.values()))
+    if isinstance(value, list):
+        return set().union(*(pointer_repositories(v, counts) for v in value))
+    return set()
+
+
+def parse(path: Path, raw: bytes) -> object:
+    structured = path.suffix in {".toml", ".json", ".jsonl", ".yaml", ".yml"}
+    text = raw.decode("utf-8", errors="strict" if structured else "ignore")
+    if path.suffix == ".toml":
+        return tomllib.loads(text)
+    if path.suffix == ".json":
+        return json.loads(text)
+    if path.suffix == ".jsonl":
+        return [json.loads(line) for line in text.splitlines() if line.strip()]
+    if path.suffix in {".yaml", ".yml"}:
+        documents = list(yaml.safe_load_all(text))
+        return documents[0] if len(documents) == 1 else documents
+    return text
+
+
+@dataclass
+class UsedSet:
+    sources: dict[str, set[str]] = field(default_factory=lambda: {label: set() for label in SOURCES})
+    instrument: dict[str, dict[str, int]] = field(default_factory=dict)
+
+    def add(self, label: str, names: set[str], size: int, files: int = 1, skipped_values: int = 0) -> None:
+        self.sources.setdefault(label, set()).update(names)
+        counts = self.instrument.setdefault(label, {"files": 0, "bytes": 0, "rows": 0, "skipped_values": 0})
+        counts["files"] += files
+        counts["bytes"] += size
+        counts["rows"] += len(names)
+        counts["skipped_values"] += skipped_values
+
+    def report(self, known_empty: set[str] | None = None) -> dict:
+        known_empty = known_empty or set()
+        if known_empty - self.sources.keys():
+            raise ValueError("unknown_source_label")
+        records = {}
+        for label, names in self.sources.items():
+            counts = self.instrument.get(label, {"files": 0, "bytes": 0, "rows": 0, "skipped_values": 0})
+            if not counts["rows"] and label not in known_empty:
+                raise ValueError("empty_source:" + label)
+            records[label] = {**counts, "count": len(names), "known_empty": label in known_empty}
+        names = set().union(*self.sources.values())
+        return {"sources": records, "count": len(names), "sha256": hashlib.sha256("\n".join(sorted(names)).encode()).hexdigest()}
+
+
+def build_used(root: Path, *, stores: Mapping[str, MemoryStore] | None = None, pointer_manifests: Iterable[Path] = ()) -> UsedSet:
+    used = UsedSet()
+
+    def read(label: str, paths: Iterable[Path], structured: bool = True) -> list[object]:
+        values = []
+        for path in sorted(set(paths)):
+            if excluded(path):
+                continue
+            raw = path.read_bytes()  # failure is an instrument failure, never a quiet empty source
+            value = parse(path, raw) if structured else raw.decode("utf-8", errors="ignore")
+            counts: dict[str, int] = {}
+            names = repositories(value, counts)
+            used.add(label, names, len(raw), skipped_values=counts.get("skipped_values", 0))
+            if label == "pairs":
+                pointer_counts: dict[str, int] = {}
+                pointers = pointer_repositories(value, pointer_counts)
+                if pointers or pointer_counts:
+                    used.add("pointers", pointers, len(raw), skipped_values=pointer_counts.get("skipped_values", 0))
+            values.append(value)
+        return values
+
+    pairs = root / "benchmarks/pairs"
+    read("pairs", (p for p in pairs.rglob("*") if p.is_file() and p.suffix in {".toml", ".json", ".jsonl", ".csv", ".yaml", ".yml"}))
+    read("pointers", pointer_manifests)
+    read("populations", (p for p in (root / "benchmarks/independent").glob("population-v[12].toml")))
+    read("harvest", (p for directory in pairs.glob("advisory-fixes*") for p in directory.rglob("*.toml")))
+    units = set((root / "plane/experiments").glob("*.units.jsonl"))
+    for path in (root / "plane/experiments").glob("*.y*ml"):
+        if excluded(path):
+            continue
+        raw = path.read_bytes()
+        manifest = parse(path, raw)
+        counts = {}
+        names = repositories(manifest, counts)
+        used.add("units", names, len(raw), skipped_values=counts.get("skipped_values", 0))
+        if isinstance(manifest, dict) and manifest.get("units_file"):
+            declared = root / Path(manifest["units_file"]).expanduser()
+            if declared.exists():  # an unregistered plan may not have frozen units yet
+                units.add(declared)
+    for label, store in (stores or {}).items():
+        if label not in {"local", "s3"}:
+            raise ValueError("unknown_store_label")
+        source = "memory_" + label
+        used.add(source, set(), 0, 0)
+        # Read actual objects, not a serialized row size masquerading as bytes read.
+        for key in store._keys("repos/"):
+            if not key.endswith(".jsonl") or excluded(Path(key)):
+                continue
+            result = store._get(key)
+            if result is None:
+                raise ValueError("missing_memory_object")
+            raw = result[0]
+            rows = parse(Path("rows.jsonl"), raw)
+            names = set()
+            counts = {}
+            for row in rows:
+                if row.get("kind") == "example" and "repo" in row:
+                    names.update(repositories({"repo": row["repo"]}, counts))
+                if row.get("kind") == "experiment" and row.get("units_file"):
+                    registered = root / Path(row["units_file"]).expanduser()
+                    if row.get("units_digest") or registered.exists():
+                        units.add(registered)
+            used.add(source, names, len(raw), skipped_values=counts.get("skipped_values", 0))
+    read("units", units)
+    read("measurements", (p for top in ("measurements", "experiments") for p in (root / "benchmarks" / top).rglob("*") if p.is_file()))
+    read(
+        "sweep",
+        (
+            p
+            for top in ("benchmarks", "src", "tests", "plane", ".kiro")
+            for p in (root / top).rglob("*")
+            if p.is_file() and not p.is_relative_to(root / "benchmarks/unseen") and not excluded(p)
+        ),
+        structured=False,
+    )
+    return used
+
+
+@dataclass(frozen=True)
+class Resolution:
+    full_name: str
+    names: frozenset[str]
+    network: str
+
+
+def resolve(candidate: str, api) -> Resolution:
+    name = normalize(candidate)
+    if name.count("/") > 1:
+        return Resolution(name, frozenset({name}), name)
+    data = api.get_repo(name)
+    canonical = normalize(data["full_name"])
+    names = {name, canonical, *(normalize(n) for n in data.get("former_names", []))}
+    for key in ("parent", "source"):
+        if data.get(key):
+            names.add(normalize(data[key]["full_name"]))
+    network = normalize((data.get("source") or data.get("parent") or data)["full_name"])
+    return Resolution(canonical, frozenset(names), network)
+
+
+def canonicalize_used(used: UsedSet, api) -> None:
+    """Resolve old used names too: GitHub exposes redirects, not a complete rename history."""
+    cache = {}
+    for names in used.sources.values():
+        for name in sorted(names):
+            if name not in cache:
+                try:
+                    cache[name] = resolve(name, api)
+                except RepositoryNotFound:
+                    cache[name] = Resolution(name, frozenset({name}), name)
+            names.update(cache[name].names)
+
+
+def rejection(candidate: Resolution, used: UsedSet, selected: set[str]) -> str | None:
+    for label, names in used.sources.items():
+        if candidate.names & names:
+            return label
+    if candidate.network in selected:
+        return "fork_network"
+    selected.add(candidate.network)
+    return None
+
+
+class RepositoryNotFound(ValueError):
+    """A historical or synthetic used identity no longer resolves; retain it verbatim."""
+
+
+class GitHub:
+    """Coordinator-only GET client. Exceptions never disclose request URLs or tokens."""
+
+    def __init__(self, token: str):
+        if not token:
+            raise ValueError("missing_gh_token")
+        self.token = token
+
+    def get_repo(self, name: str) -> dict:
+        from urllib.error import HTTPError
+        from urllib.request import Request, urlopen
+
+        request = Request(
+            "https://api.github.com/repos/" + normalize(name),
+            headers={
+                "Authorization": "Bearer " + self.token,
+                "Accept": "application/vnd.github+json",
+            },
+        )
+        try:
+            with urlopen(request, timeout=30) as response:
+                return json.load(response)
+        except HTTPError as exc:
+            if exc.code == 404:
+                raise RepositoryNotFound("github_not_found") from None
+            raise ValueError("github_resolution_failed") from None
+        except Exception:
+            raise ValueError("github_resolution_failed") from None
+
+
+def verify_tests(root: Path) -> dict:
+    """The coordinator runs the guard as a black box; only exit codes and counts escape."""
+    import subprocess
+    import sys
+    import tempfile
+    from xml.etree import ElementTree
+
+    with tempfile.TemporaryDirectory() as directory:
+        report = Path(directory) / "tests.xml"
+        done = subprocess.run(
+            [
+                sys.executable,
+                "-m",
+                "pytest",
+                "-q",
+                "-p",
+                "no:cacheprovider",
+                "tests/test_unseen_eligibility.py",
+                "tests/test_unseen_ledger.py",
+                "tests/test_independent_population.py",
+                "--junitxml=" + str(report),
+            ],
+            cwd=root,
+            stdin=subprocess.DEVNULL,
+            stdout=subprocess.DEVNULL,
+            stderr=subprocess.DEVNULL,
+            check=False,
+        )
+        if done.returncode != 0:
+            raise ValueError("guard_or_tests_failed")
+        suites = ElementTree.parse(report).getroot().iter("testsuite")
+        counts = {key: 0 for key in ("tests", "failures", "errors", "skipped")}
+        for suite in suites:
+            for key in counts:
+                counts[key] += int(suite.attrib[key])
+        if not counts["tests"]:
+            raise ValueError("empty_test_run")
+        return {"guard": "green", "test_exit_code": done.returncode, "tests": counts}
+
+
+def main(argv: list[str] | None = None) -> int:
+    import os
+
+    from openultrasast import config
+    from openultrasast.plane import memory
+    from openultrasast.plane.reconciler import results_root
+
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--root", type=Path, default=Path.cwd())
+    parser.add_argument("--local-store", default=None)
+    parser.add_argument("--s3-store", default="s3://")
+    parser.add_argument("--pointer-manifest", type=Path, action="append", default=[])
+    parser.add_argument("--known-empty", choices=(*SOURCES, "memory_local", "memory_s3"), action="append", default=[])
+    parser.add_argument("--output", type=Path)
+    parser.add_argument("--verify-tests", action="store_true")
+    args = parser.parse_args(argv)
+    step = "open_stores"
+    try:
+        config.load_dotenv(args.root / ".env")
+        stores = {
+            "local": memory.open_store(args.local_store or "file://" + str(results_root() / "memory"), read_only=True),
+            "s3": memory.open_store(args.s3_store, read_only=True),
+        }
+        step = "build_used"
+        used = build_used(args.root, stores=stores, pointer_manifests=args.pointer_manifest)
+        step = "report"
+        used.report(set(args.known_empty))  # prove inputs before API work
+        step = "canonicalize"
+        canonicalize_used(used, GitHub(os.environ.get("GH_TOKEN", "")))
+        step = "report"
+        report = used.report(set(args.known_empty))
+        if args.verify_tests:
+            step = "verify_tests"
+            report.update(verify_tests(args.root))
+        step = "report"
+        text = json.dumps(report, sort_keys=True, indent=2) + "\n"
+        if args.output:
+            args.output.parent.mkdir(parents=True, exist_ok=True)
+            with args.output.open("x") as handle:
+                handle.write(text)
+        print(text, end="")
+        return 0
+    except Exception as exc:
+        # Source paths, store exceptions and JSON parser errors can contain repository names.
+        report = {"status": "instrument_failure", "step": step, "type": type(exc).__name__}
+        if str(exc) in {"empty_source:" + label for label in (*SOURCES, "memory_local", "memory_s3")}:
+            report["source"] = str(exc).partition(":")[2]
+        print(json.dumps(report))
+        return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/unseen/extract.py

@@ -0,0 +1,519 @@
+"""Git-only change extraction; clients and subprocess execution are injectable.
+
+No scanner, engine or model runs here. Function boundaries are syntax, not a security
+judgement: the first removed code line of a fix is the SZZ sink proxy. Ambiguous sites
+are dropped, and the recorded SZZ bias remains subject to task 2.6's label check.
+"""
+
+from __future__ import annotations
+
+import ast
+import json
+import re
+import subprocess
+import tempfile
+from contextlib import contextmanager
+from pathlib import Path
+
+from .source import Candidate, Rejected
+from .storage import working_store
+
+LANGUAGES = {".py", ".js", ".jsx", ".mjs", ".cjs", ".ts", ".tsx", ".php", ".java"}
+HUNK = re.compile(r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@", re.M)
+SHA = re.compile(r"[a-f0-9]{40}")
+BIAS = {
+    "introducing": "SZZ last-writer proxy: refactors, moves and reformats can misidentify the introduction in either direction.",
+    "last_touch": "Pessimistic lower bound: the flaw may already exist in base, so a delta hook can correctly stay quiet.",
+}
+
+
+class InstrumentFailure(ValueError):
+    """A failed read or command must never masquerade as an empty history."""
+
+
+class Git:
+    def __init__(self, path: Path, *, runner=None, limit_bytes: int | None = None):
+        self.path = path
+        self.runner = runner or subprocess.run
+        self.bytes_read = 0
+        self.files_read = 0
+        self.last_blob_bytes = 0
+        self.limit_bytes = limit_bytes
+
+    @contextmanager
+    def output(self, *args: str, allowed=(0,)):
+        # Git can expand a tiny pack into enormous stdout. Spool before decoding, including
+        # stderr, so neither subprocess.communicate nor an error can consume the VM's RAM.
+        with tempfile.TemporaryFile() as stdout, tempfile.TemporaryFile() as stderr:
+            try:
+                done = self.runner(
+                    ["git", "-c", "core.quotePath=false", "-C", str(self.path), *args],
+                    stdin=subprocess.DEVNULL,
+                    stdout=stdout,
+                    stderr=stderr,
+                    timeout=180,
+                    check=False,
+                )
+            except Exception:
+                raise InstrumentFailure("git_launch_failed") from None
+            if done.returncode not in allowed:
+                raise InstrumentFailure("git_exit_" + str(done.returncode))
+            if self.limit_bytes is not None and self.disk_bytes() > self.limit_bytes:
+                raise InstrumentFailure("clone_size_cap")
+            # Injected runners may return bytes instead of writing the supplied file.
+            if done.stdout is not None:
+                stdout.write(done.stdout)
+            stdout.seek(0)
+            yield stdout
+
+    def run(self, *args: str, allowed=(0,)) -> str:
+        with self.output(*args, allowed=allowed) as stdout:
+            raw = stdout.read(8 * 1024 * 1024 + 1)
+            if len(raw) > 8 * 1024 * 1024:
+                raise InstrumentFailure("git_output_size_cap")
+            self.last_blob_bytes = len(raw)
+            return raw.decode("utf-8", errors="replace")
+
+    def history_entries(self, head):
+        with self.output("log", "--first-parent", "--format=%H%x00%P%x00%ct%x00%an%x00%B%x1e", head) as stdout:
+            pending = b""
+            while chunk := stdout.read(64 * 1024):
+                entries = (pending + chunk).split(b"\x1e")
+                pending = entries.pop()
+                for entry in entries:
+                    if len(entry) > 1024 * 1024:
+                        raise InstrumentFailure("git_history_record_size_cap")
+                    yield entry.decode("utf-8", errors="replace")
+                if len(pending) > 1024 * 1024:
+                    raise InstrumentFailure("git_history_record_size_cap")
+            if pending.strip():
+                raise InstrumentFailure("invalid_history")
+
+    def disk_bytes(self) -> int:
+        return sum(p.stat().st_size for p in self.path.rglob("*") if p.is_file() and not p.is_symlink())
+
+    def parents(self, sha: str) -> list[str]:
+        return self.run("rev-list", "--parents", "-n", "1", sha).split()[1:]
+
+    def blob(self, sha: str, path: str) -> str | None:
+        # Missing is established by a successful tree read, never by swallowing a show error.
+        if not self.run("ls-tree", sha, "--", path).strip():
+            return None
+        size = int(self.run("cat-file", "-s", f"{sha}:{path}").strip())
+        if size > 512 * 1024:
+            raise InstrumentFailure("source_blob_size_cap")
+        text = self.run("show", f"{sha}:{path}")
+        if Path(path).suffix in LANGUAGES:
+            self.files_read += 1
+            self.bytes_read += self.last_blob_bytes
+        return text
+
+    def stats(self, base: str, head: str) -> list[tuple[str, int, int]]:
+        rows = []
+        for line in self.run("diff", "--no-renames", "--numstat", base, head, "--").splitlines():
+            added, removed, path = line.split("\t", 2)
+            rows.append((path, int(added) if added.isdigit() else 0, int(removed) if removed.isdigit() else 0))
+        return rows
+
+    def hunks(self, base: str, head: str, path: str) -> list[tuple[int, int, int, int]]:
+        diff = self.run("diff", "--no-ext-diff", "--no-renames", "-U0", base, head, "--", path)
+        return [(int(m[1]), int(m[2] or 1), int(m[3]), int(m[4] or 1)) for m in HUNK.finditer(diff)]
+
+    def ancestor(self, older: str, newer: str) -> bool:
+        # rev-list avoids interpreting a command failure as a negative ancestry result.
+        return not self.run("rev-list", "--max-count=1", older, "--not", newer).strip()
+
+    def blame(self, revision: str, path: str, line: int) -> tuple[str, str, int]:
+        text = self.run("blame", "-w", "-M", "-C", "--porcelain", f"-L{line},{line}", revision, "--", path)
+        header = text.splitlines()[0].split()
+        if len(header) < 3 or not SHA.fullmatch(header[0]):
+            raise InstrumentFailure("invalid_blame")
+        filename = next((row[9:] for row in text.splitlines() if row.startswith("filename ")), path)
+        if filename.startswith('"'):
+            filename = json.loads(filename)
+        return header[0], filename, int(header[1])
+
+
+def functions(path: str, text: str) -> list[tuple[str, int, int]]:
+    """Conservative syntax boundaries. No scanner import; unresolved sites are rejected."""
+    if Path(path).suffix == ".py":
+        try:
+            tree = ast.parse(text)
+        except SyntaxError:
+            raise Rejected("function_parse") from None
+        return [
+            (n.name, n.lineno, n.end_lineno or n.lineno) for n in ast.walk(tree) if isinstance(n, ast.FunctionDef | ast.AsyncFunctionDef)
+        ]
+    # Blank comments and quoted strings before brace matching. Keep newlines/offsets.
+    masked = re.sub(
+        r'/\*.*?\*/|//[^\n]*|\#[^\n]*|"(?:\\.|[^"\\])*"|\'(?:\\.|[^\'\\])*\'|`(?:\\.|[^`\\])*`',
+        lambda m: "".join("\n" if c == "\n" else " " for c in m[0]),
+        text,
+        flags=re.S,
+    )
+    declaration = re.compile(
+        r"\bfunction\s*&?\s*(\w+)\s*\([^)]*\)\s*(?::[^\n{]+)?\s*\{|"
+        r"\b(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?(?:\([^)]*\)|\w+)\s*=>\s*\{|"
+        r"\b([\w$]+)\s*\([^;{}]*\)\s*(?:throws\s+[\w., ]+)?\s*\{"
+    )
+    found = []
+    for match in declaration.finditer(masked):
+        name = next(g for g in match.groups() if g)
+        if name in {"if", "while", "for", "switch", "catch", "with", "synchronized"}:
+            continue
+        depth = 1
+        end = match.end()
+        while end < len(masked) and depth:
+            depth += (masked[end] == "{") - (masked[end] == "}")
+            end += 1
+        if depth == 0:
+            found.append((name, masked.count("\n", 0, match.start()) + 1, masked.count("\n", 0, end) + 1))
+    return found
+
+
+def enclosing(path: str, text: str, line: int, *, global_ok=False) -> tuple[str, int, int] | None:
+    spans = [s for s in functions(path, text) if s[1] <= line <= s[2]]
+    if spans:
+        return min(spans, key=lambda s: s[2] - s[1])
+    return ("<global>", 1, max(1, len(text.splitlines()))) if global_ok else None
+
+
+def normalized(text: str) -> str:
+    return re.sub(r"\s+", "", text)
+
+
+def site_at(git, revision: str, path: str, name: str):
+    text = git.blob(revision, path)
+    if text is None:
+        return None
+    spans = [("<global>", 1, max(1, len(text.splitlines())))] if name == "<global>" else [s for s in functions(path, text) if s[0] == name]
+    if len(spans) != 1:
+        return None
+    return text, spans[0]
+
+
+def fix_sites(git, fix: str, parent: str, *, global_ok=False) -> list[dict]:
+    sites = []
+    for path, _, _ in git.stats(parent, fix):
+        if Path(path).suffix not in LANGUAGES:
+            continue
+        text = git.blob(parent, path)
+        if text is None:
+            continue
+        lines = text.splitlines()
+        for start, count, _, _ in git.hunks(parent, fix, path):
+            for line in range(max(1, start), max(1, start) + max(1, count)):
+                if line > len(lines):
+                    continue
+                span = enclosing(path, text, line, global_ok=global_ok)
+                if span is None:
+                    continue
+                sink_line = line if count else span[1]
+                sink = lines[sink_line - 1]
+                if not sink.strip() or sink.lstrip().startswith(("#", "//", "/*", "*")):
+                    continue
+                key = (path, span[0])
+                if any((s["path"], s["function"]) == key for s in sites):
+                    continue
+                sites.append({"path": path, "function": span[0], "line": sink_line, "text": sink, "absence": count == 0, "span": span})
+    return sites
+
+
+def version_bracket(git, candidate: Candidate, revision: str) -> bool:
+    if not candidate.first_affected:
+        return True
+    tags = git.run("tag", "--sort=version:refname").splitlines()
+    matching = [t for t in tags if t.removeprefix("v") == candidate.first_affected]
+    if len(matching) != 1:
+        return False
+    affected = matching[0]
+    earlier = [t for t in tags[: tags.index(affected)] if git.ancestor(t, affected)]
+    return bool(earlier) and git.ancestor(revision, affected) and not git.ancestor(revision, earlier[-1])
+
+
+def change_files(git, base: str, head: str) -> list[dict]:
+    return [
+        {"path": path, "head_lines": [[max(1, new), max(1, new + max(count, 1) - 1)] for _, _, new, count in git.hunks(base, head, path)]}
+        for path, _, _ in git.stats(base, head)
+    ]
+
+
+def introducing(git, candidate: Candidate) -> dict:
+    from collections import Counter
+
+    if git.parents(candidate.fix) != [candidate.parent]:
+        raise Rejected("fix_parents")
+    sites = fix_sites(git, candidate.fix, candidate.parent, global_ok=candidate.family == "config_secrets")
+    reasons = Counter()
+
+    def record(method, base, head, path, name):
+        resolved = site_at(git, head, path, name)
+        if resolved is None:
+            return None
+        return {
+            "kind": "vulnerability",
+            "base": base,
+            "head": head,
+            "method": method,
+            "family": candidate.family,
+            "function": name,
+            "path": path,
+            "function_lines": list(resolved[1][1:]),
+            "files": change_files(git, base, head),
+            "post_cutoff": candidate.post_cutoff,
+            "advisory": candidate.advisory,
+            "bias": BIAS[method],
+            "fallback_reasons": dict(reasons),
+        }
+
+    for site in sites:
+        revision, path, line = candidate.parent, site["path"], site["line"]
+        for _ in range(6):  # initial blame plus at most five walk-back steps
+            blamed, blamed_path, blamed_line = git.blame(revision, path, line)
+            parents = git.parents(blamed)
+            if len(parents) != 1:
+                reasons["merge_or_root"] += 1
+                break
+            base = parents[0]
+            head_site = site_at(git, blamed, blamed_path, site["function"])
+            if head_site is None:
+                reasons["unresolved_function"] += 1
+                break
+            parent_text = git.blob(base, blamed_path)
+            if parent_text is not None and site["function"] != "<global>":
+                matching = [span for span in functions(blamed_path, parent_text) if span[0] == site["function"]]
+                if len(matching) > 1:
+                    reasons["ambiguous_parent"] += 1
+                    break
+            old_site = site_at(git, base, blamed_path, site["function"])
+            occurrences = []
+            if old_site:
+                text, (_, start, end) = old_site
+                occurrences = [
+                    i for i, value in enumerate(text.splitlines(), 1) if start <= i <= end and normalized(value) == normalized(site["text"])
+                ]
+            if occurrences:
+                reasons["text_present"] += 1
+                revision, path, line = base, blamed_path, occurrences[0]
+                continue
+            stats = git.stats(base, blamed)
+            if len(stats) > 50 or sum(a + d for _, a, d in stats) > 2000:
+                reasons["size_cap"] += 1
+                break
+            if not version_bracket(git, candidate, blamed):
+                reasons["version_bracket"] += 1
+                break
+            result = record("introducing", base, blamed, blamed_path, site["function"])
+            if result:
+                return result
+        else:
+            reasons["walkback_limit"] += 1
+    for site in sites:
+        # Numeric function extent implements log -L for all five languages, including
+        # languages for which Git has no built-in :function: driver.
+        _, start, end = site["span"]
+        history = git.run("log", "--format=%H", "--no-patch", f"-L{start},{end}:{site['path']}", candidate.parent).splitlines()
+        if not history:
+            continue
+        latest = history[0]
+        parents = git.parents(latest)
+        if len(parents) != 1:
+            continue
+        result = record("last_touch", parents[0], candidate.parent, site["path"], site["function"])
+        if result:
+            return result
+    raise Rejected("no_range")
+
+
+SECURITY = re.compile(r"secur|cve|ghsa|vuln|inject|xss|csrf|ssrf|exploit|sanitiz|escap", re.I)
+DEPENDENCY = re.compile(r"\bbump\b|\bupgrade .* from\b", re.I)
+MANIFESTS = {
+    "package.json",
+    "package-lock.json",
+    "yarn.lock",
+    "pnpm-lock.yaml",
+    "composer.json",
+    "composer.lock",
+    "pom.xml",
+    "build.gradle",
+    "build.gradle.kts",
+    "gradle.lockfile",
+    "pyproject.toml",
+    "setup.py",
+    "setup.cfg",
+    "Pipfile",
+    "Pipfile.lock",
+    "poetry.lock",
+    "uv.lock",
+}
+
+
+def generated_path(path: str) -> bool:
+    return bool({"vendor", "node_modules", "dist"} & set(Path(path).parts)) or path.endswith(".min.js")
+
+
+def manifest(path: str) -> bool:
+    name = Path(path).name
+    return name in MANIFESTS or name.endswith(".lock") or bool(re.fullmatch(r"requirements[^/]*\.txt", name))
+
+
+def ordinary_rejection(row: dict, fixes: set[str], known_functions: set[tuple[str, str]]) -> str | None:
+    if len(row["parents"]) != 1:
+        return "merge_or_root"
+    if row["head"] in fixes:
+        return "advisory_fix"
+    if SECURITY.search(row["message"]):
+        return "security"
+    if re.search(r"^Revert|This reverts commit", row["message"], re.I | re.M):
+        return "revert"
+    if re.search(r"dependabot|renovate|\[bot\]", row["author"], re.I) or DEPENDENCY.search(row["message"]):
+        return "dependency"
+    files = row["files"]
+    if files and all(manifest(f["path"]) for f in files):
+        return "manifest_only"
+    source = [f for f in files if Path(f["path"]).suffix in LANGUAGES and not manifest(f["path"])]
+    if not source:
+        return "no_source"
+    if all(f["generated"] or generated_path(f["path"]) for f in source):
+        return "generated"
+    if any((f["path"], name) in known_functions for f in files for name in f["functions"]):
+        return "vulnerable_function"
+    if any((f["path"], "<global>") in known_functions for f in files):
+        return "vulnerable_function"
+    if sum(f["added"] + f["removed"] for f in source) > 1000:
+        return "size_cap"
+    return None
+
+
+def generated(git, revision: str, path: str, text: str) -> bool:
+    import fnmatch
+
+    if generated_path(path):
+        return True
+    header = "\n".join(text.splitlines()[:20])
+    if re.search(r"(?im)^\s*(?:#|//|/\*|\*|<!--).*?(?:@generated|auto.?generated|generated\s+(?:by|file|code)|do not edit)", header):
+        return True
+    result = False
+    parts = Path(path).parts
+    for index in range(len(parts)):
+        directory = "/".join(parts[:index])
+        attributes = git.blob(revision, (directory + "/" if directory else "") + ".gitattributes") or ""
+        relative = "/".join(parts[index:])
+        for line in attributes.splitlines():
+            tokens = line.split()
+            if not tokens or tokens[0].startswith("#"):
+                continue
+            pattern = tokens[0].lstrip("/")
+            target = relative if "/" in pattern else Path(relative).name
+            if fnmatch.fnmatchcase(target, pattern):
+                for token in tokens[1:]:
+                    if token in {"linguist-generated", "linguist-generated=true"}:
+                        result = True
+                    elif token in {"-linguist-generated", "!linguist-generated", "linguist-generated=false"}:
+                        result = False
+    return result
+
+
+def ordinary_history(git, head: str):
+    """Yield one commit at a time; no repository-wide blob or diff retention."""
+    for entry in git.history_entries(head):
+        if not entry.strip():
+            continue
+        fields = entry.strip().split("\x00")
+        if len(fields) != 5 or not SHA.fullmatch(fields[0]):
+            raise InstrumentFailure("invalid_history")
+        sha, parents, timestamp, author, message = fields
+        row = {"head": sha, "parents": parents.split(), "timestamp": int(timestamp), "author": author, "message": message, "files": []}
+        if len(row["parents"]) != 1:
+            yield row
+            continue
+        base = row["parents"][0]
+        for path, added, removed in git.stats(base, sha):
+            hunks = git.hunks(base, sha, path)
+            info = {
+                "path": path,
+                "added": added,
+                "removed": removed,
+                "head_lines": [[max(1, n), max(1, n + max(c, 1) - 1)] for _, _, n, c in hunks],
+                "functions": [],
+                "generated": False,
+            }
+            if Path(path).suffix in LANGUAGES:
+                # Inspect both sides: deleting a vulnerable function is still touching it.
+                for revision, side in ((base, 0), (sha, 2)):
+                    text = git.blob(revision, path)
+                    if text is None:
+                        continue
+                    spans = functions(path, text)
+                    for hunk in hunks:
+                        start, count = hunk[side : side + 2]
+                        for name, lo, hi in spans:
+                            if max(1, start) <= hi and start + max(count, 1) - 1 >= lo:
+                                info["functions"].append(name)
+                    info["generated"] |= generated(git, revision, path, text)
+            row["files"].append(info)
+        yield row
+
+
+def draw_ordinary(rows, fixes: set[str], known_functions: set[tuple[str, str]], *, seed: int) -> tuple[list[dict], dict]:
+    """Spill eligible history to disk; keep the original seeded stratified draw exactly."""
+    with working_store() as db:
+        return _draw_ordinary(db, rows, fixes, known_functions, seed=seed)
+
+
+def _draw_ordinary(db, rows, fixes, known_functions, *, seed):
+    import random
+    from collections import Counter
+
+    counts = Counter()
+    earliest = latest = None
+    db.execute("CREATE TABLE eligible (head TEXT, timestamp INTEGER, bucket INTEGER, value TEXT, third INTEGER)")
+    for row in rows:
+        timestamp = row["timestamp"]
+        earliest = timestamp if earliest is None else min(earliest, timestamp)
+        latest = timestamp if latest is None else max(latest, timestamp)
+        reason = ordinary_rejection(row, fixes, known_functions)
+        counts[reason or "eligible"] += 1
+        if reason is not None:
+            continue
+        size = sum(f["added"] + f["removed"] for f in row["files"] if Path(f["path"]).suffix in LANGUAGES and not manifest(f["path"]))
+        bucket = 0 if size <= 10 else 1 if size <= 50 else 2 if size <= 200 else 3
+        value = {
+            "kind": "ordinary",
+            "base": row["parents"][0],
+            "head": row["head"],
+            "method": "first_parent",
+            "size_bucket": bucket,
+            "files": [{"path": f["path"], "head_lines": f["head_lines"]} for f in row["files"]],
+        }
+        db.execute("INSERT INTO eligible VALUES (?, ?, ?, ?, NULL)", (row["head"], timestamp, bucket, json.dumps(value)))
+    if counts["eligible"] < 40:
+        failure = Rejected("ordinary_floor")
+        failure.counts = dict(counts)
+        raise failure
+    db.execute("UPDATE eligible SET third = min(2, 3 * (timestamp - ?) / ?)", (earliest, max(1, latest - earliest + 1)))
+    cells = {
+        (third, bucket): count for third, bucket, count in db.execute("SELECT third, bucket, count(*) FROM eligible GROUP BY third, bucket")
+    }
+    if {t for t, _ in cells} != {0, 1, 2}:
+        failure = Rejected("time_thirds")
+        failure.counts = dict(counts)
+        raise failure
+    allocation = Counter()
+    targets = {cell: 12 * count / counts["eligible"] for cell, count in cells.items()}
+    for third in range(3):
+        cell = max(sorted(c for c in cells if c[0] == third), key=lambda c: targets[c])
+        allocation[cell] = 1
+    while sum(allocation.values()) < 12:
+        cell = max(sorted(c for c in cells if allocation[c] < cells[c]), key=lambda c: targets[c] - allocation[c])
+        allocation[cell] += 1
+    db.execute("CREATE INDEX eligible_order ON eligible(third, bucket, head)")
+    rng = random.Random(seed)
+    drawn = []
+    for cell in sorted(cells):
+        for position in rng.sample(range(cells[cell]), allocation[cell]):
+            (value,) = db.execute(
+                "SELECT value FROM eligible WHERE third=? AND bucket=? ORDER BY head LIMIT 1 OFFSET ?", (*cell, position)
+            ).fetchone()
+            drawn.append({**json.loads(value), "time_third": cell[0]})
+    return drawn, dict(counts)

benchmarks/unseen/fixtures/advisory.json

@@ -0,0 +1,9 @@
+{
+  "ghsa_id": "GHSA-test-test-test",
+  "published_at": "2026-06-02T00:00:00Z",
+  "type": "reviewed",
+  "withdrawn_at": null,
+  "cwes": [{"cwe_id": "CWE-89"}],
+  "vulnerabilities": [{"package": {"ecosystem": "pip", "name": "fixture"}}],
+  "references": ["https://github.com/fixture/project/commit/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"]
+}

benchmarks/unseen/fixtures/scorer.json

@@ -0,0 +1,860 @@
+[
+  {
+    "id": "0",
+    "repo": "0",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": [
+            {
+              "cwe": "CWE-89",
+              "path": "app.py",
+              "line": 10
+            }
+          ]
+        }
+      ]
+    }
+  },
+  {
+    "id": "1",
+    "repo": "0",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "2",
+    "repo": "1",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "3",
+    "repo": "1",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "4",
+    "repo": "2",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "5",
+    "repo": "2",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "6",
+    "repo": "3",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "7",
+    "repo": "3",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "8",
+    "repo": "4",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "9",
+    "repo": "4",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "10",
+    "repo": "5",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "11",
+    "repo": "5",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "12",
+    "repo": "6",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "13",
+    "repo": "6",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "14",
+    "repo": "7",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "15",
+    "repo": "7",
+    "kind": "ordinary",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "16",
+    "repo": "8",
+    "kind": "vulnerability",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": [
+            {
+              "cwe": "CWE-89",
+              "path": "app.py",
+              "line": 10
+            }
+          ]
+        }
+      ]
+    }
+  },
+  {
+    "id": "17",
+    "repo": "8",
+    "kind": "vulnerability",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": [
+            {
+              "cwe": "CWE-89",
+              "path": "app.py",
+              "line": 10
+            }
+          ]
+        }
+      ]
+    }
+  },
+  {
+    "id": "18",
+    "repo": "9",
+    "kind": "vulnerability",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  },
+  {
+    "id": "19",
+    "repo": "9",
+    "kind": "vulnerability",
+    "family": "injection",
+    "method": "introducing",
+    "post_cutoff": true,
+    "label_check": "confirmed",
+    "path": "app.py",
+    "function": "handler",
+    "files": [
+      {
+        "path": "app.py",
+        "head_lines": [
+          [
+            10,
+            12
+          ]
+        ]
+      }
+    ],
+    "instrument": {
+      "head_verified": true,
+      "base_verified": true,
+      "changed_bytes": {
+        "app.py": 20
+      },
+      "hook_bytes_read": 20,
+      "hook_exit": 0
+    },
+    "push": {
+      "quick_tier": [
+        {
+          "files": [
+            "app.py"
+          ],
+          "bytes_read": 20,
+          "findings": []
+        }
+      ]
+    }
+  }
+]

benchmarks/unseen/journal.py

@@ -0,0 +1,215 @@
+"""Private, append-only sourcing checkpoints. No identity-bearing data leaves this directory."""
+
+from __future__ import annotations
+
+import fcntl
+import hashlib
+import json
+import math
+import os
+import time
+from contextlib import contextmanager
+from pathlib import Path
+
+from .source import compact_advisory, compact_repository
+from .storage import DiskMap, working_store
+
+MAX_RECORD_BYTES = 16 * 1024 * 1024
+
+
+def compact(stage, value):
+    if stage == "advisories":
+        return {"rows": [compact_advisory(row) for row in value["rows"]], "done": value["done"]}
+    if stage == "resolution" and value is not None:
+        return compact_repository(value)
+    return value
+
+
+STAGES = ("advisories", "resolution", "eligibility", "extraction", "completion")
+
+
+def encoded(value) -> bytes:
+    return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
+
+
+def digest(value) -> str:
+    return hashlib.sha256(encoded(value)).hexdigest()
+
+
+class ResumeMismatch(ValueError):
+    def __init__(self, parameters):
+        self.parameters = parameters
+        super().__init__("resume_inputs_changed")
+
+
+class Stopped(Exception):
+    pass
+
+
+def external(root: Path, path: Path) -> Path:
+    path = path.expanduser().resolve()
+    # Also reject the actual checkout when a different --root is supplied.
+    if any(path.is_relative_to(p.resolve()) for p in (root, Path(__file__).resolve().parents[2])):
+        raise ValueError("cache_inside_repository")
+    return path
+
+
+def atomic(path: Path, payload: bytes) -> None:
+    temporary = path.with_suffix(path.suffix + ".tmp")
+    with temporary.open("wb") as handle:
+        os.chmod(temporary, 0o600)
+        handle.write(payload)
+        handle.flush()
+        os.fsync(handle.fileno())
+    temporary.replace(path)
+    sync_directory(path.parent)
+
+
+def sync_directory(path: Path) -> None:
+    fd = os.open(path, os.O_RDONLY | os.O_DIRECTORY)
+    try:
+        os.fsync(fd)
+    finally:
+        os.close(fd)
+
+
+class Journal:
+    def __init__(self, path: Path, *, max_hours=None, clock=None, rate_limit=None):
+        if max_hours is not None and (not math.isfinite(max_hours) or max_hours < 0):
+            raise ValueError("invalid_max_hours")
+        self.path = path
+        self.clock = clock or time.monotonic
+        self.started = self.clock()
+        self.max_seconds = None if max_hours is None else max_hours * 3600
+        self.rate_limit = rate_limit or (lambda: {})
+        self.rows = {}
+        self.elapsed_before = 0.0
+        self.last = None
+        self.previous_rate = {}
+
+    @contextmanager
+    def open(self, parameters, *, compatible_parameters=None):
+        self.path.mkdir(parents=True, exist_ok=True, mode=0o700)
+        with (self.path / "LOCK").open("a") as lock:
+            try:
+                fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+            except BlockingIOError:
+                raise ValueError("journal_busy") from None
+            metadata = self.path / "inputs.json"
+            migrating = False
+            if metadata.exists():
+                old = json.loads(metadata.read_bytes())
+                # Only current, fixed parameter labels can enter stdout.
+                changed = [key for key in parameters if old.get(key) != parameters[key]]
+                if old.keys() != parameters.keys():
+                    changed.append("parameter_schema")
+                if changed:
+                    if old != compatible_parameters:
+                        raise ResumeMismatch(changed)
+                    migrating = True
+            else:
+                if any(self.path.glob("*.jsonl")):
+                    raise ValueError("journal_missing_inputs")
+                atomic(metadata, encoded(parameters))
+            with working_store(self.path) as self.db:
+                self.rows = {stage: DiskMap(self.db, stage) for stage in STAGES}
+                self._replay()
+                if migrating:
+                    atomic(self.path / "inputs-before-memory-fix.json", encoded(old))
+                    atomic(metadata, encoded(parameters))
+                self.progress("sourcing")
+                yield self
+
+    def _replay(self):
+        for stage in STAGES:
+            path = self.path / (stage + ".jsonl")
+            if not path.exists():
+                continue
+            with path.open("rb") as handle:
+                while line := handle.readline(MAX_RECORD_BYTES + 1):
+                    if len(line) > MAX_RECORD_BYTES:
+                        raise ValueError("journal_record_size_cap")
+                    # Fail closed on torn/corrupt writes; never silently lose a completed unit.
+                    if not line.endswith(b"\n"):
+                        raise ValueError("journal_incomplete_record")
+                    row = json.loads(line)
+                    if row["id"] in self.rows[stage]:
+                        raise ValueError("journal_duplicate_unit")
+                    self.rows[stage][row["id"]] = compact(stage, row["value"])
+                    if row["elapsed_seconds"] >= self.elapsed_before:
+                        self.elapsed_before = row["elapsed_seconds"]
+                        self.last = {"stage": stage, "id": row["id"]}
+                        self.previous_rate = row["rate_limit"]
+
+    def boundary(self):
+        reason = None
+        if (self.path / "STOP").exists():
+            reason = "STOP"
+        elif self.max_seconds is not None and self.clock() - self.started >= self.max_seconds:
+            reason = "max_hours"
+        if reason:
+            self.progress("stopped", reason=reason)
+            raise Stopped(reason)
+
+    def elapsed(self):
+        return self.elapsed_before + self.clock() - self.started
+
+    def progress(self, status, **extra):
+        atomic(
+            self.path / "progress.json",
+            encoded(
+                {
+                    "status": status,
+                    "counts": {stage: len(rows) for stage, rows in self.rows.items()},
+                    "last_unit": self.last,
+                    "rate_limit": self.rate_limit() or self.previous_rate,
+                    "elapsed_seconds": self.elapsed(),
+                    **extra,
+                }
+            ),
+        )
+
+    def put(self, stage, key, value):
+        if key in self.rows[stage]:
+            raise ValueError("journal_duplicate_unit")
+        value = compact(stage, value)
+        row = {"id": key, "value": value, "elapsed_seconds": self.elapsed(), "rate_limit": self.rate_limit() or self.previous_rate}
+        path = self.path / (stage + ".jsonl")
+        with path.open("ab") as handle:
+            os.chmod(path, 0o600)
+            payload = encoded(row) + b"\n"
+            if len(payload) > MAX_RECORD_BYTES:
+                raise ValueError("journal_record_size_cap")
+            handle.write(payload)
+            handle.flush()
+            os.fsync(handle.fileno())
+        sync_directory(self.path)
+        self.rows[stage][key] = value
+        self.last = {"stage": stage, "id": key}
+        self.progress("sourcing")
+
+
+class ResolvedGitHub:
+    """Persist successful repository lookups and 404s used by both eligibility and the fix index."""
+
+    def __init__(self, github, journal):
+        self.github, self.journal = github, journal
+
+    def __getattr__(self, name):
+        return getattr(self.github, name)
+
+    def get_repo(self, name):
+        from .eligibility import RepositoryNotFound, normalize
+
+        key = normalize(name)
+        if key not in self.journal.rows["resolution"]:
+            self.journal.boundary()
+            try:
+                value = self.github.get_repo(key)
+            except RepositoryNotFound:
+                value = None
+            self.journal.put("resolution", key, value)
+        value = self.journal.rows["resolution"][key]
+        if value is None:
+            raise RepositoryNotFound("not_found")
+        return value

benchmarks/unseen/ledger.py

@@ -0,0 +1,88 @@
+"""Append-only slice usage; a hash chain detects rewritten rows, not a rebuilt entire history.
+
+Keep the published final digest to authenticate a history against truncation or wholesale replacement.
+Only opaque decision/experiment labels belong here; repository identities are never accepted.
+"""
+
+from __future__ import annotations
+
+import fcntl
+import hashlib
+import json
+import re
+from datetime import date
+from pathlib import Path
+
+FIELDS = {"slice", "decision", "purpose", "date", "result_digest", "freeze_digest"}
+PURPOSES = {"baseline", "exploratory", "informed", "qualifies"}
+ZERO = "0" * 64
+
+
+def digest(row: dict) -> str:
+    return hashlib.sha256(json.dumps(row, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
+
+
+def validate(row: dict) -> None:
+    if set(row) != FIELDS or row["purpose"] not in PURPOSES:
+        raise ValueError("invalid_usage_fields")
+    if type(row["slice"]) is not int or row["slice"] < 1:
+        raise ValueError("invalid_slice")
+    if not isinstance(row["decision"], str) or not re.fullmatch(r"[A-Za-z0-9_-]+", row["decision"]):
+        raise ValueError("invalid_decision_label")
+    for key in ("result_digest", "freeze_digest"):
+        if not isinstance(row[key], str) or not re.fullmatch(r"[0-9a-f]{64}", row[key]):
+            raise ValueError("invalid_digest")
+    try:
+        if date.fromisoformat(row["date"]).isoformat() != row["date"]:
+            raise ValueError
+    except (ValueError, TypeError):
+        raise ValueError("invalid_date") from None
+
+
+def _read(text: str) -> list[dict]:
+    previous = ZERO
+    rows = []
+    if text and not text.endswith("\n"):
+        raise ValueError("broken_digest_chain")
+    for line in text.splitlines():
+        try:
+            row = json.loads(line)
+            stored = row.pop("digest")
+            if row["previous"] != previous or digest(row) != stored:
+                raise ValueError
+            validate({k: v for k, v in row.items() if k != "previous"})
+        except (ValueError, KeyError, TypeError, AttributeError):
+            raise ValueError("broken_digest_chain") from None
+        rows.append({**row, "digest": stored})
+        previous = stored
+    return rows
+
+
+def read(path: Path) -> list[dict]:
+    if not path.exists():
+        return []
+    with path.open() as handle:
+        fcntl.flock(handle, fcntl.LOCK_SH)
+        return _read(handle.read())
+
+
+def append(path: Path, row: dict) -> str:
+    validate(row)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with path.open("a+") as handle:
+        fcntl.flock(handle, fcntl.LOCK_EX)
+        handle.seek(0)
+        rows = _read(handle.read())
+        if any(all(old[key] == row[key] for key in FIELDS) for old in rows):
+            raise ValueError("duplicate_usage")
+        if rows and rows[0]["freeze_digest"] != row["freeze_digest"]:
+            raise ValueError("freeze_digest_changed")
+        if row["purpose"] == "qualifies" and any(
+            old["slice"] == row["slice"] and old["decision"] == row["decision"] and old["purpose"] == "informed" for old in rows
+        ):
+            raise ValueError("slice_informed_decision")
+        entry = {**row, "previous": rows[-1]["digest"] if rows else ZERO}
+        sha = digest(entry)
+        handle.write(json.dumps({**entry, "digest": sha}, sort_keys=True) + "\n")
+        handle.flush()
+        return sha

benchmarks/unseen/proof.py

@@ -0,0 +1,128 @@
+"""Build anonymous coordinator proof records from completed dispatcher outputs."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import subprocess
+from pathlib import Path
+
+
+def digest(value):
+    return hashlib.sha256(json.dumps(value, sort_keys=True).encode()).hexdigest()
+
+
+def records(root):
+    return [json.loads(path.read_text()) for path in sorted(root.glob("*.json")) if path.name not in {"progress.json", "inputs.json"}]
+
+
+def replay_proof(roots):
+    lanes = {lane: records(root) for lane, root in roots.items()}
+    if any(len(rows) != 5 for rows in lanes.values()):
+        raise ValueError("proof requires exactly five completed changes per lane")
+    keys = [{row["input_digest"] for row in rows} for rows in lanes.values()]
+    if not all(key == keys[0] for key in keys):
+        raise ValueError("lane input identities differ")
+    quick = {}
+    evidence = {}
+    for lane, rows in lanes.items():
+        quick[lane] = {}
+        evidence[lane] = []
+        for row in rows:
+            inst = row.get("instrument", {})
+            if (
+                row.get("status") not in {"quick_only", "engine_covered"}
+                or not inst.get("hook_bytes_read")
+                or inst.get("suspect_fast")
+                or inst.get("jvm_wall_seconds", 0) < 5
+            ):
+                raise ValueError("proof instrument has unread input or implausible JVM time")
+            quick[lane][row["input_digest"]] = digest([t.get("findings", []) for t in row["push"].get("quick_tier", [])])
+            evidence[lane].append(
+                {
+                    "input_digest": row["input_digest"],
+                    "seconds": inst["seconds"],
+                    "jvm_seconds": inst["jvm_wall_seconds"],
+                    "bytes": inst["hook_bytes_read"],
+                    "exit_code": inst["hook_exit"],
+                }
+            )
+    equal = all(value == next(iter(quick.values())) for value in quick.values())
+    if not equal:
+        raise ValueError("quick-tier findings differ across lanes")
+    return {"lanes": evidence, "quick_tier_equal": True, "quick_digests": quick, "cleanup": "completed-before-checkpoint"}
+
+
+def engine_proof(root, baseline):
+    current = records(root)
+    old = {digest([r["repo"], r["pin"]]): r for r in records(baseline) if "repo" in r and "pin" in r}
+    current = [r for r in current if "repo" in r and "pin" in r]
+    if len(current) != 2:
+        raise ValueError("engine proof requires exactly two pins")
+    evidence = []
+
+    def stable(record):
+        return [
+            {key: u.get(key) for key in ("unit", "status", "questions_asked", "questions_completed", "witness_rows", "traces")}
+            for u in record["units"]
+        ]
+
+    for row in current:
+        key = digest([row["repo"], row["pin"]])
+        previous = old[key]
+        instruments = [u.get("instrument", {}) for u in row["units"]]
+        if (
+            not row.get("done")
+            or not any(i.get("bytes", 0) > 0 for i in instruments)
+            or not any(stage.get("seconds", 0) >= 5 for i in instruments for stage in i.get("jvm", []))
+        ):
+            raise ValueError("engine proof lacks bytes or realistic JVM time")
+        evidence.append(
+            {
+                "pin_digest": key,
+                "seconds": row.get("seconds"),
+                "record_digest": digest(stable(row)),
+                "baseline_digest": digest(stable(previous)),
+                "equal": stable(row) == stable(previous),
+                "instrument": [
+                    {
+                        "bytes": u.get("instrument", {}).get("bytes", 0),
+                        "cpg_bytes": u.get("instrument", {}).get("cpg_bytes", 0),
+                        "jvm": [
+                            {key: stage.get(key) for key in ("seconds", "exit_code", "success", "heap_mb")}
+                            for stage in u.get("instrument", {}).get("jvm", [])
+                        ],
+                    }
+                    for u in row["units"]
+                ],
+            }
+        )
+    return {"pins": evidence, "differences": sum(not e["equal"] for e in evidence), "cleanup": "completed-before-checkpoint"}
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("kind", choices=("engine", "replay"))
+    parser.add_argument("--image", required=True)
+    parser.add_argument("--out", type=Path, required=True)
+    parser.add_argument("--records", type=Path)
+    parser.add_argument("--baseline", type=Path)
+    parser.add_argument("--kind-records", type=Path)
+    parser.add_argument("--ax-records", type=Path)
+    parser.add_argument("--vm-records", type=Path)
+    args = parser.parse_args()
+    result = (
+        engine_proof(args.records, args.baseline)
+        if args.kind == "engine"
+        else replay_proof({"kind": args.kind_records, "ax": args.ax_records, "vm": args.vm_records})
+    )
+    result.update(
+        image=args.image, commit=subprocess.run(["git", "rev-parse", "HEAD"], capture_output=True, text=True, check=True).stdout.strip()
+    )
+    args.out.parent.mkdir(parents=True, exist_ok=True)
+    args.out.write_text(json.dumps(result, sort_keys=True, indent=2) + "\n")
+
+
+if __name__ == "__main__":
+    main()

benchmarks/unseen/replay.py

@@ -0,0 +1,193 @@
+"""Dispatch frozen unseen changes or a five-change project-history proof."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import io
+import json
+import subprocess
+import tarfile
+import tomllib
+from pathlib import Path
+
+from benchmarks.ax.batch import AXLane, DockerLane, Item, Workload, dispatch, write_json
+from benchmarks.unseen.score import coverage
+
+
+def make_item(change, arm, image):
+    if arm not in {"default", "long_deadline"}:
+        raise ValueError("unknown replay arm")
+    identity = hashlib.sha256(str(change["id"]).encode()).hexdigest()[:14]
+    spec = {**change, "id": identity, "hook_flags": [] if arm == "default" else ["--deadline", "300"], "image": image}
+    data = json.dumps(spec, sort_keys=True).encode()
+    digest = hashlib.sha256(data).hexdigest()
+    return Item(
+        "replay-" + identity,
+        {"SPEC_URL": data},
+        "result.dat",
+        ("python3", "/app/benchmarks/unseen/replay_entry.py"),
+        {},
+        digest,
+        {"SPEC_URL": "spec.dat"},
+    )
+
+
+def validate_result(data, output):
+    with tarfile.open(fileobj=io.BytesIO(data)) as archive:
+        members = archive.getmembers()
+        required = {"push.json", "push.txt", "engine.json", "instrument.json"}
+        if {m.name for m in members} != required or any(not m.isfile() for m in members):
+            raise ValueError("replay result requires push, engine and instrument.json")
+        payload = {m.name: archive.extractfile(m).read() for m in members}
+    instrument = json.loads(payload["instrument.json"])
+    push = json.loads(payload["push.json"])
+    record = {"instrument": instrument, "push": push}
+    record["status"] = coverage(record)
+    # Transport/scheduler needs an explicit OOM signal, not a quiet failed replay.
+    if "OutOfMemoryError" in json.dumps(push) or "JVM out of memory" in json.dumps(push):
+        record["status"] = "oom"
+    output.mkdir(parents=True, exist_ok=True)
+    for name, body in payload.items():
+        (output / name).write_bytes(body)
+    (output / "result.dat").write_bytes(data)
+    write_json(output / "result.json", record)
+    return record
+
+
+def proof_changes(root):
+    """Only the operator's project history; no pool data or fetches."""
+
+    def git(*args):
+        return subprocess.run(["git", "-C", str(root), *args], capture_output=True, text=True, check=True).stdout.strip()
+
+    changes = []
+    for head in git("rev-list", "--no-merges", "--max-count=100", "HEAD", "--", "src").splitlines():
+        base = git("rev-parse", head + "^")
+        changed = git("diff", "--name-only", base, head).splitlines()
+        if not any(name.startswith("src/") and name.endswith(".py") for name in changed):
+            continue
+        changes.append(
+            {
+                "id": hashlib.sha256(head.encode()).hexdigest(),
+                "repository_url": "https://github.com/norandom/OpenUltraSAST.git",
+                "repo": "project-proof",
+                "kind": "ordinary",
+                "base": base,
+                "head": head,
+            }
+        )
+        if len(changes) == 5:
+            break
+    if len(changes) != 5:
+        raise ValueError("history lacks five source changes")
+    return changes
+
+
+def canonical(value):
+    return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
+
+
+def load_slice(path, slice_id, private_path=None):
+    if path.suffix != ".toml":
+        frozen = json.loads(path.read_text())
+        changes = frozen["changes"]
+        if frozen["sha256"] != hashlib.sha256(json.dumps(changes, sort_keys=True).encode()).hexdigest():
+            raise ValueError("frozen changes digest mismatch")
+        return changes
+    if not path.name.startswith("pool-"):
+        raise ValueError("replay requires a frozen pool, never a draft")
+    pool = tomllib.loads(path.read_text())
+    freeze = json.loads(path.with_name("freeze-" + path.stem.removeprefix("pool-") + ".json").read_text())
+    if hashlib.sha256(canonical(pool)).hexdigest() != freeze["freeze_digest"]:
+        raise ValueError("frozen pool digest mismatch")
+    private_path = private_path or path.parent / "private" / path.name
+    private = {}
+    if private_path.exists():
+        private = {row["id"]: row for row in tomllib.loads(private_path.read_text()).get("repository", [])}
+    changes = []
+    for public in pool.get("repository", []):
+        if str(public["slice"]) != str(slice_id):
+            continue
+        row = public
+        if public.get("private"):
+            row = private.get(public["id"])
+            if row is None:
+                raise ValueError("missing private pool entry")
+            payload = {"url": row["url"], "commits": [[c["base"], c["head"]] for c in row["changes"]]}
+            if hashlib.sha256(canonical(payload)).hexdigest() != public["sha256"]:
+                raise ValueError("private pool digest mismatch")
+        for change in row["changes"]:
+            identity = hashlib.sha256(canonical([public["id"], change["kind"], change["base"], change["head"]])).hexdigest()
+            changes.append(
+                {
+                    **change,
+                    "id": identity,
+                    "repo": public["id"],
+                    "repository_url": row["url"],
+                    "freeze_digest": freeze["freeze_digest"],
+                    "slice": slice_id,
+                }
+            )
+    if not changes:
+        raise ValueError("frozen slice has no changes")
+    return changes
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--manifest", type=Path, required=True, help="frozen JSON: changes array plus sha256 of canonical changes JSON")
+    parser.add_argument("--proof-history", action="store_true", help="write a five-change own-history manifest, then exit")
+    parser.add_argument("--image")
+    parser.add_argument("--private-manifest", type=Path)
+    parser.add_argument("--dry-run", action="store_true")
+    parser.add_argument("--arm", choices=("default", "long_deadline"), default="default")
+    parser.add_argument("--pool", default="p1")
+    parser.add_argument("--slice", default="1")
+    parser.add_argument("--out", type=Path)
+    parser.add_argument("--lane", choices=("ax", "kind", "docker", "mixed"), default="mixed")
+    parser.add_argument("--ax-bin", default="ax")
+    parser.add_argument("--kubeconfig", type=Path)
+    parser.add_argument("--atespace", default="default")
+    args = parser.parse_args()
+    if args.proof_history:
+        changes = proof_changes(Path.cwd())
+        write_json(args.manifest, {"changes": changes, "sha256": hashlib.sha256(json.dumps(changes, sort_keys=True).encode()).hexdigest()})
+        return 0
+    if not args.image:
+        parser.error("--image digest is required")
+    changes = load_slice(args.manifest, args.slice, args.private_manifest)
+    items = [make_item(change, args.arm, args.image) for change in changes]
+    if len({i.id for i in items}) != len(items):
+        parser.error("duplicate change identity")
+    args.out = args.out or Path.home() / "ousast-results/unseen" / args.pool / args.slice / args.arm
+    if args.dry_run:
+        print(json.dumps({"items": len(items), "input_digest": hashlib.sha256(canonical([i.input_digest for i in items])).hexdigest()}))
+        return 0
+    if args.out.resolve().is_relative_to(Path(__file__).resolve().parents[2]):
+        parser.error("raw replay output must be outside the repository")
+    write_json(
+        args.out / "inputs.json",
+        [{"item_id": item.id, "change": change, "input_digest": item.input_digest} for item, change in zip(items, changes, strict=True)],
+    )
+    workload = Workload(args.image, frozenset({"SPEC_URL", "RESULT_URL"}), 600 if args.arm == "default" else 900, validate_result)
+    lanes = ("ax", "ax", "docker") if args.lane == "mixed" else (args.lane,) * (2 if args.lane in {"ax", "kind"} else 1)
+    executors = {}
+    for lane in set(lanes):
+        transport = (DockerLane if lane == "docker" else AXLane)(args, workload)
+
+        def execute(item, output, attempt, transport=transport):
+            record = {**json.loads(item.inputs["SPEC_URL"]), **transport(item, output, attempt)}
+            if (output / "result.dat").exists():
+                # Durable archive lives separately from attempt staging objects, which are deleted.
+                key = f"unseen/{item.input_digest}/result.dat"
+                transport.store._put(key, (output / "result.dat").read_bytes())
+                record["result_object"] = key
+            return record
+
+        executors[lane] = execute
+    return 2 if dispatch(items, workload, args.out, executors, lanes=lanes) else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/unseen/replay_entry.py

@@ -0,0 +1,219 @@
+"""Credential-free replay actor. Standard library only; PUT is completion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+import tarfile
+import time
+import urllib.error
+import urllib.request
+from pathlib import Path
+from urllib.parse import urlsplit
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "learn"))
+from engine_task_entry import transfer
+
+CLUSTER_FLAGS = (
+    "-Xms1700m -Xmx1700m -XX:MaxMetaspaceSize=256m -XX:ReservedCodeCacheSize=128m "
+    "-XX:MaxDirectMemorySize=256m -Xss512k -XX:ActiveProcessorCount=2 -XX:+UseG1GC"
+)
+
+
+class ReplayFailure(Exception):
+    """Locally controlled diagnostic only."""
+
+
+def validate_spec(spec):
+    parts = urlsplit(spec["repository_url"])
+    if (
+        parts.scheme != "https"
+        or not parts.hostname
+        or "." not in parts.hostname
+        or parts.username
+        or parts.password
+        or parts.query
+        or parts.fragment
+    ):
+        raise ReplayFailure("repository must be public HTTPS")
+    if parts.port not in (None, 443) or parts.hostname.endswith((".local", ".internal", ".localhost")):
+        raise ReplayFailure("repository must be public HTTPS")
+    import ipaddress
+
+    try:
+        ipaddress.ip_address(parts.hostname)
+    except ValueError:
+        pass
+    else:
+        raise ReplayFailure("repository must use public DNS")
+    if any(not re.fullmatch("[0-9a-f]{40,64}", spec[key]) for key in ("base", "head")):
+        raise ReplayFailure("invalid revision")
+    if spec.get("hook_flags", []) not in ([], ["--deadline", "300"]):
+        raise ReplayFailure("unsupported hook flags")
+
+
+def main(*, work_root=Path("/tmp"), runner=subprocess.run, heap_profile="cluster", clock=time.monotonic):
+    output = work_root / "out"
+    output.mkdir(parents=True, exist_ok=True)
+    instrument = {
+        "status": "instrument_failure",
+        "head_verified": False,
+        "base_verified": False,
+        "changed_bytes": {},
+        "checkout_files": 0,
+        "checkout_bytes": 0,
+        "hook_bytes_read": 0,
+        "jvm_wall_seconds": 0,
+        "jvm_time_basis": "scan build_seconds + query_seconds",
+        "hook_exit": None,
+        "timings": {},
+        "heap_profile": heap_profile,
+    }
+    spec = {}
+    step = "GET spec"
+    started = clock()
+
+    def run(command, *, timeout=180, env=None):
+        before = clock()
+        result = runner(command, capture_output=True, text=True, timeout=timeout, env=env)
+        instrument["timings"][step] = instrument["timings"].get(step, 0) + clock() - before
+        if result.returncode:
+            raise ReplayFailure(f"exit {result.returncode}")
+        return result
+
+    try:
+        spec_path = work_root / "spec.dat"
+        transfer(os.environ["SPEC_URL"], spec_path, readiness_budget=60)
+        step = "validate spec"
+        spec = json.loads(spec_path.read_text())
+        instrument["image"] = spec.get("image")
+        validate_spec(spec)
+        case = work_root / "case"
+        case.mkdir()
+        git = ["git", "-c", "core.hooksPath=/dev/null", "-c", "protocol.file.allow=never", "-C", str(case)]
+        step = "git init"
+        run([*git, "init", "-q"])
+        run([*git, "remote", "add", "origin", spec["repository_url"]])
+        # One total fetch budget, independent of the hook deadline.
+        fetch_end = clock() + 180
+        for key in ("head", "base"):
+            step = "fetch " + key
+            remaining = fetch_end - clock()
+            if remaining <= 0:
+                raise ReplayFailure("fetch deadline")
+            run([*git, "fetch", "--no-tags", "--depth", "1", "origin", spec[key]], timeout=remaining)
+            step = "verify " + key
+            run([*git, "cat-file", "-e", spec[key] + "^{commit}"])
+            instrument[key + "_verified"] = True
+        step = "checkout"
+        run([*git, "checkout", "--detach", spec["head"]])
+        paths = run([*git, "ls-files", "-z"]).stdout.split("\0")
+        for name in filter(None, paths):
+            path = case / name
+            if path.is_file() and not path.is_symlink():
+                data = path.read_bytes()
+                instrument["checkout_files"] += 1
+                instrument["checkout_bytes"] += len(data)
+                if instrument["checkout_bytes"] > 500 * 1024 * 1024:
+                    raise ReplayFailure("checkout size limit")
+        step = "changed files"
+        changed = run([*git, "diff", "--name-only", "--diff-filter=ACMRT", "-z", spec["base"], spec["head"]]).stdout.split("\0")
+        for name in filter(None, changed):
+            path = case / name
+            if path.is_file() and not path.is_symlink() and path.resolve().is_relative_to(case.resolve()):
+                instrument["changed_bytes"][name] = len(path.read_bytes())
+        if not instrument["checkout_bytes"] or not sum(instrument["changed_bytes"].values()):
+            instrument["suspect_fast"] = True
+            raise ReplayFailure("no changed bytes read")
+        step = "pre-push"
+        env = {
+            key: value for key, value in os.environ.items() if key not in {"SPEC_URL", "RESULT_URL", "_JAVA_OPTIONS", "JDK_JAVA_OPTIONS"}
+        }
+        env["OPENULTRASAST_CPG_HEAP_MB"] = "1700" if heap_profile == "cluster" else "2560"
+        env["JAVA_TOOL_OPTIONS"] = CLUSTER_FLAGS if heap_profile == "cluster" else "-Xmx2560m"
+        command = [
+            "ousast",
+            "pre-push",
+            str(case),
+            "--base",
+            spec["base"],
+            "--head",
+            spec["head"],
+            "--artifact",
+            str(output / "push.json"),
+            *spec.get("hook_flags", []),
+        ]
+        before = clock()
+        completed = runner(command, capture_output=True, text=True, timeout=330 if spec.get("hook_flags") else 60, env=env)
+        instrument["timings"][step] = clock() - before
+        instrument["hook_exit"] = completed.returncode
+        (output / "push.txt").write_text(completed.stdout)
+        if completed.returncode not in (0, 1):
+            raise ReplayFailure(f"hook exit {completed.returncode}")
+        step = "read push artifact"
+        push = json.loads((output / "push.json").read_text())
+        instrument["hook_bytes_read"] = sum(t.get("bytes_read", 0) for t in push.get("quick_tier", []))
+        scans = push.get("scans", [])
+        instrument["jvm_wall_seconds"] = sum(s.get("scan", {}).get(k, 0) for s in scans for k in ("build_seconds", "query_seconds"))
+        uncovered = "language_not_covered" in json.dumps(push.get("admission", {}).get("coverage_reasons", []))
+        partitions = [part for scan in scans for part in scan.get("scan", {}).get("partitions", [])]
+        unsupported_only = (
+            uncovered
+            and instrument["hook_bytes_read"] == 0
+            and bool(partitions)
+            and all(part.get("status") == "unsupported" for part in partitions)
+        )
+        instrument["suspect_fast"] = bool(scans) and not unsupported_only and instrument["jvm_wall_seconds"] < 5
+        if instrument["suspect_fast"] or (instrument["hook_bytes_read"] <= 0 and not uncovered):
+            instrument["suspect_fast"] = True
+            raise ReplayFailure("suspect_fast or unread hook input")
+        engine = {
+            "repository": spec["repository_url"],
+            "revisions": {key: spec[key] for key in ("base", "head")},
+            "scans": scans,
+            "admission": push.get("admission", {}),
+        }
+        (output / "engine.json").write_text(json.dumps(engine))
+        instrument["status"] = "ok"
+    except Exception as exc:
+        detail = (
+            f"HTTP {exc.code}"
+            if isinstance(exc, urllib.error.HTTPError)
+            else str(exc)
+            if isinstance(exc, ReplayFailure)
+            else type(exc).__name__
+        )
+        instrument["failure"] = {"step": step, "status": detail}
+        # A crash artifact must never be mistaken for a completed scan.
+        if not (output / "push.json").exists():
+            (output / "push.json").write_text("{}")
+        if not (output / "push.txt").exists():
+            (output / "push.txt").write_text("")
+        if not (output / "engine.json").exists():
+            (output / "engine.json").write_text("{}")
+    instrument["seconds"] = clock() - started
+    (output / "instrument.json").write_text(json.dumps(instrument))
+    try:
+        archive = work_root / "result.dat"
+        with tarfile.open(archive, "w") as tar:
+            for name in ("push.json", "push.txt", "engine.json", "instrument.json"):
+                tar.add(output / name, arcname=name)
+        request = urllib.request.Request(
+            os.environ["RESULT_URL"], data=archive.read_bytes(), method="PUT", headers={"Content-Type": "application/x-tar"}
+        )
+        transfer(request, readiness_budget=60)
+    except Exception as exc:
+        detail = f"HTTP {exc.code}" if isinstance(exc, urllib.error.HTTPError) else type(exc).__name__
+        print("PUT result: " + detail, file=sys.stderr)
+        return 1
+    return int(instrument["status"] != "ok")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--heap-profile", choices=("cluster", "vm"), default="cluster")
+    raise SystemExit(main(heap_profile=parser.parse_args().heap_profile))

benchmarks/unseen/score.py

@@ -0,0 +1,308 @@
+"""Push-level unseen evaluation, explicit coverage and paired adoption evidence."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import random
+from collections import Counter, defaultdict
+from pathlib import Path
+
+from openultrasast.model.taxonomy import load_families
+from openultrasast.push.report import CAPABILITY_VETOES
+
+STATES = ("instrument_failure", "unanalysable", "quick_only", "engine_covered")
+
+
+def coverage(record):
+    instrument = record.get("instrument", {})
+    push = record.get("push")
+    if (
+        not isinstance(push, dict)
+        or not instrument.get("head_verified")
+        or not instrument.get("base_verified")
+        or instrument.get("suspect_fast")
+        or instrument.get("failure")
+        or instrument.get("hook_exit") not in (0, 1)
+        or sum(instrument.get("changed_bytes", {}).values()) <= 0
+    ):
+        return "instrument_failure"
+    if "language_not_covered" in json.dumps([push.get("skipped", []), push.get("admission", {}).get("coverage_reasons", [])]) and not any(
+        tier.get("bytes_read", 0) for tier in push.get("quick_tier", [])
+    ):
+        return "unanalysable"
+    if instrument.get("hook_bytes_read", 0) <= 0:
+        return "instrument_failure"
+    changed = set(instrument["changed_bytes"])
+    if any(
+        outcome.get("status") == "completed" and outcome.get("identity", {}).get("path") in changed
+        for scan in push.get("scans", [])
+        if scan.get("side") == "head"
+        for outcome in scan.get("scan", {}).get("question_outcomes", [])
+    ):
+        return "engine_covered"
+    return "quick_only"
+
+
+def findings(record):
+    push = record.get("push", {})
+    admission = push.get("admission", {})
+    for defect in admission.get("defects", []):
+        for path, line in defect.get("locations", []):
+            yield "alerts", {**defect, "path": path, "line": line}
+    seen = set()
+    for disposition in admission.get("dispositions", []):
+        delta = disposition.get("candidate", {}).get("delta", {})
+        operation = delta.get("head_operation")
+        if (
+            disposition.get("admitted")
+            or delta.get("novelty") not in {"new", "worsened"}
+            or not set(disposition.get("reasons", [])) <= CAPABILITY_VETOES
+            or operation is None
+            or delta.get("witness") is None
+            or delta.get("defect_id") in seen
+        ):
+            continue
+        seen.add(delta.get("defect_id"))
+        yield "advisory", {**operation, "family": delta.get("family")}
+    taxonomy = load_families()
+    for tier in push.get("quick_tier", []):
+        for finding in tier.get("findings", []):
+            family = taxonomy.family_of_cwe(finding.get("cwe") or "")
+            yield "quick", {**finding, "family": family.id if family else None}
+
+
+def location_match(record, finding):
+    for file in record.get("files", []):
+        if finding.get("path") != file["path"]:
+            continue
+        line = finding.get("line", 0)
+        if any(start - 10 <= line <= end + 10 for start, end in file.get("head_lines", [])):
+            return True
+        if record.get("family") != "config_secrets" and file["path"] == record.get("path", file["path"]):
+            function = record.get("function")
+            if function and function != "<global>" and finding.get("function") == function:
+                return True
+            span = record.get("function_lines") or file.get("function_lines")
+            if span and span[0] <= line <= span[1]:
+                return True
+    return False
+
+
+def catches(record):
+    return sorted(
+        {
+            stream
+            for stream, finding in findings(record)
+            if finding.get("family") == record.get("family") and location_match(record, finding)
+        }
+    )
+
+
+def false_alarm(record, *, alerts_only=False):
+    return any(not alerts_only or stream == "alerts" for stream, _ in findings(record))
+
+
+def wilson(successes, n):
+    if n == 0:
+        return [None, None]
+    p, z = successes / n, 1.96
+    centre = (p + z * z / (2 * n)) / (1 + z * z / n)
+    half = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
+    return [max(0, centre - half), min(1, centre + half)]
+
+
+def required_n(p, width):
+    for n in range(1, 1000000):
+        low, high = wilson(p * n, n)
+        if max(p - low, high - p) <= width:
+            return n
+    raise ValueError("requested precision exceeds sizing range")
+
+
+def rate(values):
+    n, k = len(values), sum(values)
+    return {"n": n, "count": k, "rate": k / n if n else None, "wilson95": wilson(k, n)}
+
+
+def cluster_interval(rows, *, seed=0, draws=2000):
+    clusters = defaultdict(list)
+    for repo, value in rows:
+        clusters[repo].append(value)
+    if not clusters:
+        return [None, None]
+    rng = random.Random(seed)
+    groups = list(clusters.values())
+    samples = []
+    for _ in range(draws):
+        values = [value for group in rng.choices(groups, k=len(groups)) for value in group]
+        samples.append(sum(values) / len(values))
+    samples.sort()
+    return [samples[int(0.025 * (draws - 1))], samples[int(0.975 * (draws - 1))]]
+
+
+def icc(rows):
+    clusters = defaultdict(list)
+    for repo, value in rows:
+        clusters[repo].append(value)
+    n, k = len(rows), len(clusters)
+    if k < 2 or n <= k:
+        return None
+    mean = sum(v for _, v in rows) / n
+    between = sum(len(v) * (sum(v) / len(v) - mean) ** 2 for v in clusters.values()) / (k - 1)
+    within = sum(sum((x - sum(v) / len(v)) ** 2 for x in v) for v in clusters.values()) / (n - k)
+    size = (n - sum(len(v) ** 2 for v in clusters.values()) / n) / (k - 1)
+    denominator = between + (size - 1) * within
+    return (between - within) / denominator if denominator else 0.0
+
+
+def mcnemar(a, b):
+    pairs = list(zip(a, b, strict=True))
+    losses = sum(x and not y for x, y in pairs)
+    gains = sum(y and not x for x, y in pairs)
+    n = losses + gains
+    return min(1.0, 2 * sum(math.comb(n, i) for i in range(min(losses, gains) + 1)) / 2**n) if n else 1.0
+
+
+def score(records, *, seed=0):
+    counts = Counter(coverage(r) for r in records)
+    if not records or counts["instrument_failure"] / len(records) > 0.05:
+        raise ValueError("slice exceeds 5% instrument_failure (or is empty)")
+    usable = [r for r in records if coverage(r) != "instrument_failure"]
+
+    def metrics(rows):
+        ordinary = [r for r in rows if r["kind"] == "ordinary"]
+        vulnerable = [r for r in rows if r["kind"] != "ordinary"]
+        clustered = [(r["repo"], int(false_alarm(r))) for r in ordinary]
+        return {
+            "catch": rate([bool(catches(r)) for r in vulnerable]),
+            "false_alarm": {
+                **rate([false_alarm(r) for r in ordinary]),
+                "cluster95": cluster_interval(clustered, seed=seed),
+                "icc": icc(clustered),
+            },
+            "alerts_only": rate([false_alarm(r, alerts_only=True) for r in ordinary]),
+        }
+
+    breakdown = {}
+    for key in ("family", "method", "post_cutoff"):
+        groups = defaultdict(list)
+        for r in usable:
+            groups[str(r.get(key, "unknown"))].append(r)
+        breakdown[key] = {key: metrics(rows) for key, rows in sorted(groups.items())}
+    breakdown["coverage"] = {state: metrics([r for r in usable if coverage(r) == state]) for state in STATES[1:]}
+    breakdown["stream"] = {
+        stream: rate([stream in catches(r) for r in usable if r["kind"] != "ordinary"]) for stream in ("alerts", "advisory", "quick")
+    }
+    return {
+        **metrics(usable),
+        "total": len(records),
+        "coverage": {state: counts[state] for state in STATES},
+        "breakdowns": breakdown,
+        "confirmed": metrics([r for r in usable if r.get("label_check", r.get("confirmed")) in ("confirmed", True)]),
+    }
+
+
+def budgets(rows, *, thresholds=None, binary=False):
+    ordinary = [r for r in rows if r["kind"] == "ordinary"]
+    vulnerable = [r for r in rows if r["kind"] != "ordinary"]
+    if binary:
+        fa = rate([r["prediction"] for r in ordinary])
+        return {
+            "catch": rate([r["prediction"] for r in vulnerable]),
+            "false_alarm": fa,
+            "budgets_met": [f"{budget}%" for budget in (1, 5, 10) if fa["rate"] is not None and fa["rate"] <= budget / 100],
+        }
+    output = {}
+    for budget in (1, 5, 10):
+        key = f"{budget}%"
+        if thresholds is not None and key not in thresholds:
+            continue
+        if not ordinary:
+            output[key] = {"label": "not_run"}
+            continue
+        if thresholds is not None:
+            threshold = thresholds[key]
+        else:
+            candidates = sorted({r["score"] for r in rows})
+            candidates.append(math.nextafter(candidates[-1], math.inf))
+            threshold = next(t for t in candidates if sum(r["score"] >= t for r in ordinary) / len(ordinary) <= budget / 100)
+        output[key] = {
+            "threshold": threshold,
+            "label": "fixed" if thresholds is not None else "exploratory",
+            "catch": rate([r["score"] >= threshold for r in vulnerable]),
+            "false_alarm": rate([r["score"] >= threshold for r in ordinary]),
+        }
+    return output
+
+
+def gate(a, b, *, slice_id=None, informed=False, binary=True, threshold=None, threshold_fixed=False, seed=0):
+    not_run = {"gate": "not_run"}
+    if not slice_id or informed or not a or not b or (not binary and (not threshold_fixed or threshold is None)):
+        return not_run
+    try:
+        score(a, seed=seed)
+        score(b, seed=seed)
+    except ValueError:
+        return not_run
+    aa, bb = {r["id"]: r for r in a}, {r["id"]: r for r in b}
+    if len(aa) != len(a) or len(bb) != len(b) or aa.keys() != bb.keys():
+        return not_run
+    pairs = [
+        (aa[key], bb[key]) for key in sorted(aa) if coverage(aa[key]) != "instrument_failure" and coverage(bb[key]) != "instrument_failure"
+    ]
+    if any(x["kind"] != y["kind"] or x["repo"] != y["repo"] for x, y in pairs):
+        return not_run
+    vulnerable = [(x, y) for x, y in pairs if x["kind"] != "ordinary"]
+    ordinary = [(x, y) for x, y in pairs if x["kind"] == "ordinary"]
+    if not vulnerable or not ordinary:
+        return not_run
+    ca = [bool(catches(x)) for x, _ in vulnerable]
+    cb = [bool(catches(y)) if binary else y["score"] >= threshold for _, y in vulnerable]
+    fa = [false_alarm(x) for x, _ in ordinary]
+    fb = [false_alarm(y) if binary else y["score"] >= threshold for _, y in ordinary]
+    catch_p, fa_p = mcnemar(ca, cb), mcnemar(fa, fb)
+    regression = (sum(cb) < sum(ca) and catch_p < 0.05) or (binary and sum(fb) > sum(fa) and fa_p < 0.05)
+    return {
+        "gate": "regression" if regression else "pass",
+        "slice": slice_id,
+        "catch": {"difference": sum(cb) / len(cb) - sum(ca) / len(ca), "mcnemar_p": catch_p},
+        "false_alarm": {
+            "difference": sum(fb) / len(fb) - sum(fa) / len(fa),
+            "mcnemar_p": fa_p,
+            "cluster95": cluster_interval([(x["repo"], int(v) - int(u)) for (x, _), u, v in zip(ordinary, fa, fb, strict=True)], seed=seed),
+        },
+    }
+
+
+def load_records(path):
+    if path.is_file():
+        return json.loads(path.read_text())
+    rows = []
+    for entry in json.loads((path / "inputs.json").read_text()):
+        result = path / (hashlib.sha256(entry["item_id"].encode()).hexdigest() + ".json")
+        record = json.loads(result.read_text()) if result.exists() else {}
+        if record.get("input_digest") != entry["input_digest"]:
+            record = {}
+        rows.append({**entry["change"], **record, "id": entry["change"]["id"]})
+    return rows
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("records", type=Path, help="JSON array of joined change/instrument/push records")
+    parser.add_argument("--out", type=Path, required=True)
+    parser.add_argument("--against", type=Path, help="production-arm records for a paired adoption gate")
+    parser.add_argument("--slice-id")
+    parser.add_argument("--informed", action="store_true")
+    args = parser.parse_args()
+    current = load_records(args.records)
+    result = score(current)
+    result["unseen"] = gate(load_records(args.against) if args.against else [], current, slice_id=args.slice_id, informed=args.informed)
+    args.out.write_text(json.dumps(result, sort_keys=True, indent=2) + "\n")
+
+
+if __name__ == "__main__":
+    main()

benchmarks/unseen/source.py

@@ -0,0 +1,305 @@
+"""Advisory selection. Network adapters are constructed only by the coordinator CLI."""
+
+from __future__ import annotations
+
+import base64
+import json
+import random
+import re
+from collections import OrderedDict
+from dataclasses import dataclass
+from datetime import date
+from urllib.parse import urlencode, urlsplit
+
+from openultrasast.model.taxonomy import load_families
+
+from .eligibility import RepositoryNotFound, normalize
+
+ECOSYSTEMS = ("pip", "npm", "maven", "composer")
+OSV_ECOSYSTEMS = dict(zip(ECOSYSTEMS, ("PyPI", "npm", "Maven", "Packagist"), strict=True))
+FAMILIES = {"injection", "path", "deserialization", "access_control", "output_encoding", "untrusted_destination", "config_secrets"}
+PERMISSIVE = {"MIT", "Apache-2.0", "BSD-2-Clause", "BSD-3-Clause", "ISC", "0BSD", "Unlicense", "Zlib", "PostgreSQL", "BSL-1.0"}
+FIX = re.compile(r"https://github\.com/([\w.-]+/[\w.-]+)/commit/([a-fA-F0-9]{7,40})(?:[/?#].*)?$", re.I)
+
+
+class Rejected(ValueError):
+    """Expected eligibility rejection; messages are fixed reason codes, never names."""
+
+
+@dataclass(frozen=True)
+class Candidate:
+    repository: str
+    url: str
+    fix: str
+    parent: str
+    advisory: str
+    ecosystem: str
+    family: str
+    post_cutoff: bool
+    first_affected: str | None
+    license_spdx: str
+    license_class: str
+    license_ref: str
+    license_path: str
+
+
+def compact_advisory(row: dict) -> dict:
+    """Keep exactly the input fields used by selection and the all-advisory fix index."""
+    return {
+        **{key: row[key] for key in ("ghsa_id", "published_at", "type", "withdrawn_at") if key in row},
+        "cwes": [{"cwe_id": c["cwe_id"]} for c in row.get("cwes", [])],
+        "references": [f"https://github.com/{name}/commit/{sha}" for name, sha in sorted(fix_links(row))],
+        "vulnerabilities": [{"package": {key: v["package"][key] for key in ("ecosystem", "name")}} for v in row.get("vulnerabilities", [])],
+    }
+
+
+def compact_repository(row: dict) -> dict:
+    return {
+        **{key: row[key] for key in ("full_name", "private", "size", "clone_url", "former_names") if key in row},
+        **{key: {"full_name": row[key]["full_name"]} for key in ("parent", "source") if row.get(key)},
+    }
+
+
+def post_cutoff(row: dict) -> bool:
+    return date.fromisoformat(row["published_at"][:10]) > date(2026, 6, 1)
+
+
+def fix_links(row: dict) -> set[tuple[str, str]]:
+    links = set()
+    for reference in row.get("references", []):
+        url = reference if isinstance(reference, str) else reference.get("url", "")
+        match = FIX.fullmatch(url)
+        if match:
+            links.add((normalize(match[1]), match[2].lower()))
+    return links
+
+
+def ecosystem(row: dict) -> str | None:
+    values = {v["package"]["ecosystem"].lower() for v in row.get("vulnerabilities", [])}
+    return next((e for e in ECOSYSTEMS if e in values), None)
+
+
+def ordered_advisories(rows: list[dict], seed: int) -> list[dict]:
+    """Stable seeded ordering, post-cutoff first in each ecosystem; round-robin strata."""
+    from .storage import AdvisoryRows
+
+    if isinstance(rows, AdvisoryRows):
+        return rows.ordered(seed)
+    rng = random.Random(seed)
+    groups = []
+    for eco in ECOSYSTEMS:
+        group = sorted((r for r in rows if ecosystem(r) == eco), key=lambda r: r["ghsa_id"])
+        rng.shuffle(group)
+        group.sort(key=lambda r: not post_cutoff(r))
+        groups.append(group)
+    return [group[i] for i in range(max(map(len, groups), default=0)) for group in groups if i < len(group)]
+
+
+def first_affected(record: dict, row: dict, eco: str) -> str | None:
+    packages = {v["package"]["name"] for v in row["vulnerabilities"] if v["package"]["ecosystem"].lower() == eco}
+    versions = set()
+    for affected in record.get("affected", []):
+        package = affected.get("package", {})
+        if package.get("ecosystem") != OSV_ECOSYSTEMS[eco] or package.get("name") not in packages:
+            continue
+        for span in affected.get("ranges", []):
+            if span.get("type") not in {"ECOSYSTEM", "SEMVER"}:
+                continue
+            for event in span.get("events", []):
+                if event.get("introduced") not in {None, "0"}:
+                    versions.add(event["introduced"])
+    if len(versions) > 1:
+        raise Rejected("ambiguous_version_bracket")
+    return next(iter(versions), None)
+
+
+def license_info(record: dict) -> tuple[str, str]:
+    # Explicit pinned-file SPDX wins. For GitHub-recognized full texts its license endpoint
+    # provides the detected SPDX. Unknown/custom/compound declarations stay private.
+    declared = re.search(r"SPDX-License-Identifier:\s*([^\r\n]+)", record.get("text", ""))
+    spdx = declared[1].strip().removesuffix("*/").strip() if declared else record.get("spdx_id") or "NONE"
+    return spdx, "permissive" if spdx in PERMISSIVE else "private"
+
+
+def candidate(row: dict, github, osv) -> Candidate:
+    if date.fromisoformat(row["published_at"][:10]) < date(2019, 1, 1):
+        raise Rejected("date")
+    if row.get("type", "reviewed") != "reviewed" or row.get("withdrawn_at"):
+        raise Rejected("advisory_status")
+    eco = ecosystem(row)
+    if eco is None:
+        raise Rejected("ecosystem")
+    taxonomy = load_families()
+    families = {f.id for c in row.get("cwes", []) if (f := taxonomy.family_of_cwe(c["cwe_id"])) and f.id in FAMILIES}
+    if len(families) != 1:
+        raise Rejected("family")
+    links = fix_links(row)
+    if len(links) != 1:
+        raise Rejected("fix_links")
+    name, fix = next(iter(links))
+    repo = github.get_repo(name)
+    url = urlsplit(repo.get("clone_url", ""))
+    if (
+        repo.get("private", True)
+        or not 0 <= repo.get("size", -1) <= 500 * 1024
+        or url.scheme != "https"
+        or url.hostname != "github.com"
+        or url.username
+        or url.password
+        or url.query
+        or url.fragment
+        or normalize(repo["clone_url"]) != normalize(repo["full_name"])
+    ):
+        raise Rejected("repository")
+    commit = github.get_commit(repo["full_name"], fix)
+    if len(commit["parents"]) != 1:
+        raise Rejected("parents")
+    parent = commit["parents"][0]["sha"]
+    license_record = github.get_license(repo["full_name"], parent)
+    spdx, license_class = license_info(license_record)
+    return Candidate(
+        normalize(repo["full_name"]),
+        repo["clone_url"],
+        commit["sha"],
+        parent,
+        row["ghsa_id"],
+        eco,
+        next(iter(families)),
+        post_cutoff(row),
+        first_affected(osv.get(row["ghsa_id"]), row, eco),
+        spdx,
+        license_class,
+        parent,
+        license_record.get("path", ""),
+    )
+
+
+class JSONTransport:
+    """Coordinator GET transport with bounded retries; all I/O and waits are injectable."""
+
+    def __init__(self, *, opener=None, sleeper=None, clock=None):
+        import time
+        from urllib.request import urlopen
+
+        self.opener = opener or urlopen
+        self.sleeper = sleeper or time.sleep
+        self.clock = clock or time.time
+        self.attempts = 0
+        self.rate_limit = {}
+
+    def get(self, url: str, headers: dict) -> object:
+        from urllib.error import HTTPError
+        from urllib.request import Request
+
+        for attempt in range(5):
+            self.attempts += 1
+            try:
+                with self.opener(Request(url, headers=headers), timeout=60) as response:
+                    self.record_rate(getattr(response, "headers", {}))
+                    return json.load(response)
+            except HTTPError as exc:
+                self.record_rate(exc.headers)
+                if exc.code == 404:
+                    raise RepositoryNotFound("not_found") from None
+                limited = exc.code == 429 or (
+                    exc.code == 403 and (exc.headers.get("X-RateLimit-Remaining") == "0" or exc.headers.get("Retry-After"))
+                )
+                transient = exc.code in {500, 502, 503, 504}
+                if attempt == 4 or not (limited or transient):
+                    raise ValueError("http_failure") from None
+                try:
+                    delay = float(exc.headers.get("Retry-After", "0"))
+                    if limited and not delay:
+                        delay = max(60, float(exc.headers.get("X-RateLimit-Reset", "0")) - self.clock() + 1)
+                    delay = max(delay, 2**attempt)
+                    if delay > 3605:
+                        raise ValueError("retry_window")
+                except (TypeError, ValueError):
+                    raise ValueError("http_retry_failed") from None
+                self.sleeper(delay)
+            except Exception:
+                raise ValueError("http_failure") from None
+        raise ValueError("http_failure")
+
+    def record_rate(self, headers):
+        self.rate_limit = {
+            key: headers.get(header)
+            for key, header in (
+                ("remaining", "X-RateLimit-Remaining"),
+                ("limit", "X-RateLimit-Limit"),
+                ("reset", "X-RateLimit-Reset"),
+                ("retry_after", "Retry-After"),
+            )
+            if headers.get(header) is not None
+        }
+
+
+class GitHub:
+    def __init__(self, token: str, transport=None):
+        if not token:
+            raise ValueError("missing_gh_token")
+        self.transport = transport or JSONTransport()
+        self.headers = {"Authorization": "Bearer " + token, "Accept": "application/vnd.github+json"}
+        self.repos: OrderedDict[str, dict] = OrderedDict()
+        self.calls = 0
+
+    def get(self, path: str):
+        self.calls += 1
+        return self.transport.get("https://api.github.com" + path, self.headers)
+
+    def get_repo(self, name: str) -> dict:
+        name = normalize(name)
+        if name not in self.repos:
+            self.repos[name] = compact_repository(self.get("/repos/" + name))
+            if len(self.repos) > 128:
+                self.repos.popitem(last=False)
+        self.repos.move_to_end(name)
+        return self.repos[name]
+
+    def get_commit(self, name: str, sha: str) -> dict:
+        return self.get("/repos/" + normalize(name) + "/commits/" + sha)
+
+    def get_license(self, name: str, ref: str) -> dict:
+        try:
+            row = self.get("/repos/" + normalize(name) + "/license?" + urlencode({"ref": ref}))
+        except RepositoryNotFound:
+            return {"spdx_id": "NONE", "text": "", "path": ""}
+        return {
+            "spdx_id": (row.get("license") or {}).get("spdx_id"),
+            "path": row.get("path", ""),
+            "text": base64.b64decode(row.get("content", "")).decode("utf-8", errors="replace"),
+        }
+
+    def advisory_page(self, page: int) -> dict:
+        # Oldest first: newly published advisories do not shift already persisted pages.
+        rows = self.get(
+            "/advisories?" + urlencode({"type": "reviewed", "per_page": 100, "page": page, "sort": "published", "direction": "asc"})
+        )
+        if not isinstance(rows, list):
+            raise ValueError("invalid_advisory_page")
+        return {"rows": rows, "done": len(rows) < 100}
+
+    def advisories(self):
+        # Enumerate every reviewed advisory, including non-candidate fixes.
+        page = 1
+        while True:
+            result = self.advisory_page(page)
+            yield from result["rows"]
+            if result["done"]:
+                break
+            page += 1
+
+
+class OSV:
+    def __init__(self, transport=None):
+        self.transport = transport or JSONTransport()
+        self.calls = 0
+
+    def get(self, identifier: str) -> dict:
+        if not re.fullmatch(r"GHSA-[\w-]+", identifier):
+            raise ValueError("invalid_advisory_id")
+        self.calls += 1
+        try:
+            return self.transport.get("https://api.osv.dev/v1/vulns/" + identifier, {})
+        except RepositoryNotFound:
+            return {}  # No OSV bracket is distinct from a failed service call.

benchmarks/unseen/storage.py

@@ -0,0 +1,170 @@
+"""Disposable private SQLite working sets; JSONL remains the durable source of truth."""
+
+from __future__ import annotations
+
+import json
+import sqlite3
+import tempfile
+from collections.abc import Mapping, Sequence, Set
+from contextlib import contextmanager
+from pathlib import Path
+
+
+@contextmanager
+def working_store(directory=None):
+    # TemporaryDirectory is mode 0700. No identity-bearing scratch data enters the checkout.
+    with tempfile.TemporaryDirectory(prefix="sourcing-", dir=directory) as temporary:
+        db = sqlite3.connect(Path(temporary) / "working.sqlite")
+        try:
+            db.execute("PRAGMA journal_mode=OFF")
+            db.execute("PRAGMA synchronous=OFF")
+            db.execute("PRAGMA temp_store=FILE")
+            db.execute("PRAGMA cache_size=-2048")
+            db.execute("PRAGMA mmap_size=0")
+            db.execute("CREATE TABLE records (kind TEXT, key TEXT, value TEXT, PRIMARY KEY (kind, key)) WITHOUT ROWID")
+            db.execute("CREATE TABLE sequences (kind TEXT, position INTEGER, value TEXT, PRIMARY KEY (kind, position)) WITHOUT ROWID")
+            yield db
+        finally:
+            db.close()
+
+
+class DiskMap(Mapping):
+    def __init__(self, db, kind):
+        self.db, self.kind = db, kind
+        self.count = 0
+
+    def __len__(self):
+        return self.count
+
+    def __iter__(self):
+        for (key,) in self.db.execute("SELECT key FROM records WHERE kind=? ORDER BY key", (self.kind,)):
+            yield key
+
+    def __contains__(self, key):
+        return self.db.execute("SELECT 1 FROM records WHERE kind=? AND key=?", (self.kind, key)).fetchone() is not None
+
+    def __getitem__(self, key):
+        row = self.db.execute("SELECT value FROM records WHERE kind=? AND key=?", (self.kind, key)).fetchone()
+        if row is None:
+            raise KeyError(key)
+        return json.loads(row[0])
+
+    def __setitem__(self, key, value):
+        if key not in self:
+            self.count += 1
+        self.db.execute("INSERT OR REPLACE INTO records VALUES (?, ?, ?)", (self.kind, key, json.dumps(value)))
+
+    def values(self):
+        for (value,) in self.db.execute("SELECT value FROM records WHERE kind=? ORDER BY key", (self.kind,)):
+            yield json.loads(value)
+
+
+class DiskSequence(Sequence):
+    def __init__(self, db, kind):
+        self.db, self.kind = db, kind
+        self.count = 0
+
+    def __len__(self):
+        return self.count
+
+    def __getitem__(self, position):
+        if position < 0:
+            position += self.count
+        row = self.db.execute("SELECT value FROM sequences WHERE kind=? AND position=?", (self.kind, position)).fetchone()
+        if row is None:
+            raise IndexError(position)
+        return json.loads(row[0])
+
+    def __setitem__(self, position, value):
+        self.db.execute("INSERT OR REPLACE INTO sequences VALUES (?, ?, ?)", (self.kind, position, json.dumps(value)))
+
+    def append(self, value):
+        self[self.count] = value
+        self.count += 1
+
+    def __iter__(self):
+        for (value,) in self.db.execute("SELECT value FROM sequences WHERE kind=? ORDER BY position", (self.kind,)):
+            yield json.loads(value)
+
+
+class DiskSet(Set):
+    def __init__(self, db, kind):
+        self.rows = DiskMap(db, kind)
+
+    def __len__(self):
+        return len(self.rows)
+
+    def __contains__(self, value):
+        return json.dumps(value) in self.rows
+
+    def __iter__(self):
+        for key in self.rows:
+            value = json.loads(key)
+            yield tuple(value) if isinstance(value, list) else value
+
+    def add(self, value):
+        self.rows[json.dumps(value)] = None
+
+    def update(self, values):
+        for value in values:
+            self.add(value)
+
+
+class AdvisoryRows(DiskMap):
+    def __iter__(self):
+        return self.values()
+
+    def ordered(self, seed):
+        import random
+
+        from .source import ECOSYSTEMS, ecosystem
+
+        rng = random.Random(seed)
+        groups = [DiskSequence(self.db, "order_" + eco) for eco in ECOSYSTEMS]
+        for row in self.values():  # key order is the original ghsa_id sort
+            eco = ecosystem(row)
+            if eco in ECOSYSTEMS:
+                groups[ECOSYSTEMS.index(eco)].append(row)
+        for group in groups:
+            rng.shuffle(group)  # Same Fisher-Yates draws as the in-memory implementation.
+        cursors = [
+            self.db.execute(
+                "SELECT value FROM sequences WHERE kind=? "
+                "ORDER BY substr(json_extract(value, '$.published_at'), 1, 10) <= '2026-06-01', position",
+                (group.kind,),
+            )
+            for group in groups
+        ]
+        for _ in range(max(map(len, groups), default=0)):
+            for cursor in cursors:
+                row = cursor.fetchone()
+                if row is not None:
+                    yield json.loads(row[0])
+
+
+class FixIndex:
+    def __init__(self, db):
+        self.db = db
+        db.execute("CREATE TABLE fixes (repository TEXT, sha TEXT, PRIMARY KEY (repository, sha)) WITHOUT ROWID")
+
+    def add(self, repository, sha):
+        self.db.execute("INSERT OR IGNORE INTO fixes VALUES (?, ?)", (repository, sha))
+
+    def get(self, repository, default=None):
+        fixes = RepositoryFixes(self.db, repository)
+        return fixes if fixes else default
+
+
+class RepositoryFixes(Set):
+    def __init__(self, db, repository):
+        self.db, self.repository = db, repository
+
+    def __len__(self):
+        return self.db.execute("SELECT count(*) FROM fixes WHERE repository=?", (self.repository,)).fetchone()[0]
+
+    def __iter__(self):
+        for (sha,) in self.db.execute("SELECT sha FROM fixes WHERE repository=? ORDER BY sha", (self.repository,)):
+            yield sha
+
+    def __contains__(self, sha):
+        return self.db.execute("SELECT 1 FROM fixes WHERE repository=? AND sha=?", (self.repository, sha)).fetchone() is not None

docs/architecture.md

@@ -1,14 +1,22 @@
 # Architecture
 
-How a scan works, how a finding earns its place, how severity and the project score are
-decided, and how the ruleset improves itself. State as of 2026-10-02; module paths are under
-`src/openultrasast/`.
+This page covers four things:
+
+- how a scan works;
+- how a finding earns its place;
+- how severity and the project score are decided;
+- how the ruleset improves itself.
+
+State as of 2026-10-02; module paths are under `src/openultrasast/`.
 
 ## The scan pipeline
 
-The CLI is thin; `HarnessRuntime` owns the lifecycle and writes an event trace for every stage
-(`trace/events.jsonl`). The stages a mode requests are `MODE_STAGES` in `stages.py`:
-`quick` runs STATIC, `standard` adds MAP, `deep` adds REGRESS.
+The CLI is thin. `HarnessRuntime` owns the lifecycle. It writes an event trace for every stage
+(`trace/events.jsonl`). `MODE_STAGES` in `stages.py` lists the stages each mode requests:
+
+- `quick` runs STATIC;
+- `standard` adds MAP;
+- `deep` adds REGRESS.
 
 ```
 policy_load ─▶ ruleset_load ─▶ policy_check            (fail loud on an ungoverned CWE)
@@ -28,14 +36,19 @@ policy_load ─▶ ruleset_load ─▶ policy_check            (fail loud on an
 ```
 
 **The model layer** (`model/`, `cpg/`) is the core of `standard`. It builds one code property
-graph per scan with Joern (`joern-parse`, then batched `joern --script` queries from
-`cpg/queries/*.sc`) and dispatches each question to the arbiter of its spec type: taint
-(source to sink with no discharging sanitizer), dominance (does a guard govern the obligated
-operation) and configuration (constant abstraction over a value). Taint facts ship for C,
-Java, JavaScript, PHP and Python (`ruleset/semantic/*.toml`). Without Joern the layer records
-`cpg_unavailable` and the scan is otherwise unchanged; the Docker image ships Joern and
-`php-cli` (which the PHP frontend needs). The representations and techniques behind each stage
-are explained in [Detection techniques](detection-techniques.md).
+graph (CPG: a graph of the program's syntax, control flow and data flow) per scan with Joern.
+It runs `joern-parse`, then batched `joern --script` queries from `cpg/queries/*.sc`. Each
+question goes to the arbiter (a checker that decides one kind of question) for its spec type:
+
+- taint (untrusted input reaching a dangerous call): source to sink with no discharging sanitizer;
+- dominance: does a guard govern the obligated operation;
+- configuration: constant abstraction over a value.
+
+Taint facts ship for C, Java, JavaScript, PHP and Python (`ruleset/semantic/*.toml`). Without
+Joern, the layer records `cpg_unavailable` and the rest of the scan is unchanged. The Docker
+image ships Joern and `php-cli`, which the PHP frontend needs. [Detection
+techniques](detection-techniques.md) explains the representations and techniques behind each
+stage.
 
 The model layer reports on a rung ladder:
 
@@ -47,23 +60,29 @@ model_corroborated    the graph found the shape; a judge agreed on the residual
 suspicion             proposed, nothing corroborated it
 ```
 
-The LLM (`model/pipeline.py`, `residual_question`) is asked only the residual question the graph
-cannot settle, for a flow the graph already found; it can confirm or contradict, never create a
-flow. When the graph resolved nothing, the enumerator's candidates (at most
-`MAX_JUDGED_CANDIDATES`, 8) are put to it, and anything it affirms is reported at `suspicion`.
-Without a provider key neither question is asked and only the graph's entailed findings are
-reported.
+The LLM answers only the residual question: the part the graph cannot settle
+(`model/pipeline.py`, `residual_question`). It is asked only about a flow the graph already
+found. It can confirm or contradict that flow, but never create one. When the graph resolved
+nothing, the LLM sees the enumerator's candidates (at most `MAX_JUDGED_CANDIDATES`, 8).
+Anything it affirms is reported at `suspicion`. Without a provider key, neither question is
+asked. Only the graph's entailed findings are reported then.
 
 **REGRESS** (`regress/`, `sandbox/`) runs only when `docker info` succeeds. Each promoted
-candidate gets a small language recipe (Python, JavaScript, C) that loads the candidate's
-source inside a container with no network, a read-only `/workspace`, a non-root user and the
-`[sandbox]` limits. A structural deny-list rejects snippets mentioning sockets, curl, host
-networking or writes under `/workspace` before anything runs.
+candidate gets a small language recipe (Python, JavaScript, C). The recipe loads the
+candidate's source inside a container with:
+
+- no network;
+- a read-only `/workspace`;
+- a non-root user;
+- the `[sandbox]` limits.
+
+Before anything runs, a structural deny-list rejects snippets that mention sockets, curl, host
+networking or writes under `/workspace`.
 
 ## Evidence ladder: how a false positive is eliminated
 
 Evidence level is a state machine, not a label a model may assign. The verifier
-(`verification.py`) runs on independent context and enforces:
+(`verification.py`) runs on independent context and enforces it:
 
 ```
 suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_explained
@@ -75,31 +94,36 @@ suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_
   call-graph, route, CLI, parser or dynamic evidence;
 - `static_corroboration` with function-level reachability: `ACCEPTED`.
 
-A rejected finding becomes a **scoped false-positive learning** (`calibration.py`) with a
-reason from a fixed taxonomy (`unreachable_path`, `missing_attacker_control`,
+A rejected finding becomes a **scoped false-positive learning** (`calibration.py`). It has a
+scope and a reason from a fixed taxonomy: `unreachable_path`, `missing_attacker_control`,
 `sanitizer_disproved`, `static_rule_mismatch`, `incorrect_model_assumption`, `duplicate`,
-`insufficient_impact`, `unsupported`, `contradicted`, `unverified`) and a scope. A rejection in
-`auth/` demotes only `auth/...`, never the whole class
+`insufficient_impact`, `unsupported`, `contradicted`, `unverified`. A rejection in `auth/`
+demotes only `auth/...`, never the whole class
 (`tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`).
 
 ## How a false negative is surfaced
 
-Every expected vulnerability that no finding matched becomes a `BenchmarkCalibrationRecord` in
-`calibration_records.json`, with a `next_improvement_candidate` (a static rule, SARIF source,
-entry-point mapping, retrieval package, hunter prompt, dynamic reproducer or skill route).
-Once a miss is closed the detection gate keeps it closed in CI.
+An expected vulnerability that no finding matched becomes a `BenchmarkCalibrationRecord` in
+`calibration_records.json`. Each record names a `next_improvement_candidate`: a static rule,
+SARIF source, entry-point mapping, retrieval package, hunter prompt, dynamic reproducer or
+skill route. Once a miss is closed, the detection gate keeps it closed in CI.
 
 ## Central CWE policy and the project score
 
 **Policy** (`policy/verycode.py`): a vendored `CWE_Score.tsv` (verycode-policies) is the single
-source of truth. Each CWE carries a flaw category, a severity (0-5) and `static`/`dynamic` scope
-flags. A finding's severity comes only from policy, keyed on CWE. CWEs that are not `static`
-resolve to 0 and are report-only. `assert_rules_resolve` runs as the `policy_check` stage: an
-enabled rule naming an ungoverned CWE aborts the scan. Verycode has no CWE-120, so the C/C++
+source of truth. Each CWE carries:
+
+- a flaw category;
+- a severity (0-5);
+- `static`/`dynamic` scope flags.
+
+A finding's severity comes only from policy, keyed on CWE. CWEs that are not `static` resolve
+to 0 and are report-only. `assert_rules_resolve` runs as the `policy_check` stage. An enabled
+rule that names an ungoverned CWE aborts the scan. Verycode has no CWE-120, so the C/C++
 buffer rules carry CWE-121.
 
 **Score** (`scoring/project_score.py`): each finding's penalty is
-`SEV_WEIGHT[severity] x REACH_MULT[reachability]`, and the score is `100 * e^(-total/k)` with
+`SEV_WEIGHT[severity] x REACH_MULT[reachability]`. The score is `100 * e^(-total/k)` with
 `k = 60`.
 
 | Severity | Weight | | Reachability | Multiplier |
@@ -109,10 +133,13 @@ buffer rules carry CWE-121.
 | 3 | 10 | | `unknown` | 0.4 |
 | 2 / 1 / 0 | 2 / 1 / 0 | | | |
 
-A confirmed false positive lowers a finding's reachability multiplier instead of deleting the
-rule. A severity-5 reachable finding always fails the score gate; `score < min_score`
-(default 80) fails it only with `[score] blocking = true`. The `score` stage writes
-`score.json` and merges it into `manifest.json`.
+A confirmed false positive lowers a finding's reachability multiplier. The rule is not
+deleted. The score gate works like this:
+
+- a severity-5 reachable finding always fails it;
+- `score < min_score` (default 80) fails it only with `[score] blocking = true`.
+
+The `score` stage writes `score.json` and merges it into `manifest.json`.
 
 ## Self-improvement
 
@@ -124,41 +151,61 @@ scan ─▶ verify ─▶ non-accepted findings ─▶ scoped learnings
 next scan ─▶ rank ─▶ calibrate (demote rejected scopes) ─▶ findings
 ```
 
-`record_calibration` (after `verify`) merges this run's non-accepted outcomes into the ledger,
-de-duplicated by finding ID; `calibrate` (after `rank`) demotes the priority of every scope
-that produced a rejection and writes `applied_calibrations.json`. Demotion is scoped and
-reversible (`tests/test_pipeline_calibration.py`).
+1. `record_calibration` runs after `verify`. It merges this run's non-accepted outcomes into
+   the ledger, de-duplicated by finding ID.
+2. `calibrate` runs after `rank`. It demotes the priority of every scope that produced a
+   rejection and writes `applied_calibrations.json`.
+
+Demotion is scoped and reversible (`tests/test_pipeline_calibration.py`).
 
 ### Rule-level loop: `ousast improve`
 
-Each round (`improve/evolve.py`): benchmark with the current ledger, per-rule signals
-(`rule_signals.json`), bounded proposals, validation (`improve/validator.py`), replay smoke,
-re-benchmark, and a hard gate. The levers are rule status (`enabled -> shadow` before
-`disabled`) and score constants; the loop can never edit pattern text
-or the authoritative CWE severity. A round is accepted only if recall >= 90%, FP < 10%, the
-project score did not regress and no matched finding was lost; otherwise the ledger is not
-written. With `--pair-catalog`, every round is also gated per provenance profile on the
-catalog's holdout pairs (`--no-pair-gate` skips this and is not for accepted ledgers). The same
-detection gate runs in CI (`python -m openultrasast.gate`).
+Each round (`improve/evolve.py`) runs these steps:
+
+1. benchmark with the current ledger;
+2. per-rule signals (`rule_signals.json`);
+3. bounded proposals;
+4. validation (`improve/validator.py`);
+5. replay smoke;
+6. re-benchmark;
+7. a hard gate.
+
+The levers are rule status (`enabled -> shadow` before `disabled`) and score constants. The
+loop can never edit pattern text or the authoritative CWE severity. A round is accepted only if
+all of these hold:
+
+- recall >= 90%;
+- FP < 10%;
+- the project score did not regress;
+- no matched finding was lost.
+
+Otherwise the ledger is not written. With `--pair-catalog`, each round is also gated per
+provenance profile on the catalog's holdout pairs. `--no-pair-gate` skips this check; do not
+use it for accepted ledgers. The same detection gate runs in CI (`python -m openultrasast.gate`).
 
 ### Proposals from plane memory
 
-With `--memory [STORE]`, `ousast improve` also reads the rows plane runs stored per repository
-and pin (`improve/memory.py`). Two deterministic rules turn them into rule-status proposals: a
-rule whose alerts are repeatedly false across repositories is shadowed, and a shadow rule that
-keeps hitting agreed, declared vulnerable sites is re-enabled. Rows from holdout pairs, from the
-gated manifest's own cases and from any `--qualify-population` are dropped before any rule sees
-them. Each proposal records the rows it came from (`memory_proposals.jsonl` next to the
-journal) and goes through the same validator and gate.
+With `--memory [STORE]`, `ousast improve` also reads the rows that plane runs stored per
+repository and pin (`improve/memory.py`). Two deterministic rules turn these rows into
+rule-status proposals:
+
+- a rule whose alerts are repeatedly false across repositories is shadowed;
+- a shadow rule that keeps hitting agreed, declared vulnerable sites is re-enabled.
+
+Some rows are dropped before any rule sees them: rows from holdout pairs, from the gated
+manifest's own cases and from any `--qualify-population`. Each proposal records the rows it
+came from (`memory_proposals.jsonl` next to the journal). It then goes through the same
+validator and gate.
 
 ### The loop as a plane Run
 
 `ousast plane workspaces <population> --validation-set ... --loop` appends the improvement loop
-to a generated Run: `alerts` per case (the quick scan on the vulnerable and fixed pins), then
-the steps `snapshot` (task `memory-snapshot`), `measure`, `propose` and `improve` (`plane/tasks/loop.py`), each model-free
-with a zero budget. The snapshot applies the same train-on-test guard before any row leaves the
-store. A recorded live run: `benchmarks/measurements/2026-09-30-harnessx-removal-loop/`.
+to a generated Run. The Run holds `alerts` per case: the quick scan on the vulnerable and fixed
+pins. Then come the steps `snapshot` (task `memory-snapshot`), `measure`, `propose` and
+`improve` (`plane/tasks/loop.py`). Each step is model-free with a zero budget. The snapshot
+applies the same train-on-test guard before any row leaves the store. A recorded live run:
+`benchmarks/measurements/2026-09-30-harnessx-removal-loop/`.
 
-For PHP, which quick mode covered with zero rules when that run was made, `ousast plane
+When that run was made, quick mode covered PHP with zero rules. So for PHP, `ousast plane
 alerts-engine` produces a Run's `alerts` from the engine image on the host instead
 (`benchmarks/measurements/2026-09-30-php-engine-alerts/`).

docs/decision-engine.md

@@ -1,51 +1,77 @@
 # The learned decision engine (in development)
 
-**Status 2026-10-02: in development, not adopted.** No scan, pre-push check or report uses it;
-every user-facing path is today's deterministic one until a compiled program is adopted. The
-design is the `learned-decision-engine` specification (maintainer tooling); the code is in
+**Status 2026-10-02: in development, not adopted.** No scan, pre-push check or report uses it.
+Every user-facing path stays on today's deterministic one until a compiled program is adopted.
+The design is the `learned-decision-engine` specification (maintainer tooling). The code is in
 `src/openultrasast/learn/`.
 
 ## Why
 
-Every detection gain before the engine was a hand edit tuned on the cases that exposed it, and
-engine tuning measured on population v1 did not transfer to v2. The reasoning, and the step-by-step
-chain that replaced hand edits, are on [Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
+Before the engine, every detection gain was a hand edit. Each edit was tuned on the cases that
+exposed the miss. Engine tuning measured on population v1 did not transfer to v2. [Where we
+stand](where-we-stand.md#2-our-approach-to-detection-step-by-step) gives the reasoning and the
+step-by-step chain that replaced hand edits.
 
 ## Design
 
-- **Signals, not verdicts.** Every instrument (quick rules, the Joern engine, model sink and
-  role classification, the plane's verify passes and agreement, repository facts such as
-  callers) contributes a feature record per candidate `(path, function, family)`, including an
-  explicit state when it did not run or failed. None decides alone.
+- **Signals, not verdicts.** Every instrument contributes a feature record per candidate
+  `(path, function, family)`. The instruments are quick rules, the Joern engine, model sink and
+  role classification, the plane's verify passes and agreement, and repository facts such as
+  callers. The record includes an explicit state when an instrument did not run or failed. No
+  instrument decides alone.
 - **Labels from ground truth.** `ousast learn labels` derives labels only from sources on a
-  fail-closed allow-list (`learn/sources.toml`): the vulnerable and fixed sides of pairs, the
-  declared sites and fix ranges of spent populations, benign deltas and recorded adjudications.
+  fail-closed allow-list (`learn/sources.toml`). Fail-closed means a source not on the list is
+  never used. The sources are:
+  - the vulnerable and fixed sides of pairs;
+  - the declared sites and fix ranges of spent populations;
+  - benign deltas;
+  - recorded adjudications.
+
   "Rejected by the plane" is never a negative. Reserved populations (v3) are not on the list.
-- **An AI classifier with local memory, compiled in house in the DSPy style.** The program is an
-  instruction, a few demonstrations and up to six retrieved labelled examples, then the
-  candidate's code excerpt and signals; the model (`deepseek-flash`) answers
-  vulnerable / not vulnerable / unsure with a rationale, sampled `k` times, giving a score `s`.
-  Compilation bootstraps demonstrations and searches instructions on a compile split, scored by
-  a pre-registered balanced Brier score (`learn/compile.toml`).
-- **Retrieval** from the memory store: nearest neighbours by Gower distance on the signal
-  profile, re-ranked by cosine similarity of code embeddings (OpenAI `text-embedding-3-small`
-  through OpenRouter, cached by excerpt hash), under a label/family/repository balance rule. An
-  evaluation-boundary filter keeps a candidate's own repository group out of what it is shown.
+- **An AI classifier with local memory, compiled in house in the DSPy style.** The program has
+  an instruction, a few demonstrations and up to six retrieved labelled examples. Then come the
+  candidate's code excerpt and signals. The model (`deepseek-flash`) answers vulnerable / not
+  vulnerable / unsure with a rationale. It is sampled `k` times, which gives a score `s`.
+  Compilation bootstraps demonstrations and searches instructions on a compile split. A
+  pre-registered balanced Brier score (`learn/compile.toml`) scores the candidates. The Brier score
+  measures how far predicted probabilities are from the true labels.
+- **Retrieval** from the memory store works in three steps:
+  1. find nearest neighbours by Gower distance (a distance over mixed feature types) on the
+     signal profile;
+  2. re-rank them by cosine similarity of code embeddings (OpenAI `text-embedding-3-small`
+     through OpenRouter, cached by excerpt hash);
+  3. apply a label/family/repository balance rule.
+
+  An evaluation-boundary filter keeps a candidate's own repository group out of what it is
+  shown.
 - **Repository-grouped evaluation.** The compile split (25% of repository groups) is never
-  evaluated. The rest is evaluated by outer grouped 5-fold cross-validation: the Platt
-  calibration map is cross-fitted on the other folds and the operating points are chosen on the
-  other folds (nested). One compile per profile, not one per fold. Intervals are 95%
-  repository-cluster bootstrap.
-- **Two operating points.** BLOCK is offered only when calibration holds (slope CI contains 1,
-  intercept CI contains 0, ECE <= 0.05) and a fold reaches the required precision; otherwise the
-  engine is ADVISORY only. An `unsure` answer can never BLOCK.
+  evaluated. The rest is evaluated by outer grouped 5-fold cross-validation:
+  - the Platt calibration map (which turns the score into a probability) is cross-fitted on
+    the other folds;
+  - the operating points are chosen on the other folds (nested).
+
+  There is one compile per profile, not one per fold. Intervals are 95% repository-cluster
+  bootstrap.
+- **Two operating points.** BLOCK is offered only when both of these hold:
+  - calibration holds: slope CI contains 1, intercept CI contains 0, ECE <= 0.05;
+  - a fold reaches the required precision.
+
+  Otherwise the engine is ADVISORY only. An `unsure` answer can never BLOCK. ECE (expected
+  calibration error) is the average gap between predicted and observed rates.
 
 ## First measured slice (2026-10-01)
 
 Record: `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json` (counts
-only). Family **injection** only; memory of 724 static examples from all families; input profile
-v1 with the engine signals and function length withheld (a leak audit found them separating the
-sides of a pair by construction: `2026-10-01-decision-engine-leak-audit/`); framework priors off.
+only). The setup:
+
+- family **injection** only;
+- memory of 724 static examples from all families;
+- input profile v1, with the engine signals and function length withheld;
+- framework priors off.
+
+The signals were withheld because a leak audit found them separating the sides of a pair by
+construction (`2026-10-01-decision-engine-leak-audit/`). AUC below is the chance the engine
+ranks a real bug above a fixed one.
 
 | Measure | Value (95% repository-cluster CI) |
 | --- | --- |
@@ -62,11 +88,17 @@ sides of a pair by construction: `2026-10-01-decision-engine-leak-audit/`); fram
 ## Second slice: six families (2026-10-02)
 
 Record: `benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json` (counts only).
-After the advisory-fix harvest the memory holds 904 static examples (724 before). One program was
-compiled per evaluable family (injection recompiled, since the memory changed) and evaluated on
-that family's own candidates, out of repository as above: compile split never evaluated, outer
-grouped 5-fold, calibration cross-fitted, operating points nested, 95% repository-cluster
-bootstrap intervals (2,000 resamples); input profile v1, priors off, k = 5.
+After the advisory-fix harvest, the memory holds 904 static examples (724 before).
+
+One program was compiled per evaluable family. Injection was recompiled, since the memory
+changed. Each program was evaluated on its family's own candidates, out of repository as above:
+
+- compile split never evaluated;
+- outer grouped 5-fold;
+- calibration cross-fitted;
+- operating points nested;
+- 95% repository-cluster bootstrap intervals (2,000 resamples);
+- input profile v1, priors off, k = 5.
 
 | Family | Candidates (groups; positive) | Pooled AUC of `s` (95% CI) | Within-pair AUC (paired groups) | ADVISORY recall / precision | Canary agreement |
 | --- | --- | --- | --- | --- | --- |
@@ -77,45 +109,174 @@ bootstrap intervals (2,000 resamples); input profile v1, priors off, k = 5.
 | path | 35 (21; 21) | 0.89 (0.79 - 0.97) | 0.83 (14) | 0.86 / 0.69 | 0.90 |
 | untrusted destination | 62 (37; 38) | 0.85 (0.76 - 0.94) | 0.72 (18) | 0.84 / 0.73 | 0.87 |
 
-- **BLOCK is offered for no family.** Calibration holds only for output encoding (ECE 0.045), and
-  there no fold reaches the required precision (`precision_unreachable`: no fold reaches a Wilson
-  lower bound of 0.95). For the other five families calibration does not hold (ECE 0.071 to 0.119).
-- **The pooled AUC is the more favourable number.** Every family's pool includes candidates that
-  carry only a positive label, with no fixed-side counterpart (7 to 19 per family; there are no
-  single-label negatives), so the pooled AUC is inflated by positives that never had to be told
-  apart from their own fix. The within-pair AUC over the paired groups, 0.72 to 0.93, is the
-  stricter reading (`paired_groups_check` in the record).
+- **BLOCK is offered for no family.** Calibration holds only for output encoding (ECE 0.045).
+  There, no fold reaches the required precision (`precision_unreachable`: no fold reaches a
+  Wilson lower bound of 0.95). For the other five families, calibration does not hold (ECE
+  0.071 to 0.119).
+- **The pooled AUC is the more favourable number.** Every family's pool includes candidates
+  that carry only a positive label, with no fixed-side counterpart (7 to 19 per family; there
+  are no single-label negatives). These positives never had to be told apart from their own
+  fix, so they inflate the pooled AUC. The within-pair AUC over the paired groups, 0.72 to
+  0.93, is the stricter reading (`paired_groups_check` in the record).
 - **Re-run agreement (canary, 30 candidates per family) is below 0.9 for three families:**
   injection 0.77, access control 0.83, untrusted destination 0.87.
-- **One memory-size point for injection** (paired: the same program, candidates and folds, with a
-  retrieval memory of 724 vs 904 examples): AUC 0.76 vs 0.77, within the intervals. No injection
-  example was added, so the new examples reach an injection prompt only as contrast.
-- **Spend:** $2.88 metered at list price for DeepSeek under an $8 ceiling, $0.0009 for
-  embeddings; $0.0027 to $0.0032 per evaluated candidate. The access-control evaluation replayed
-  from the response cache at $0 with identical predictions.
-- **An instrument fix is recorded:** compiled demonstrations ignored the near-duplicate rule
-  (cosine >= 0.98) until the pre-call boundary assertion stopped the untrusted-destination
-  evaluation; demonstrations are now swapped or dropped like retrieved examples, and the stopped
-  part was replayed after the fix.
+- **One memory-size point for injection.** This run is paired: the same program, candidates
+  and folds, with a retrieval memory of 724 vs 904 examples. AUC was 0.76 vs 0.77, within the
+  intervals. No injection example was added. So the new examples reach an injection prompt
+  only as contrast.
+- **Spend:** $2.88 metered at list price for DeepSeek under an $8 ceiling, and $0.0009 for
+  embeddings. That is $0.0027 to $0.0032 per evaluated candidate. The access-control
+  evaluation replayed from the response cache at $0 with identical predictions.
+- **An instrument fix is recorded.** Compiled demonstrations ignored the near-duplicate rule
+  (cosine >= 0.98). The pre-call boundary assertion caught this and stopped the
+  untrusted-destination evaluation. Demonstrations are now swapped or dropped like retrieved
+  examples. The stopped part was replayed after the fix.
 
-**Not adopted.** What is still ahead is listed once, on
-[Where we stand](where-we-stand.md#5-open-items).
+**Not adopted.** [Where we stand](where-we-stand.md#5-open-items) lists, in one place, what is
+still ahead.
 
 ## Experiments
 
 Changes to prompts, models, pass counts, features or sampling are compared by controlled
-experiments from a pre-registered manifest (`plane/experiments/<id>.yaml`), so nothing is adopted
-on one run's number. Two are registered. exp-001 (known callers) has not run. exp-002 (a retrieval
-ensemble against today's temperature sampling) ran on 160 paired candidates in 56 repository
-groups and was **rejected**: within-pair AUC 0.815 against 0.741 and $0.0021 against $0.0051 per
-candidate, with re-run agreement 0.875 against 0.906 and an interval spanning zero
-(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`). The reading is on
-[Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
+experiments. Each experiment comes from a pre-registered manifest
+(`plane/experiments/<id>.yaml`). This way nothing is adopted on one run's number.
+
+Four experiments are registered:
+
+- exp-001 (known callers) has not run.
+- exp-002 compared a retrieval ensemble against today's temperature sampling. It ran on 160
+  paired candidates in 56 repository groups and was **rejected**. Within-pair AUC was 0.815
+  against 0.741 and the cost was $0.0021 against $0.0051 per candidate. The record shows re-run agreement
+  0.875 against 0.906 and an interval spanning zero
+  (`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
+- exp-003 compared two BLOCK rules on the same scores. It was **inconclusive**. Neither rule flags anything.
+- exp-004 tested one pooled BLOCK threshold with a per-family floor. It was **inconclusive**.
+
+### exp-003: precision-bound blocking (2026-10-03)
+
+The question: is the calibration gate what keeps BLOCK out of reach?
+Arm A is today's rule: a calibrated probability, offered only when calibration holds.
+Arm B picks a raw-score threshold per family, on training folds only.
+It takes the lowest threshold whose Wilson 95% lower bound on precision reaches 0.95.
+The held-out repository group is then scored with that threshold.
+Arm C relaxes the bound to point precision 0.95 over at least 10 flags.
+Arm C was registered as secondary and does not decide.
+
+- **Registered first.** The manifest and units were committed and registered before the run.
+- **Primary metric:** arm B's held-out Wilson lower bound on flagged precision, per family.
+- **Primary population:** paired units only. Positives-only groups are excluded; pooled figures are secondary.
+- **Zero cost.** Scores came only from the response cache, behind a zero-ceiling meter.
+  The meter recorded 0 client calls and $0.0.
+- **Exclusions.** 142 of 353 units had an uncached response and were excluded, not asked.
+  Path lost all 39 units. Injection and access control lost none.
+- **Evaluated:** 211 units in 89 repository groups, 138 of them paired.
+
+| Family | Evaluated (excluded) | Paired positives | Flagged A / B / C | Coverage A / B / C |
+| --- | --- | --- | --- | --- |
+| access control | 41 (0) | 16 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
+| deserialization | 14 (34) | 2 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
+| injection | 119 (0) | 38 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
+| output encoding | 24 (21) | 8 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
+| path | 0 (39) | 0 | 0 / 0 / 0 | none |
+| untrusted destination | 13 (48) | 4 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
+
+No arm chose a threshold on any fold, so flagged precision is undefined everywhere.
+Arm A failed calibration where it was checked: ECE 0.077 (access control), 0.1067 (injection).
+The other families had too few units for a calibration check.
+
+**Why B flags nothing.** A Wilson lower bound of 0.95 needs 73 flags without one error.
+Injection, the largest family, holds 38 paired positives across all five folds.
+The registration predicted this zero before the run.
+On this data the bound, not the calibration gate, makes BLOCK unreachable.
+
+**The zero is the rule's, not the instrument's.** The scores are real: 48 distinct values for injection.
+In access control, the 10 highest-scored paired units are all positive.
+In injection, a negative appears at rank 4 of the paired ranking.
+Pooled rankings look cleaner because positives-only units fill the top.
+Injection answered `unsure` for 42 of 119 units, and `unsure` never blocks.
+
+Record: `benchmarks/experiments/exp-003-precision-bound-blocking/result.json`.
+
+**Supplementary: all 353 units (2026-10-03).** This is not a new decision; exp-003's verdict stands.
+The 142 excluded units had their responses filled with the same programs and request keys.
+The fill cost 697 client calls and $0.331759 under a $1.00 ceiling.
+The replay then decided all 353 units at zero cost: 244 paired, 146 repository groups.
+
+| Family | Paired units (positives) | Flagged A / B / C | ECE (arm A) |
+| --- | --- | --- | --- |
+| access control | 32 (16) | 0 / 0 / 0 | 0.077 |
+| deserialization | 34 (17) | 0 / 0 / 0 | 0.0596 |
+| injection | 76 (38) | 0 / 0 / 0 | 0.1067 |
+| output encoding | 32 (16) | 0 / 0 / 0 | 0.0761 |
+| path | 32 (16) | 0 / 0 / 0 | 0.062 |
+| untrusted destination | 38 (19) | 0 / 0 / 0 | 0.0454 |
+
+Still no arm flags any unit, so the registered rule reads inconclusive.
+Calibration holds only for untrusted destination.
+There arm A still finds no BLOCK point that reaches the bound.
+Record: `benchmarks/experiments/exp-003-precision-bound-blocking/result-complete-cache.json`.
+
+### exp-004: pooled block gate (2026-10-03)
+
+The experiment tested one pooled BLOCK threshold across families.
+Each family needed at least 10 clean flags to block.
+Each family also had its own quiet threshold.
+The run evaluated 501 units across 220 repository groups.
+Of these, 390 units were paired.
+The cache fill cost $1.238223 for 2,073 calls.
+The replay cost $0.0 and made 0 client calls.
+
+The verdict was **inconclusive**.
+No fold found a `t_block` threshold.
+The instrument check confirmed real scores.
+The pooled top 30 contained 27 positives.
+The first negative appeared at rank 12.
+The bound needs 73 clean flags without an error.
+
+A within-pair win means the vulnerable side scored above its fixed counterpart.
+
+| Family | Within-pair wins | Pairs |
+| --- | --- | --- |
+| access control | 22 | 28 |
+| deserialization | 18 | 25 |
+| injection | 30 | 53 |
+| output encoding | 18 | 28 |
+| path | 25 | 29 |
+| untrusted destination | 18 | 32 |
+
+Precision at the top of the ranking now limits blocking.
+Data volume alone cannot resolve this.
+The next levers are graph evidence in the prompt and the combiner.
+
+Records: `benchmarks/experiments/exp-004-pooled-block-gate/record.json` and `benchmarks/experiments/exp-004-pooled-block-gate/result.json`.
+
+### exp-004: pooled block gate with a family floor (registered, not run)
+
+This registration describes the setup before the run reported above.
+
+The maintainer amended the BLOCK rule on 2026-10-03 (requirements, Req 6.4).
+Precision is now shown across all families together, not per family.
+Each family's score is first mapped by its own Platt fit, on training folds.
+One threshold, t_block, then serves every family.
+It is the lowest one whose pooled Wilson lower bound reaches 0.95 on training folds.
+A family blocks only with at least 10 held-out flags and no error.
+Otherwise its flags stay advisory.
+Below a per-family t_quiet, which keeps 90% of positives, findings are silent.
+No held-out label moves a threshold.
+
+- **Primary metric:** the pooled held-out Wilson lower bound of flagged precision, paired units, before the floor.
+- **Decision:** reject below point precision 0.95; adopt if the bound reaches 0.95 and a family passes the floor.
+- **Units:** frozen from the memory after the negatives harvest.
+- **Cost:** the run replayed the cache after a paid cache fill.
+
+Manifest: `plane/experiments/exp-004-pooled-block-gate.yaml`.
+
+The reading is on [Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
 
 ## Commands
 
 `ousast learn` is a maintainer command group over the engine's data. Paid steps take
-`--budget-usd`; `--replay-only` answers every call from the response cache at no cost.
+`--budget-usd`. `--replay-only` answers every call from the response cache at no cost.
 
 | Command | Does |
 | --- | --- |
@@ -127,8 +288,36 @@ candidate, with re-run agreement 0.875 against 0.906 and an interval spanning ze
 | `ousast learn curve --program ID` | the metric over memory size on a fixed candidate subset |
 | `ousast learn audit-leaks` | how well each signal alone separates the sides of a pair (no model) |
 | `ousast learn experiment register\|units\|run\|analyse MANIFEST` | A/B experiments from a pre-registered manifest (`plane/experiments/<id>.yaml`): record its digest, freeze the units (no model), run the arms paired (arm A may replay from the cache at $0), analyse (repository bootstrap, exact McNemar, two looks, the adoption verdict) |
+| `ousast learn experiment fill MANIFEST --budget-usd X` | Fill missing responses for a registered `kind: rule` experiment's frozen units before its zero-cost run; `--dry-run` inspects and estimates without calls or writes |
+
+`experiment fill` accepts `--memory URL`, `--profile static|plane` (default `static`),
+`--manifest-model PATH` (default `plane/models/deepseek-flash.yaml`) and `--out PATH`.
+Freeze, commit and register the units first. Fill uses the incumbent programs and the
+same retrieval, sampling and cache keys as replay. The first missing request is the
+smoke call; a successful smoke response is kept. Any provider failure stops the fill.
+
+The JSON report on stdout gives each family's units, distinct requests, initially
+cached and missing responses, newly filled responses and still-missing responses,
+plus total client calls and metered USD. `--out` also saves it, except during dry-run,
+which writes nothing. Exit codes are 0 for completion or dry-run, 1 for a provider/cache
+failure, 2 for invalid setup and 3 for budget exhaustion. Resume a partial fill with a
+new invocation and budget; that budget covers only the new invocation, including smoke.
+
+Dry-run uses each family's mean billed cached response cost when recorded usage exists.
+Model prices account for cache-hit input, cache-miss input, and output tokens.
+Without cached usage, it uses the program's input-token estimate and **300 output tokens**.
+Each family's `estimate_method` reports `measured` or `estimated`.
+It reports potential retries separately as `conditional_retries`: an unseen answer
+may need one parse retry, whose necessity cannot be known without calling the model.
+Known invalid cached answers include their retry requests in the inventory.
+
+Fill reserves at least **$0.01** and the largest observed call cost before each call.
+For the production client it also reserves a conservative input-byte token bound plus
+the program's enforced output-token limit; hidden provider retries are disabled.
+Consequently a small budget may stop before making any call, and a fill may finish
+with unused budget. Calls without token usage stop as failures, never as free calls.
 
 The memory store is the plane's (`OUSAST_MEMORY`; see [ops/ax/README.md](ops/ax/README.md)).
 The harvests that produced the plane signals ran as plane Runs generated by
-`ousast plane harvest` (`benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`
-and, for the advisory-fix pairs, `2026-10-02-decision-engine-harvest-advisory/record.json`).
+`ousast plane harvest`. Records: `benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`
+and, for the advisory-fix pairs, `2026-10-02-decision-engine-harvest-advisory/record.json`.

docs/deployment.md

@@ -1,17 +1,36 @@
 # Deployment
 
-How the agentic plane is deployed today, and what moving it to a separate Kubernetes cluster
-would take. State as of 2026-10-02 (v2.0.0). A scan needs none of this: `ousast scan` and
-`ousast pre-push` run from the Python package (and the Docker image for Joern, see
-[Engine and pre-push](ops/README.md)). The plane is the maintainers' surface for model work over
-whole repositories, driven by `ousast plane`.
+This page covers two things. First, how the agentic plane is deployed today. Second, what moving it to a
+separate Kubernetes cluster would take. It describes the state as of 2026-10-02 (v2.0.0).
 
-Two words are used strictly on this page. **Implemented** means the code on `main` does it and it
-has run on the maintainer's host. **Planned** means it has not been done; the guidance is derived
-from the code and the host's lessons, and the code changes it needs are named with their file paths.
+A scan needs none of this. `ousast scan` and `ousast pre-push` run from the Python package. Joern runs
+from the Docker image (see [Engine and pre-push](ops/README.md)). The plane is the maintainers' tool for
+model work over whole repositories. `ousast plane` drives it.
+
+This page uses two words strictly:
+
+- **Implemented** means the code on `main` does it, and it has run on the maintainer's host.
+- **Planned** means it has not been done. The guidance comes from the code and from what the host taught
+  us. Each code change it needs is named with its file path.
 
 **No deployment of the plane to a cluster other than the maintainer's single-node kind cluster has
-been made.** Section B is guidance, not a record.
+been made.** Section B is guidance, not a record. (kind runs a Kubernetes cluster inside Docker
+containers on one machine.)
+
+The kind profile runs on a 7.7 GB server VM. Observability is off by default
+(`kind_observability = false` in `ops/k8s/profiles/kind.toml`). This saves the measured
+930 MiB used by `otel-system`, about 40% of the cluster's 2.3 GB. The smoke Run still
+completes with those deployments stopped. To enable the stack, run
+`OUSAST_KIND_OBSERVABILITY=1 ops/ax/up.sh` or set `kind_observability = true` in the profile.
+
+`up.sh` stages copied Substrate manifests in a temporary checkout view and installs through
+`hack/install-ate-kind.sh`. The local `ops/ax/substrate-kind-lean/` overlay removes the collector,
+Prometheus, Jaeger and their supporting resources before installation. This preserves the
+installer's secret, CRD and image setup. The upstream checkout is unchanged. The shared
+`ate-otel-config` disables trace, metric and log export and the OTEL SDK; it removes the old
+collector endpoint and metric timing keys. On an existing cluster, `up.sh` scales the three
+observability deployments to zero, replaces the ConfigMap data and rolls its consumers when
+the configuration changes. Opting in restores the upstream ConfigMap and observability manifests.
 
 ## What runs where today (implemented)
 
@@ -27,11 +46,11 @@ flowchart LR
         sub["Agent Substrate (ate-system): atenet-router, atenet-egress"]
         act["Task actors in gVisor workers (WorkerPool ousast-pool)"]
         svc["Service ousast-receiver + EndpointSlice to the host"]
-        reg["Registry localhost:5001: runner image by digest"]
+        reg["Registry named by the profile: runner image by digest"]
     end
     mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")]
     rec -- "ax apply, resume, get, delete" --> ax
-    rec -- "EgressPolicy per actor; start request via port-forward" --> sub
+    rec -- "EgressPolicy per actor; start request through the kube-context tunnel" --> sub
     sub --> act
     act -- "HTTP by Service name through the gateway" --> svc
     svc --> rcv
@@ -46,19 +65,26 @@ flowchart LR
 | How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` |
 | ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` |
 | Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` |
-| Runner image | the kind-local registry `localhost:5001`, pinned by digest (`~/.cache/ousast/ax-src/runner-image`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
-| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through a `kubectl port-forward` to `svc/atenet-router` | `src/openultrasast/plane/router.py` |
+| Runner image | the registry the profile names (`registry` in `ops/k8s/profiles/kind.toml`: the kind-local one), pinned by digest in the profile's images file (`ops/k8s/profiles/kind-images.json`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
+| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through the tunnel `router.py` opens to `svc/atenet-router` in the profile's kube context | `src/openultrasast/plane/router.py` |
 | Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://<bucket>[/<prefix>]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) |
 | Run state and artifacts | `$OUSAST_RESULTS/plane/<run>/` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) |
 
-Only the reconciler talks to the memory store: a sandboxed task cannot read it, so the loop's
-`memory-snapshot` is seeded by the reconciler before the Run ([The plane on ax](plane.md)).
+Terms in the table:
+
+- **ax** is Google's controller that runs agent tasks on Kubernetes.
+- **Agent Substrate** is the layer below ax that owns the machines and runs sandboxes.
+- **gVisor** is a sandbox runtime that isolates each actor from the node's kernel.
+- An **actor** is one running Task inside a sandbox.
+
+Only the reconciler talks to the memory store. A sandboxed task cannot read it. So the reconciler seeds
+the loop's `memory-snapshot` before the Run starts ([The plane on ax](plane.md)).
 
 ## A. This host: the local setup (implemented)
 
-What `ops/ax/up.sh` needs on the PATH: `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` (it
-refuses to start when one is missing). Tools live in `~/go/bin` and `~/.local/bin`; sources and
-rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
+`ops/ax/up.sh` needs these tools on the PATH: `docker`, `go`, `kubectl`, `kind`, `ko` and `ax`. It
+refuses to start when one is missing. Tools live in `~/go/bin` and `~/.local/bin`. Sources and rendered
+files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
 
 1. **Bring up the cluster and the plane.**
 
@@ -66,25 +92,39 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
     ops/ax/up.sh
     ```
 
-    Idempotent: each step is skipped when already done. It creates the kind cluster
-    (`KIND_CLUSTER_NAME`, default `ousast`) with its registry (`KO_DOCKER_REPO`, default
-    `localhost:5001`), installs Agent Substrate, applies the egress gateway (the agentgateway
-    variant, one prebuilt image) when the install left it out, deploys the ax control plane, applies
-    the receiver Service with the host's kind-gateway address as its endpoint, applies the gVisor
-    worker pool, builds and pushes the runner image and writes its digest pin to
-    `~/.cache/ousast/ax-src/runner-image`, installs the `ax` CLI from the deployed checkout (a release
-    CLI skews from the server), and runs one smoke Task. It ends by printing the host's memory and the
-    kind containers' usage.
+    The script is idempotent: it skips each step that is already done. In order, it:
+
+    - creates the kind cluster (`KIND_CLUSTER_NAME`, default `ousast`) with its registry
+      (`KO_DOCKER_REPO`, default: the `registry` of `ops/k8s/profiles/kind.toml`);
+    - installs Agent Substrate;
+    - applies the egress gateway if the install left it out (the agentgateway variant, one prebuilt image);
+    - deploys the ax control plane;
+    - applies the receiver Service, with the host's kind-gateway address as its endpoint;
+    - applies the gVisor worker pool;
+    - builds and pushes the runner image, and writes its digest pin to `~/.cache/ousast/ax-src/runner-image`;
+    - installs the `ax` CLI from the deployed checkout (a release CLI does not match the server version);
+    - runs one smoke Task.
+
+    At the end it prints the host's memory and the kind containers' usage.
 
 2. **Check it.**
 
     ```bash
-    uv run ousast plane doctor
+    uv run ousast plane doctor --profile kind
     ```
 
-    Four checks: the kind context is reachable, every pod in `ate-system` and in `ax-system` is ready,
-    and the registry at `localhost:5001` holds `ousast-runner`. Each failing line names
-    `ops/ax/up.sh` as the fix. `doctor` is informational: `ousast plane run` does not call it.
+    It checks the profile you pass (`--profile`, default `kind`; the profiles are in `ops/k8s/profiles/`):
+
+    - its configured addresses, printed on the first line;
+    - the kube context is reachable;
+    - every pod in `ate-system` is ready;
+    - every pod in `ax-system` is ready;
+    - the runner image of the profile's images file is in its registry (a local registry over its v2
+      API, a remote one through `docker manifest inspect`);
+    - the memory store opens (an S3 bucket is verified).
+
+    Each failing line names the fix (`ops/ax/up.sh` for the kind profile). `doctor` is informational:
+    `ousast plane run` does not call it.
 
 3. **Run the end-to-end smoke Run.**
 
@@ -92,23 +132,27 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
     ops/ax/smoke-run.sh
     ```
 
-    It renders `ops/ax/smoke-run.yaml.tmpl` with the current runner digest and submits the Run
-    `ax-e2e-smoke` through the reconciler: one model-free `repo-facts` Task (`budget: {usd: 0,
-    calls: 0}`) over a files-only Workspace, resumed, started through the router, its `facts.json` and
-    `summary.json` delivered to the host, then `ousast plane status ax-e2e-smoke` prints the attribution
-    table. The script calls the repository's `.venv/bin/ousast`, so `uv sync` first.
+    It renders `ops/ax/smoke-run.yaml.tmpl` with the current runner digest. It then submits the Run
+    `ax-e2e-smoke` through the reconciler. The Run has one model-free `repo-facts` Task (`budget: {usd: 0,
+    calls: 0}`) over a files-only Workspace, resumed, started through the router, its `facts.json`
+    and `summary.json` delivered to the host. Then `ousast plane status ax-e2e-smoke`
+    prints the attribution table. The script calls the repository's `.venv/bin/ousast`, so run `uv sync`
+    first.
 
-4. **Keys and the store.** Put `DEEPSEEK_API_KEY` (chat) and `OPENROUTER_API_KEY` (embeddings) in
-    `.env` or export them; a Task's Model names the variable (`secretKey.key`), and the key travels
-    only in that Task's start request. Set `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT`,
-    `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (optional `S3_REGION`, `S3_BUCKET`) for the S3 store;
-    the bucket must be set up once by an admin as [RustFS setup](rustfs.md) describes, and the store
-    verifies it at every start.
+4. **Keys and the store.**
+
+    - Put `DEEPSEEK_API_KEY` (chat) and `OPENROUTER_API_KEY` (embeddings) in `.env`, or export them.
+    - A Task's Model names the variable (`secretKey.key`). The key travels only in that Task's start
+      request.
+    - For the S3 store, set `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID` and
+      `AWS_SECRET_ACCESS_KEY`. `S3_REGION` and `S3_BUCKET` are optional.
+    - An admin must set up the bucket once, as [RustFS setup](rustfs.md) describes. The store verifies
+      it at every start.
 
 5. **Run a real Run**, as in [Examples](examples.md#7-running-the-agentic-plane): `ousast plane run
     plane/runs/validation-46.yaml`, then `status` and `remember`.
 
-6. **Tear down** with `ops/ax/down.sh`, which deletes the cluster and its registry.
+6. **Tear down** with `ops/ax/down.sh`. It deletes the cluster and its registry.
 
 **Footprint** (measured 2026-09-29, idle, two workers; [ax on this host](ops/ax/README.md#measured-footprint-2026-09-29-idle-two-workers)):
 
@@ -119,44 +163,70 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
 | node volume (images, containerd) | 4.9 GB on disk |
 | worker pool | 2 gVisor workers, 1 CPU / 1.5 GiB limit each |
 
-Disk is the binding constraint: the Substrate install and the `ko` builds need several GB of Go
-caches (`go clean -cache -modcache` afterwards), and every Task is its own actor template with a
-golden actor (about 24 MB each under `/var/lib/ate/actors` on the node). The failures the host
-produced, and what each one taught, are listed in
-[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run).
+Disk is the tightest limit, for two reasons:
+
+- The Substrate install and the `ko` builds need several GB of Go caches. Run `go clean -cache -modcache`
+  afterwards.
+- Every Task is its own actor template with a golden actor (a saved snapshot the actor boots from). Each
+  takes about 24 MB under `/var/lib/ate/actors` on the node.
+
+[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run) lists the
+failures the host produced and what each one taught.
 
 ## B. A separate Kubernetes cluster (planned)
 
-The target: a cluster that runs Agent Substrate with its egress gateway and the ax control plane,
-pulls the runner image from a registry of yours, receives task artifacts without a developer
-laptop in the path, and keeps its memory in your S3 bucket. The provider key already travels only
-in the start request through `atenet-router`, which works the same through a port-forward to any
-cluster. Everything else below is either a step you take or a code change that is not made yet.
+The target cluster would do four things:
+
+- run Agent Substrate with its egress gateway and the ax control plane;
+- pull the runner image from a registry of yours;
+- receive task artifacts without a developer laptop in the path;
+- keep its memory in your S3 bucket.
+
+The provider key already travels only in the start request through `atenet-router`. That works the same
+through the kube-context tunnel to any cluster. Everything else below is either a step you take or a code
+change that is not made yet.
 
 ### B.1 Cluster with Agent Substrate, its egress gateway and ax
 
-- Install Agent Substrate on the cluster following its own instructions. `ops/ax/up.sh` uses
-  Substrate's kind helpers (`hack/create-kind-cluster.sh`, `hack/install-ate-kind.sh`), which do
-  not apply to another cluster.
+- Install Agent Substrate on the cluster with its own instructions. `ops/ax/up.sh` uses Substrate's kind
+  helpers (`hack/create-kind-cluster.sh`, `hack/install-ate-kind.sh`). These do not apply to another
+  cluster.
 - Apply the egress gateway. On this host it is the agentgateway variant
-  (`manifests/ate-install/agentgateway-egress` in the Substrate checkout, applied with
-  `kubectl kustomize --load-restrictor=LoadRestrictionsNone`), which needs no Rust build; the
-  Envoy variant does. Without the gateway an actor has no network at all, and the per-task
-  EgressPolicies the reconciler writes (`src/openultrasast/plane/egress.py`) work unchanged
-  against the Substrate-installed gateway: deny by default, hostnames only, plain HTTP to the
-  receiver, TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
+  (`manifests/ate-install/agentgateway-egress` in the Substrate checkout). It is applied with
+  `kubectl kustomize --load-restrictor=LoadRestrictionsNone`. It needs no Rust build; the Envoy variant
+  does.
+- Without the gateway an actor has no network at all.
+- The reconciler writes one EgressPolicy per task (`src/openultrasast/plane/egress.py`). These work
+  unchanged against the gateway that Substrate installs. The rules are:
+    - deny by default;
+    - hostnames only;
+    - plain HTTP to the receiver;
+    - TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
 - Deploy ax (`make deploy AX_IMAGE_REPO=<your registry>` in the ax checkout) **after** B.2.
 - Apply a WorkerPool (B.6). Substrate installs none by itself ("no free workers").
+- **gVisor on the nodes (planned, checked by hand on the day k3s exists).** Substrate does not use a Kubernetes
+  `RuntimeClass`: the `ateom-gvisor` worker runs `runsc` itself, from the gVisor release tarball named in
+  `manifests/ate-install/sandboxconfig-gvisor.yaml` of the Substrate checkout
+  (`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd`, sha256-pinned per architecture), which
+  the atelet fetches on the worker node. The node must reach that URL, or a copied `SandboxConfig` must name a
+  mirrored copy with the same sha256 and the ActorTemplates must name that config. The check is the atelet's
+  fetch line in `kubectl -n ate-system logs` of the worker pod; it is labelled planned until that log has been
+  read on a k3s node (plane-on-kubernetes task 1.7, design open question 3).
 
 ### B.2 ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET`
 
-ax's deploy manifest (`deploy/ax-server.yaml` in the ax checkout) sets `AX_SNAPSHOTS_BUCKET` to
-the ax authors' own bucket (`gs://dberkov-gke-dev3/ate-env/` in the checkout used here). Point it
-at **your own bucket before deploying ax**. On this host the bucket named there did not exist in
-the S3 server, deleted golden actors piled up in `DELETING` and filled the disk during a 600-task
-harvest (2026-10-01); creating the bucket let deletes complete, and even then a deleted actor's
-directory stays on the node, so long Runs need a janitor that removes directories no live actor
-owns.
+Point `AX_SNAPSHOTS_BUCKET` at **your own bucket before deploying ax**. ax's deploy manifest
+(`deploy/ax-server.yaml` in the ax checkout) sets it to the ax authors' own bucket. In the checkout used
+here that is `gs://dberkov-gke-dev3/ate-env/`.
+
+This host shows what goes wrong otherwise (2026-10-01):
+
+1. The bucket named there did not exist in the S3 server.
+2. Deleted golden actors piled up in `DELETING`.
+3. They filled the disk during a 600-task harvest.
+
+Creating the bucket let deletes complete. Even then, a deleted actor's directory stays on the node. So
+long Runs need a janitor that removes directories no live actor owns.
 
 ### B.3 A registry the workers can pull from, the runner image pinned by digest
 
@@ -168,83 +238,109 @@ owns.
     docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' <registry>/ousast-runner:dev
     ```
 
-- Substrate rejects tags: every Task's `image` must be the `<registry>/ousast-runner@sha256:...`
-  reference. Write it into a file (on this host, `~/.cache/ousast/ax-src/runner-image`) and pass
-  `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`, which re-pin the
-  Tasks they generate. The committed templates under `plane/tasks/` and the generated
-  `plane/tasks/validation-46.yaml` carry `localhost:5001/ousast-runner@sha256:...` and need the same
-  re-pin (regenerate, or replace the reference) before they run elsewhere.
-- Workers must be able to pull from the registry (credentials, if any, are the cluster's concern;
-  ax's default runner image on `gcr.io` needs credentials and is not used here).
-- **Not yet configurable:** `ousast plane doctor` checks the registry at `localhost:5001` by name
-  (`src/openultrasast/plane/doctor.py:45`), so its fourth check fails on another cluster even when
-  the image is in place. `doctor` is informational, `plane run` does not depend on it.
+- Substrate rejects tags. Every Task's `image` must be the `<registry>/ousast-runner@sha256:...`
+  reference.
+- Write that reference into a file. On this host the file is `~/.cache/ousast/ax-src/runner-image`.
+- Pass `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`. They re-pin the
+  Tasks they generate.
+- The committed templates under `plane/tasks/` and the generated `plane/tasks/validation-46.yaml` carry
+  the kind profile's registry and digest. They need the same re-pin before they run elsewhere.
+  Regenerate them, or replace the reference.
+- Workers must be able to pull from the registry. Credentials, if any, are the cluster's concern. ax's
+  default runner image on `gcr.io` needs credentials and is not used here.
+- `ousast plane doctor --profile <name>` looks up the runner reference of the profile's images file in
+  the profile's `registry` (implemented 2026-10-02, plane-on-kubernetes task 1.2). `doctor` is
+  informational; `plane run` does not depend on it.
 
-### B.4 One configurable kube context
+### B.4 One configurable kube context (implemented 2026-10-02)
 
-The reconciler, the egress client and the router port-forward all use the context
-`kind-$KIND_CLUSTER_NAME` (`kind-ousast` by default). **Not yet configurable**: the prefix `kind-`
-is hard-coded in `src/openultrasast/plane/doctor.py:39`, `src/openultrasast/plane/egress.py:115`
-and `:148`, and `src/openultrasast/plane/router.py:145`; `ops/ax/up.sh:24` builds the same name.
-The planned change is one variable (for example `OUSAST_KUBE_CONTEXT`) read in those three
-modules. Until then the only way to drive another cluster from the unchanged code is to give its
-kubeconfig context a name of that form (`kubectl config rename-context <ctx> kind-<name>` and
-`KIND_CLUSTER_NAME=<name>`), which is a workaround, not a supported configuration. The egress
-client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names) able to use that
-context.
+The reconciler, the egress client and the router tunnel take the kube context from the profile.
+
+- The setting is `kube_context` in `ops/k8s/profiles/<name>.toml`; `OUSAST_KUBE_CONTEXT` overrides it
+  (plane-on-kubernetes task 1.2).
+- `doctor.py`, `egress.py` and `router.py` are handed the context. They build none themselves.
+- `tests/test_plane_literals.py` fails on any module, manifest or page that names this host's cluster,
+  registry or addresses.
+
+The egress client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names). It must be
+able to use that context.
 
 ### B.5 The receiver: in-cluster instead of on the host
 
-Today the receiver is a thread inside the reconciler process (`egress.py`, `Receiver`), and the
-reconciler learns of a task's completion by watching that object's `delivered` set in memory
-(`reconciler.py`). Actors reach it through the Service `ousast-receiver.ax-system` whose
-EndpointSlice points at the host. Three paths, in order of effort:
+Today the receiver is a thread inside the reconciler process (`egress.py`, `Receiver`). The reconciler
+learns that a task finished by watching that object's `delivered` set in memory (`reconciler.py`).
+Actors reach it through the Service `ousast-receiver.ax-system`. That Service's EndpointSlice points at
+the host.
 
-1. **Interim, works with the unchanged code:** keep the host receiver and render
-   `ops/ax/receiver-service.yaml.tmpl` with an endpoint address the egress gateway pod can reach
-   (`KIND_GATEWAY` is just an IPv4 address in the template, `OUSAST_ARTIFACT_PORT` the port). The
-   reconciler's host must be reachable from the cluster (a VPN or a routed address), and the
-   gateway dials the address the actor connected to, so the Service's ClusterIP must still map to it.
-2. **Planned: the receiver as an in-cluster Deployment** behind the same Service name, with the
-   reconciler reading deliveries and serving inputs from it instead of from its own thread. This
-   needs code: the receiver and its `delivered`/`inputs` state are in-process today, and the
-   runner's `GET /inputs/<producer>/<artifact>` is served by that same process.
-3. **Follow-on: presigned-URL delivery to the S3 store.** The runner would `PUT` its output tar to a
-   presigned URL on the memory store's bucket and fetch inputs the same way, and the reconciler
-   would read from the bucket. Nothing of this exists; the EgressPolicy would then have to allow
-   the store's hostname for the task.
+There are three paths, from least to most effort:
+
+1. **Interim, works with the unchanged code:** keep the host receiver.
+   Render `ops/ax/receiver-service.yaml.tmpl` with an endpoint address the egress gateway pod can reach.
+   In the template, `KIND_GATEWAY` is just an IPv4 address and `OUSAST_ARTIFACT_PORT` is the port.
+   The cluster must be able to reach the reconciler's host, through a VPN or a routed address.
+   The gateway dials the address the actor connected to. So the Service's ClusterIP must still map to it.
+2. **Planned: the receiver as an in-cluster Deployment** behind the same Service name.
+   The reconciler would read deliveries from it and serve inputs from it, instead of from its own thread.
+   This needs code. The receiver and its `delivered`/`inputs` state are in-process today.
+   The same process serves the runner's `GET /inputs/<producer>/<artifact>`.
+3. **Follow-on: presigned-URL delivery to the S3 store.** (A presigned URL grants one upload or download
+   without credentials.)
+   The runner would `PUT` its output tar to a presigned URL on the memory store's bucket.
+   It would fetch inputs the same way, and the reconciler would read from the bucket.
+   None of this exists. The EgressPolicy would then have to allow the store's hostname for the task.
 
 ### B.6 Worker pool sizing
 
-`ops/ax/workerpool.yaml.tmpl` declares two warm gVisor workers of 1 CPU / 1.5 GiB (requests 250m /
-1.5 GiB) for the 7 GB host, selected onto nodes by `ate.dev/substrate-version`. An actor occupies a
-whole worker, so the pool size is the number of tasks that can run at once; set `--workers` on
-`ousast plane run` to match. Size the workers above the largest Task's limits (`verify`: requests
-250m / 384Mi, limits 1 CPU / 1 GiB; `roles` and the others are in `plane/tasks/`), and keep the
-golden-actor storage on the node in mind (B.2).
+`ops/ax/workerpool.yaml.tmpl` declares two warm gVisor workers for the 7 GB host. Each has 1 CPU /
+1.5 GiB (requests 250m / 1.5 GiB). They are placed on nodes by `ate.dev/substrate-version`.
+
+An actor occupies a whole worker. So the pool size is the number of tasks that can run at once. Set
+`--workers` on `ousast plane run` to match.
+
+Size the workers above the largest Task's limits. For `verify` these are requests 250m / 384Mi and
+limits 1 CPU / 1 GiB. `roles` and the others are in `plane/tasks/`. Also keep the golden-actor storage
+on the node in mind (B.2).
 
 ### B.7 The S3 memory store reachable from where the reconciler runs
 
-The `s3://` store already works against any reachable S3-compatible server with versioning,
-lifecycle rules, object tags and S3 Select (RustFS is the tested one; the bucket setup and the
-agent policy are on [RustFS setup](rustfs.md)). Only the reconciler reads and writes it, so today
-it must be reachable from the operator's host; if the reconciler moves into the cluster, from its
-pod. No Task talks to the store directly; the day one does (B.5, path 3), that Task's EgressPolicy
-must allow the store's hostname (the policies allow hostnames only, never addresses), the way a
-Model declares its hosts today through `openultrasast.io/egress-hosts`.
+The `s3://` store already works against any reachable S3-compatible server. The server must support:
+
+- versioning;
+- lifecycle rules;
+- object tags;
+- S3 Select (a server-side query over JSON objects).
+
+RustFS is the tested server. The bucket setup and the agent policy are on [RustFS setup](rustfs.md).
+
+Only the reconciler reads and writes the store. So today the operator's host must reach it. If the
+reconciler moves into the cluster, its pod must reach it.
+
+No Task talks to the store directly. If one ever does (B.5, path 3), its EgressPolicy must allow the
+store's hostname. The policies allow hostnames only, never addresses. A Model already declares its hosts
+this way today, through `openultrasast.io/egress-hosts`.
 
 ### B.8 Secrets in Kubernetes Secrets, injected into the reconciler's environment
 
-The reconciler reads the variable a Model's `secretKey.key` names (`DEEPSEEK_API_KEY`,
-`OPENROUTER_API_KEY`) and the store's `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`
-(optional `AWS_SESSION_TOKEN`, `S3_REGION`, `S3_BUCKET`) from its own environment or `.env`. When
-the reconciler runs in the cluster, keep them in Kubernetes Secrets and inject them into **the
-reconciler's** pod environment (`envFrom` with `secretRef`, or `valueFrom.secretKeyRef` per
-variable). Never put a value into a manifest: the Model manifests name a variable, the rendered
-Task carries no key, and the key reaches an actor only in the start request, where the runner keeps
-it in memory and redacts it from echoed stderr. The `secretKey.name` values in `plane/models/`
-(`deepseek`, `openrouter`) are the names such Secrets would carry. Running the reconciler
-in-cluster is itself planned: today it is a host process, and its state directory
+The reconciler reads these variables from its own environment or `.env`:
+
+- the variable a Model's `secretKey.key` names (`DEEPSEEK_API_KEY`, `OPENROUTER_API_KEY`);
+- the store's `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`;
+- optionally `AWS_SESSION_TOKEN`, `S3_REGION` and `S3_BUCKET`.
+
+When the reconciler runs in the cluster, keep these in Kubernetes Secrets. Inject them into **the
+reconciler's** pod environment. Use `envFrom` with `secretRef`, or `valueFrom.secretKeyRef` per variable.
+
+Never put a value into a manifest. The key stays out of every manifest:
+
+- the Model manifests name a variable;
+- the rendered Task carries no key;
+- the key reaches an actor only in the start request;
+- the runner keeps it in memory and redacts it from echoed stderr.
+
+The `secretKey.name` values in `plane/models/` (`deepseek`, `openrouter`) are the names such Secrets
+would carry.
+
+Running the reconciler in-cluster is itself planned. Today it is a host process. Its state directory
 (`OUSAST_RESULTS`) would need a persistent volume.
 
 ### What is implemented and what is planned
@@ -255,10 +351,10 @@ in-cluster is itself planned: today it is a host process, and its state director
 | Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented |
 | S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) |
 | Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented |
-| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates still name `localhost:5001` |
+| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates name the kind profile's registry (`tests/test_plane_literals.py` checks it) |
 | Deployment to a cluster other than this host's kind cluster | **not done** |
-| One configurable kube context | **planned**; `kind-` hard-coded in `doctor.py`, `egress.py`, `router.py` |
-| `doctor` registry check against a configurable registry | **planned**; `localhost:5001` hard-coded in `doctor.py:45` |
+| One configurable kube context | implemented (2026-10-02): `kube_context` in the profile, `OUSAST_KUBE_CONTEXT` overrides |
+| `doctor` registry check against a configurable registry | implemented (2026-10-02): the profile's `registry` and images file |
 | Receiver as an in-cluster Deployment | **planned** (code change) |
 | Presigned-URL delivery to the S3 store | **follow-on**, not started |
 | Reconciler running in-cluster with Secrets injected into its environment | **planned** |
@@ -266,12 +362,20 @@ in-cluster is itself planned: today it is a host process, and its state director
 
 ## Production topology: ax on a Kubernetes cluster
 
-The maintainer's questions, answered from the code and the upstream checkouts under
-`~/.cache/ousast/ax-src/`: how is it hosted in ax, is there one container, how does ax relate to
-Kubernetes, and what scaling the pre-push hook would take. The decisions of 2026-10-02 set the
-target: **two profiles of one code base**, `kind` (this laptop, kept as a first-class local
-execution mode, where every change is proven first) and `k3s` (production). Everything marked
-**planned** is not built; the rest is what runs on this host today.
+This section answers the maintainer's questions from the code and the upstream checkouts under
+`~/.cache/ousast/ax-src/`:
+
+- How is it hosted in ax?
+- Is there one container?
+- How does ax relate to Kubernetes?
+- What would scaling the pre-push hook take?
+
+The decisions of 2026-10-02 set the target: **two profiles of one code base**.
+
+- `kind` is this server VM. It stays a first-class local execution mode. Every change is proven here first.
+- `k3s` is production. (k3s is a small Kubernetes distribution.)
+
+Everything marked **planned** is not built. The rest is what runs on this host today.
 
 ```mermaid
 flowchart LR
@@ -279,7 +383,7 @@ flowchart LR
         hook["ousast pre-push: local, 30 s deadline"]
         cli["ousast plane run, remote mode (planned)"]
     end
-    subgraph k3s ["k3s cluster (the kind profile is the same picture on one laptop)"]
+    subgraph k3s ["k3s cluster (the kind profile is the same picture on one server VM)"]
         subgraph axns ["Namespace ax-system"]
             axs["ax-server Deployment + Redis"]
         end
@@ -287,7 +391,7 @@ flowchart LR
             ctl["ate-api-server, ate-controller, atelet"]
             rt["atenet-router"]
             eg["atenet-egress gateway"]
-            wp["WorkerPool: gVisor workers, runsc RuntimeClass, HPA (planned)"]
+            wp["WorkerPool: gVisor workers, runsc in the worker, HPA (planned)"]
             ewp["Engine WorkerPool: Joern as a task (planned)"]
         end
     end
@@ -308,124 +412,173 @@ flowchart LR
     cli -- "polls: status, attribution" --> s3
 ```
 
-No inbound path exists in this picture: the CLI reaches ax-server and the router from outside, the
-tasks push their artifacts to the store through the egress gateway, and the CLI reads the store.
-Today's EndpointSlice back to a laptop (`ops/ax/receiver-service.yaml.tmpl`) is the kind profile's
-device and does not appear in production.
+This picture has no inbound path:
+
+- the CLI reaches ax-server and the router from outside;
+- the tasks push their artifacts to the store through the egress gateway;
+- the CLI reads the store.
+
+Today's EndpointSlice back to the server VM (`ops/ax/receiver-service.yaml.tmpl`) belongs to the kind
+profile only. It does not appear in production.
 
 ### ax, Kubernetes and Agent Substrate in three sentences
 
-**ax is a Kubernetes application**: a Deployment `ax-server` with a Redis beside it in the
-namespace `ax-system` (`ax/deploy/ax-server.yaml` in the ax checkout; its controller runs with
-`--template=default-template --template-atespace=ax-system`), and the `ax` CLI reaches it through a
-`kubectl port-forward` to `svc/ax-server` (`ax/internal/tunnel/tunnel.go`), so `ousast plane run`
-needs a kube context or an ingress, not a public endpoint. **ax does not run containers itself**:
-it turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
-`CreateActorTemplate`), built from the Task's image and environment, and the image must be pinned
-by digest (Substrate's `SandboxConfig` validation rejects tags: "All images must include a
-digest"). **Agent Substrate is the layer that owns the machines**: in the namespace `ate-system`
-it runs `ate-api-server`, `ate-controller`, the `atelet` DaemonSet, `atenet-router`, the
-`atenet-egress` gateway, a Postgres and a RustFS (`substrate/manifests/ate-install/`), and three
-CRDs under `ate.dev/v1alpha1`, `WorkerPool`, `SandboxConfig` and `CSIDriverConfig`; a
-`WorkerPool` keeps warm worker pods (`ko://.../cmd/ateom-gvisor`), and an actor is one gVisor
-sandbox inside one of those workers, booted from the golden snapshot that
-[The plane on ax](plane.md) describes, suspended and resumed by the controller. Names are limited
-to 63 bytes (`MaxNameLength` in `ax/pkg/apis/v1alpha1/types.go`), which is why the generated Task
-names are short.
+**ax is a Kubernetes application.**
 
-**What k3s changes** (planned; nothing has been applied to a k3s cluster): k3s runs containerd
-with no Docker socket, so the host-Docker path of `ousast plane alerts-engine` does not exist
-there and the engine must become a task (below); Substrate's gVisor workers need `runsc` and a
-`RuntimeClass` on the nodes they are scheduled to; k3s ships Traefik (the ingress the CLI can use
-instead of a kubeconfig) and local-path storage. Images are built by GitHub Actions from tagged
-commits, published to GitHub Container Registry and digest-pinned in the manifests; a private
-package needs an `imagePullSecret` on the worker ServiceAccount. Substrate's and ax's own images
-are built with `ko` from their checkouts and pushed to the same registry. The kind profile keeps
-`localhost:5001` and `ops/ax/up.sh`.
+- It is a Deployment `ax-server` with a Redis beside it, in the namespace `ax-system`
+  (`ax/deploy/ax-server.yaml` in the ax checkout).
+- Its controller runs with `--template=default-template --template-atespace=ax-system`.
+- The `ax` CLI reaches it through a tunnel to `svc/ax-server` over the kube context
+  (`ax/internal/tunnel/tunnel.go`).
+- So `ousast plane run` needs a kube context or an ingress, not a public endpoint.
+
+**ax does not run containers itself.**
+
+- It turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
+  `CreateActorTemplate`).
+- The ActorTemplate is built from the Task's image and environment.
+- The image must be pinned by digest. Substrate's `SandboxConfig` validation rejects tags: "All images
+  must include a digest".
+
+**Agent Substrate is the layer that owns the machines.**
+
+- In the namespace `ate-system` it runs `ate-api-server`, `ate-controller`, the `atelet` DaemonSet,
+  `atenet-router`, the `atenet-egress` gateway, a Postgres and a RustFS
+  (`substrate/manifests/ate-install/`).
+- It defines three CRDs (custom Kubernetes resource types) under `ate.dev/v1alpha1`: `WorkerPool`,
+  `SandboxConfig` and `CSIDriverConfig`.
+- A `WorkerPool` keeps warm worker pods (`ko://.../cmd/ateom-gvisor`).
+- An actor is one gVisor sandbox inside one of those workers.
+- The actor boots from the golden snapshot that [The plane on ax](plane.md) describes. The controller
+  suspends and resumes it.
+- Names are limited to 63 bytes (`MaxNameLength` in `ax/pkg/apis/v1alpha1/types.go`). That is why the
+  generated Task names are short.
+
+**What k3s changes** (planned; nothing has been applied to a k3s cluster):
+
+- k3s runs containerd with no Docker socket. So the host-Docker path of `ousast plane alerts-engine`
+  does not exist there. The engine must become a task (below).
+- Substrate's gVisor workers need no Kubernetes `RuntimeClass`: the worker fetches a pinned gVisor
+  tarball and runs `runsc` inside its own pod. The node checklist is pod security, tarball
+  reachability and local-path storage (B.1).
+- k3s ships Traefik and local-path storage. Traefik is an ingress the CLI can use instead of a kubeconfig.
+- GitHub Actions builds the images from tagged commits. They are published to GitHub Container Registry
+  and digest-pinned in the manifests.
+- A private package needs an `imagePullSecret` on the worker ServiceAccount.
+- Substrate's and ax's own images are built with `ko` from their checkouts. They are pushed to the same
+  registry.
+- The kind profile keeps its local registry (`ops/k8s/profiles/kind.toml`) and `ops/ax/up.sh`.
 
 ### One image, and what concurrency equals
 
-There is **one container image** for every Task: `plane/Dockerfile.runner` (`python:3.12-slim`
-plus this package, PID 1 `/usr/local/bin/ax-task-runner`, which is
-`openultrasast.plane.runner`). Each Task manifest under `plane/tasks/` names the same image by
-digest (the pin is kept in `~/.cache/ousast/ax-src/runner-image`) and selects its work with
-`spec.command`: the runner starts `python -m openultrasast.plane.tasks.<command[0]>`
-(`runner.py`, `task_module`). `repo-facts`, `verify`, `agree`, `features`, `remember`, `alerts`,
-`loop` and `roles` are modules in that image, not images of their own; the engine would be the
-second image (planned). So the production answer to "how many containers" is: one image, one
-actor per Task, and **an actor occupies a whole worker** (its limits are the ActorTemplate's, the
-worker holds one actor at a time). Concurrency is therefore the number of workers in the
-`WorkerPool`, nothing else: `ops/ax/workerpool.yaml.tmpl` declares `replicas: 2` of 1 CPU /
-1.5 GiB, so at most two Tasks run at once on this host, and `ousast plane run --workers` must not
-exceed the pool. Substrate ships an autoscaled pool
-(`substrate/demos/autoscaled-workerpool/README.md`: an HPA on the external metric
-`ate_workerpool_workers{state=at_capacity}` through prometheus-adapter, writing
-`WorkerPool.spec.replicas`; its README says the demo is kind-only today) and request parking
-(`substrate/demos/parking/`: the router holds a resume until a worker frees up). Neither is
-applied by `ops/ax/up.sh`; the production profile's pool manifests and autoscaling rule are
+There is **one container image** for every Task: `plane/Dockerfile.runner`. It is `python:3.12-slim`
+plus this package. PID 1 is `/usr/local/bin/ax-task-runner`, which is `openultrasast.plane.runner`.
+
+Each Task manifest under `plane/tasks/` names the same image by digest. The pin is kept in
+`~/.cache/ousast/ax-src/runner-image`. The manifest picks its work with `spec.command`. The runner
+starts `python -m openultrasast.plane.tasks.<command[0]>` (`runner.py`, `task_module`).
+
+`repo-facts`, `verify`, `agree`, `features`, `remember`, `alerts`, `loop` and `roles` are modules in
+that image. They are not images of their own. The engine would be the second image (planned).
+
+So the production answer to "how many containers" is:
+
+- one image;
+- one actor per Task;
+- **an actor occupies a whole worker**. Its limits are the ActorTemplate's, and the worker holds one
+  actor at a time.
+
+Concurrency is therefore the number of workers in the `WorkerPool`, nothing else.
+`ops/ax/workerpool.yaml.tmpl` declares `replicas: 2` of 1 CPU / 1.5 GiB. So at most two Tasks run at
+once on this host. `ousast plane run --workers` must not exceed the pool.
+
+Substrate ships two scaling aids:
+
+- **An autoscaled pool** (`substrate/demos/autoscaled-workerpool/README.md`). An HPA (Horizontal Pod
+  Autoscaler) reads the external metric `ate_workerpool_workers{state=at_capacity}` through
+  prometheus-adapter. It writes `WorkerPool.spec.replicas`. Its README says the demo is kind-only today.
+- **Request parking** (`substrate/demos/parking/`). The router holds a resume until a worker frees up.
+
+`ops/ax/up.sh` applies neither. The production profile's pool manifests and autoscaling rule are
 **planned**.
 
 ### The three components that are bound to the host today
 
 | Component | Today (the kind profile) | Production (the k3s profile, planned) |
 | --- | --- | --- |
-| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against `kind-ousast`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
+| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against the profile's `kube_context`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
 | Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction |
 | Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path |
 
-The S3 memory store (RustFS tested) is already external; in remote mode it is also the artifact
-path, so it must be reachable from the CLI's host and, through the egress gateway, from the
-workers.
+The S3 memory store (RustFS tested) is already external. In remote mode it is also the artifact path.
+So the CLI's host must reach it, and the workers must reach it through the egress gateway.
 
 ### Scaling the pre-push hook in production
 
-**Today** `ousast pre-push` is a local, deterministic check: it materialises base and head,
-discovers the changed regions, runs the evidence ranker and arbiter on head and compares with base,
-all within `--deadline` (30 s by default) and in `--mode advisory` unless blocking is opted into
-([Pre-push: the delta check](scanning.md#pre-push-the-delta-check-experimental),
-`src/openultrasast/push/`, [Engine and pre-push](ops/README.md#experimental-pre-push-integration)).
-**The plane is not in that path**: no hook submits a Run, no model is called without an explicit
-`--model-config`, and nothing leaves the developer's machine.
+**Today** `ousast pre-push` is a local, deterministic check. It does four things:
+
+1. materialises base and head;
+2. finds the changed regions;
+3. runs the evidence ranker and arbiter on head;
+4. compares the result with base.
+
+All of this runs within `--deadline` (30 s by default). It runs in `--mode advisory` unless you opt
+into blocking. See [Pre-push: the delta check](scanning.md#pre-push-the-delta-check-experimental),
+`src/openultrasast/push/` and [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
+
+**The plane is not in that path.**
+
+- No hook submits a Run.
+- No model is called without an explicit `--model-config`.
+- Nothing leaves the developer's machine.
 
 **The production shape** (planned, none of it built) splits the work by latency:
 
-1. **Blocking, inside the hook's deadline**: what runs today, the quick rules and the delta
-   engine on the changed regions. It stays local and deterministic, so the push is never held for
-   a model or a cluster.
-2. **Asynchronous, on the plane**: the hook or the CI job calls the CLI in remote mode with a
-   *delta Run* whose candidates are the functions the push changed; the Run is the per-case chain
-   of [The plane on ax](plane.md#per-case) (`facts`, `verify` a and b, `agree`, `verify` c on the
-   disputed only, `final`). The caller either waits, with its own deadline, for the CLI to print
-   the status and attribution table, or returns at once and a later `ousast plane status <run>`
-   reports from the store; a commit status check or a review comment would be built on that. The
-   worker pool is sized to the push rate or autoscaled on `at_capacity` workers. An unattended
-   in-cluster reconciler that accepts submissions without a CLI session is the later option named
-   above.
+1. **Blocking, inside the hook's deadline**: what runs today, the quick rules and the delta engine on
+   the changed regions. It stays local and deterministic. So the push never waits for a model or a
+   cluster.
+2. **Asynchronous, on the plane**: the hook or the CI job calls the CLI in remote mode with a *delta
+   Run*.
+    - The Run's candidates are the functions the push changed.
+    - The Run is the per-case chain of [The plane on ax](plane.md#per-case): `facts`, `verify` a and b,
+      `agree`, `verify` c on the disputed only, `final`.
+    - The caller can wait, with its own deadline, for the CLI to print the status and attribution table.
+    - Or the caller returns at once, and a later `ousast plane status <run>` reports from the store.
+    - A commit status check or a review comment would be built on that.
+    - The worker pool is sized to the push rate, or autoscaled on `at_capacity` workers.
+    - An unattended in-cluster reconciler that accepts submissions without a CLI session is the later
+      option named above.
 
-**Cost per push, estimated from measured rates.** Two measured per-candidate rates exist for one
-model verdict: `$0.003215` (`spend.cost_per_candidate_evaluate_usd` in
-`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`) and `$0.0021`
-(exp-002 arm A, `overall.usd_per_candidate.A` in
-`benchmarks/measurements/2026-10-02-exp-002/record.json`). A push that changes five functions needs
-two verify passes per candidate: 5 x 2 x $0.0021 = $0.021, or 5 x 2 x $0.003215 = $0.032 at the
-higher rate, plus a third pass on the disputed candidates only. That is where "about two cents per
-push" comes from; it is arithmetic over the decision engine's measured rates, not a measurement of
-a push Run, and `ousast plane status` would print the real figure per Task.
+**Cost per push, estimated from measured rates.** Two measured per-candidate rates exist for one model
+verdict:
+
+- `$0.003215` (`spend.cost_per_candidate_evaluate_usd` in
+  `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`);
+- `$0.0021` (exp-002 arm A, `overall.usd_per_candidate.A` in
+  `benchmarks/measurements/2026-10-02-exp-002/record.json`).
+
+A push that changes five functions needs two verify passes per candidate. That gives 5 x 2 x $0.0021 =
+$0.021, or 5 x 2 x $0.003215 = $0.032 at the higher rate. A third pass runs on the disputed candidates
+only. That is where "about two cents per push" comes from. It is arithmetic over the decision engine's
+measured rates, not a measurement of a push Run. `ousast plane status` would print the real figure per
+Task.
 
 **What is not built** (each is a named gap, not a configuration):
 
 - The delta-Run command. `ousast plane scan <repo> --base <commit> --head <commit>` **does not
-  exist**; `ousast plane` has `run`, `status`, `doctor`, `remember`, `memory-normalise`,
+  exist**. `ousast plane` has `run`, `status`, `doctor`, `remember`, `memory-normalise`,
   `workspaces`, `harvest` and `alerts-engine` (`src/openultrasast/cli.py`).
 - A candidates task for an arbitrary repository. `repo-facts` takes its candidates from a
-  population's validation set; a step that derives them from a diff is not written.
+  population's validation set. No step derives them from a diff yet.
 - Workspace generation from a URL and a commit outside a population. `plane/generate.py` is
-  population-driven (`read_cases` reads a validation set and a population TOML), so a Run for a
+  population-driven: `read_cases` reads a validation set and a population TOML. So a Run for a
   repository that is not in a population cannot be generated today.
-- The remote execution mode itself: the `local`/`remote` switch, configurable cluster context,
-  ax-server and router addresses (the `kind-` prefix is hard-coded, B.4), presigned delivery from
-  the runner to the store, and the CLI reading completion and `summary.json` from the store
-  instead of its receiver thread.
+- The remote execution mode itself:
+    - the `local`/`remote` switch;
+    - a configurable cluster context and ax-server and router addresses (the `kind-` prefix is
+      hard-coded, B.4);
+    - presigned delivery from the runner to the store;
+    - the CLI reading completion and `summary.json` from the store instead of its receiver thread.
 - The engine task and its WorkerPool, the GHCR build, the production pool and autoscaling
   manifests, and the k3s runbook and checklist.
 - The check or comment that carries the asynchronous verdict back to the push.
@@ -433,8 +586,7 @@ a push Run, and `ousast plane status` would print the real figure per Task.
 ### Sizing
 
 The worker is the unit. Every Task's `limits` fit the 1 CPU / 1.5 GiB worker of
-`ops/ax/workerpool.yaml.tmpl`; the requests are what the Task declares, the worker is what it
-occupies.
+`ops/ax/workerpool.yaml.tmpl`. The requests are what the Task declares. The worker is what it occupies.
 
 | Task (`plane/tasks/*.yaml`) | Requests | Limits | Per push |
 | --- | --- | --- | --- |
@@ -446,17 +598,18 @@ occupies.
 | `roles` | 250m / 384Mi | 1 CPU / 1Gi | not in a push Run (harvest Runs only) |
 | `engine` (planned) | from a measured run | above 3 GiB (the host run uses `--memory 3g`) | its own pool, not the default one |
 
-- **Actors per push**: 4 (`facts`, `va`, `vb`, `agree`), 6 with a tie-break (`vc`, `final`);
+- **Actors per push**: 4 (`facts`, `va`, `vb`, `agree`), or 6 with a tie-break (`vc`, `final`).
   `features` and `remember` add two more when the verdict is to be kept.
-- **Critical path**: `facts` then `va` and `vb` side by side, then `agree`, then (sometimes) `vc`
-  and `final`: three to five Task boots in series, each a golden-snapshot restore plus the model
-  calls of the verify passes.
-- **One actor per worker**: `agree` asks for 256Mi and still holds a 1.5 GiB worker while it runs,
-  because the worker's capacity, not the Task's request, is the unit. A pool of `W` workers runs
-  at most `W/2` pushes through their verify phase at once; on this host (`W = 2`) that is one
-  push at a time, and every further push waits (the reconciler's `--workers` bound, or Substrate's
-  request parking if the pool is saturated). Autoscaling the pool on `at_capacity` workers is the
-  upstream answer and is **planned** here.
+- **Critical path**: `facts`, then `va` and `vb` side by side, then `agree`, then sometimes `vc`
+  and `final`. That is three to five Task boots in series. Each boot is a golden-snapshot restore plus
+  the model calls of the verify passes.
+- **One actor per worker**: `agree` asks for 256Mi but still holds a 1.5 GiB worker while it runs. The
+  worker's capacity, not the Task's request, is the unit.
+    - A pool of `W` workers runs at most `W/2` pushes through their verify phase at once.
+    - On this host (`W = 2`) that is one push at a time.
+    - Every further push waits, on the reconciler's `--workers` bound or on Substrate's request parking
+      if the pool is saturated.
+    - Autoscaling the pool on `at_capacity` workers is the upstream answer. It is **planned** here.
 
 ## C. The manifests under `plane/`
 
@@ -472,6 +625,12 @@ occupies.
 | `ops/ax/smoke-task.yaml.tmpl`, `smoke-run.yaml.tmpl` | yes | rendered with `RUNNER_IMAGE` from your pin file |
 | `ops/ax/up.sh`, `down.sh` | no | kind-specific; they do not apply to another cluster |
 
-The Run and Task contract itself (`plane/runner.py` as PID 1 of the runner image, `AX_TASK_YAML`
-and `AX_WORKSPACES_YAML`, `/healthz` and `/readyz`, the start request, the tar delivery as
-completion) does not depend on where the cluster is: [The plane on ax](plane.md).
+The Run and Task contract does not depend on where the cluster is. That contract covers:
+
+- `plane/runner.py` as PID 1 of the runner image;
+- `AX_TASK_YAML` and `AX_WORKSPACES_YAML`;
+- `/healthz` and `/readyz`;
+- the start request;
+- the tar delivery as completion.
+
+See [The plane on ax](plane.md).

docs/evaluation.md

@@ -1,7 +1,8 @@
 # Evaluation
 
-What has been measured on code the tool was not tuned on, and how. State as of 2026-10-02.
-Every number below is quoted from a committed record; the path is given with it.
+This page shows what has been measured on code the tool was not tuned on, and how. State as of
+2026-10-02. Every number below is quoted from a committed record, and the record's path is
+given with it.
 
 ## Instruments
 
@@ -11,32 +12,47 @@ Every number below is quoted from a committed record; the path is given with it.
 | Pair corpus | fire on the vulnerable function, stay silent on its fix | `ousast pairs`, `benchmarks/pairs/` |
 | Independent populations | real repositories at vulnerable, fixed and benign pins, scored under a pre-registered protocol | `benchmarks/independent/` |
 
-The detection gate and the local pairs are CI gates. They catch regressions; they do not
+Recall means the share of known bugs found. FP means false positives.
+
+The detection gate and the local pairs are CI gates. They catch regressions. They do not
 estimate recall on real code, because the rules were written against them.
 
 ## Pair corpora
 
-`benchmarks/pairs/catalog.toml` and the slice directories (`local`, `github`, `sast`, `vfc`,
-`vibe-py`, `vfc-js`, `agent-vfc`, `owasp`) are runnable with `ousast pairs --slice <name>`; each
-pair carries a provenance profile and a train/holdout split (`--profile`, `--split`). Pointer
-pairs are not vendored: `--pointers` fetches them into `~/.cache/openultrasast/pairs/`.
-`benchmarks/pairs/README.md` documents the schema and the review tiers.
+A pair is a vulnerable function and its fixed version. You can run the catalog
+`benchmarks/pairs/catalog.toml` and the slice directories with `ousast pairs --slice <name>`.
+The slices are `local`, `github`, `sast`, `vfc`, `vibe-py`, `vfc-js`, `agent-vfc` and `owasp`.
+
+- Each pair carries a provenance profile and a train/holdout split (`--profile`, `--split`).
+- Pointer pairs are not vendored. `--pointers` fetches them into `~/.cache/openultrasast/pairs/`.
+- `benchmarks/pairs/README.md` documents the schema and the review tiers.
 
 **`advisory-fixes`** (`benchmarks/pairs/advisory-fixes/`, 2026-10-01) holds 97 pointer pairs,
-one per repository: a public CVE/GHSA fix commit whose advisory-named function is declared on
-both sides and changed by the fix. It was assembled to give the decision engine real negatives
-for families the corpus had few of (access control, deserialization, output encoding, path,
-untrusted destination). It is a label source of the decision engine
-(`src/openultrasast/learn/sources.toml`), not an `ousast pairs` slice; the research log and
+one per repository. Each is a public CVE/GHSA fix commit. The function the advisory names is
+declared on both sides and changed by the fix.
+
+It was assembled to give the decision engine real negatives (fixed, non-vulnerable code). It
+targets families the corpus had few of: access control, deserialization, output encoding,
+path, untrusted destination. It is a label source of the decision engine
+(`src/openultrasast/learn/sources.toml`), not an `ousast pairs` slice. The research log and
 every rejection are in `research-2026-10-01.md`.
 
 ## Independent populations and the M4 gates
 
-The pre-push safety net's qualification milestone (M4) requires, together: at least 95%
-actionable precision, at least 90% supported recall, at least 95% supported-check completion,
-the runtime gates, and zero fixed-side or benign false alerts. A population is frozen before
-any of its cases is scanned, scored once under a pre-registered protocol, and then **spent**:
-it may inform changes, but it can no longer qualify anything.
+M4 is the qualification milestone of the pre-push safety net. It requires all of these
+together:
+
+- at least 95% actionable precision;
+- at least 90% supported recall;
+- at least 95% supported-check completion;
+- the runtime gates;
+- zero fixed-side or benign false alerts.
+
+A population is a set of real repositories. Its lifecycle has three steps:
+
+1. It is frozen before any of its cases is scanned.
+2. It is scored once under a pre-registered protocol.
+3. It is then **spent**: it may inform changes, but it can no longer qualify anything.
 
 | Population | Status | Result | Record |
 | --- | --- | --- | --- |
@@ -45,9 +61,9 @@ it may inform changes, but it can no longer qualify anything.
 | v2, model-driven pipeline | exploratory (v2 already spent) | stopped after 7 of 17 cases when the account emptied: recall 4/7; precision not adjudicated; no gate measurable | `results-v2-model-pipeline.json` |
 | v3 (PHP, 15 cases; one SSRF slot left empty by the selection rule) | frozen, untouched | none: it is the one-time final check, run once under `protocol-v3.md` after a prediction is committed | `population-v3-php.toml`, `freeze-v3-php.json` |
 
-v3 is reserved: development and analysis work never reads its cases, and
-`tests/test_independent_population.py` fails when anything in the development tree (a recipe,
-catalog, manifest or measurement) names one of its repositories.
+v3 is reserved. Development and analysis work never reads its cases.
+`tests/test_independent_population.py` fails when anything in the development tree names one
+of its repositories. That covers a recipe, catalog, manifest or measurement.
 
 ```mermaid
 stateDiagram-v2
@@ -66,9 +82,10 @@ stateDiagram-v2
 
 ## Folds: how learned decisions are kept out of their own training data
 
-The decision engine is evaluated on repositories it was not shown (`src/openultrasast/learn/folds.py`).
-Every split is by repository group (the label builder's normalised `owner/name`, merged across
-URLs, forks and shared advisories), across all corpora, seeded and deterministic.
+The decision engine is evaluated on repositories it was not shown
+(`src/openultrasast/learn/folds.py`). Every split is by repository group. A group is the label
+builder's normalised `owner/name`, merged across URLs, forks and shared advisories. Splits
+cover all corpora and are seeded and deterministic.
 
 ```mermaid
 flowchart TD
@@ -81,44 +98,50 @@ flowchart TD
     rest --> lofo["Leave-one-framework-out: frameworks with at least 10 groups"]
 ```
 
-`assert_disjoint` runs before a compile: the compile split and the evaluation folds share no
-group. A framework with fewer than 10 groups gets `insufficient data` instead of a fold. As of
-2026-10-02 the outer folds have been run for the six evaluable families; leave-one-source-out
-and leave-one-framework-out have not
+`assert_disjoint` runs before a compile. It checks that the compile split and the evaluation
+folds share no group. A framework with fewer than 10 groups gets `insufficient data` instead of
+a fold. As of 2026-10-02, the outer folds have been run for the six evaluable families.
+Leave-one-source-out and leave-one-framework-out have not
 ([decision-engine.md](decision-engine.md#second-slice-six-families-2026-10-02)).
 
 ## The plane increment
 
-The first increment of the agentic plane ran the model-driven pipeline's validation set (46
-candidates in 15 cases of v2, 20 declared vulnerable sites) on google/ax and compared it with
-the script pipeline on the same cases, candidates, pins and recorded triage. The pre-registered
-gate: at least 16 of 20 declared sites agreed, at a cost per candidate under $0.022.
+The first increment of the agentic plane ran on google/ax. It used the model-driven pipeline's
+validation set: 46 candidates in 15 cases of v2, with 20 declared vulnerable sites. It was
+compared with the script pipeline on the same cases, candidates, pins and recorded triage.
+
+The pre-registered gate: at least 16 of 20 declared sites agreed, at a cost per candidate under
+$0.022.
 
 - **Increment 1** (2026-09-29, `benchmarks/independent/plane-increment-1.json`): 14 of 20
-  declared sites agreed against the reference's 16; $0.0194 per candidate with recorded triage.
-  **NOT MET** on recall.
-  *Correction (2026-09-30, in the record):* the first version read agree's
-  `declared_sites_matched` (coverage by candidates, 19) as "found in either pass" and concluded
-  detection was unchanged. Recounted from the agreed rows, the plane's passes flagged 17 of 20
-  declared sites in at least one pass against the reference's 19.
-- **Increment 2** (2026-09-30, `benchmarks/independent/plane-increment-2.json`): after the design
-  revisit, a tie-break pass c on the 10 disputed candidates with 2-of-3 agreement. 16 of 20
-  declared sites agreed (reference 16) at $0.0205 per candidate over 46 ($0.02195 over the 43
-  after triage, against the reference's $0.022). **MET**, with two caveats the record carries:
-  the tie-break was chosen after seeing the first measurement on the same validation set, so
-  this compares plane with script and qualifies nothing on an independent population; and the
-  plane's passes still flag fewer declared sites than the reference (17 vs 19). Precision is
-  not part of this gate and remains the open problem.
+  declared sites agreed, against the reference's 16. Cost was $0.0194 per candidate with
+  recorded triage. **NOT MET** on recall.
+  *Correction (2026-09-30, in the record):* the first version misread agree's
+  `declared_sites_matched`. That field is coverage by candidates (19), but it was read as
+  "found in either pass". So the first version concluded detection was unchanged. Recounted
+  from the agreed rows, the plane's passes flagged 17 of 20 declared sites in at least one
+  pass. The reference flagged 19.
+- **Increment 2** (2026-09-30, `benchmarks/independent/plane-increment-2.json`): came after the
+  design revisit. It added a tie-break pass c on the 10 disputed candidates, with 2-of-3
+  agreement. 16 of 20 declared sites agreed (reference 16). Cost was $0.0205 per candidate over
+  46, and $0.02195 over the 43 after triage (reference: $0.022). **MET**, with two
+  caveats the record carries:
+  1. The tie-break was chosen after seeing the first measurement on the same validation set.
+     So this compares plane with script and qualifies nothing on an independent population.
+  2. The plane's passes still flag fewer declared sites than the reference (17 vs 19).
+
+  Precision is not part of this gate and remains the open problem.
 
 ## The decision engine
 
-Two slices have been measured out of repository (injection, 2026-10-01; six families,
-2026-10-02). The engine is not adopted. The state against the gates, in one place with the plane
-increment and the populations above, is on [Where we stand](where-we-stand.md#1-detection-capability-and-goals-no-go-measured);
-the per-family tables are on [decision-engine.md](decision-engine.md).
+Two slices have been measured out of repository: injection (2026-10-01) and six families
+(2026-10-02). The engine is not adopted. [Where we
+stand](where-we-stand.md#1-detection-capability-and-goals-no-go-measured) shows the state
+against the gates in one place, with the plane increment and the populations above. The
+per-family tables are on [decision-engine.md](decision-engine.md).
 
 ## Removal of the earlier agentic extra
 
-The removal of the earlier agentic extra (2026-09-30) was checked for byte-identical
-deterministic outputs (pair replay, quick benchmark, standard scan, `improve --dry-run`):
+The earlier agentic extra was removed on 2026-09-30. The removal was checked for byte-identical
+deterministic outputs: pair replay, quick benchmark, standard scan, `improve --dry-run`. Records:
 `benchmarks/measurements/2026-09-30-harnessx-removal-baseline/` and `-equality/`.

docs/examples.md

@@ -1,7 +1,7 @@
 # End-to-end examples
 
-Each example is self-contained. Artifacts land under
-`<target>/.openultrasast/runs/<scan-id>/` and `.openultrasast/` is gitignored.
+Each example stands on its own. Artifacts land under `<target>/.openultrasast/runs/<scan-id>/`.
+`.openultrasast/` is gitignored.
 
 ## 1. Quick scan a local repo (deterministic, no keys)
 
@@ -21,7 +21,7 @@ ousast scan . --mode standard       # Joern model layer, container network off:
 ousast-with-judge scan . --mode standard   # network on, so the model is asked (needs DEEPSEEK_API_KEY)
 ```
 
-Without a key the graph still decides: its entailed findings are reported, and the LLM's residual
+Without a key, the graph still decides. Its entailed findings are reported. The LLM's residual
 and suspicion questions go unasked.
 
 ## 3. CI gate
@@ -52,10 +52,11 @@ uv run ousast pairs --split holdout --profile human   # filter by declared split
 uv run python -m openultrasast.pair_gate         # CI: local pairs must all pass
 ```
 
-Each case is two isolated trees (vuln file vs patched file, same relative path).
-`pair_correct` means the harness fired on the vuln side and stayed silent on the
-fix. Misses and fix-side leaks become `miss`/`fp` signals for `ousast improve`.
-The catalog and public dataset pointers live in `benchmarks/pairs/`.
+Each case is two isolated trees: the vuln file and the patched file, at the same relative path.
+
+- `pair_correct` means the harness fired on the vuln side and stayed silent on the fix.
+- Misses become `miss` signals and fix-side leaks become `fp` signals for `ousast improve`.
+- The catalog and public dataset pointers live in `benchmarks/pairs/`.
 
 ## 6. Self-improve the ruleset from benchmark feedback
 
@@ -68,10 +69,11 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
 
 ## 7. Running the agentic plane
 
-Model work over whole repositories runs as a Run manifest on ax, one isolated actor
-per task (bring-up: [ops/ax/README.md](ops/ax/README.md)). The provider key is
-read from the environment (or `.env`, which never overrides an exported variable) by
-`ousast` and sent only in each task's start request.
+Model work over whole repositories runs as a Run manifest on ax. Each task gets one isolated
+actor. Bring-up is in [ops/ax/README.md](ops/ax/README.md).
+
+`ousast` reads the provider key from the environment, or from `.env`. `.env` never overrides an
+exported variable. The key is sent only in each task's start request.
 
 ```bash
 uv run ousast plane doctor                               # kind, Agent Substrate, ax controller, runner image
@@ -83,16 +85,25 @@ uv run ousast plane remember validation-46               # re-ingest the run's m
 uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
 ```
 
-Each task in the Run carries its own Model and `budget: {usd, calls}`; a task that
-reaches its ceiling stops as `unfinished` and resumes on a rerun with a larger
-budget. `ousast plane status` prints the token attribution table: calls, prompt,
-cache-hit and output tokens and USD per task.
+Each task in the Run carries its own Model and `budget: {usd, calls}`.
+
+- A task that reaches its ceiling stops as `unfinished`. It resumes on a rerun with a larger
+  budget.
+- `ousast plane status` prints the token attribution table. It shows calls, prompt, cache-hit and
+  output tokens, and USD per task.
 
 ## 8. Drive it from an MCP client (OpenCode / IDE)
 
+MCP (Model Context Protocol) lets an editor or agent call the tool's functions.
+
 ```jsonc
 { "command": "uv", "args": ["run", "ousast", "mcp"] }
 ```
 
-Then: `openultrasast.scan {path}` → `run_dir` → `openultrasast.findings {run_dir}` →
-`openultrasast.explain {run_dir, finding_id}`. See the README section "Agent integrations".
+Then call the tools in this order:
+
+1. `openultrasast.scan {path}` returns a `run_dir`.
+2. `openultrasast.findings {run_dir}` lists the findings.
+3. `openultrasast.explain {run_dir, finding_id}` explains one.
+
+See the README section "Agent integrations".

docs/index.md

@@ -1,13 +1,13 @@
 # OpenUltraSAST
 
-OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
-repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
-and separates what it can prove from what it merely suspects. A scan that could not look at
-something records a degradation instead of reporting a clean result.
+OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
+repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
+keeps what it can prove apart from what it only suspects. When a scan could not look at
+something, it records a degradation. It does not report a clean result.
 
 **State as of 2026-10-02 (v2.0.0):** the pre-push safety net is **NO-GO** for rollout.
-No hook capability is enabled, and no independent population has passed the qualification gates
-(see [Evaluation](evaluation.md)).
+No hook capability is enabled. No independent population (a case set frozen before any scan) has
+passed the qualification gates (see [Evaluation](evaluation.md)).
 
 ## Three layers
 
@@ -17,6 +17,9 @@ No hook capability is enabled, and no independent population has passed the qual
 | **The agentic plane on ax** | Model-driven work over whole repositories, run as a DAG of tasks on google/ax, one isolated actor per task. What the runs learn is kept in a memory store. Maintainer surface: `ousast plane`. | [The plane on ax](plane.md), [Memory](memory.md) |
 | **The decision engine** | A learned classifier that weighs every instrument's signals against similar labelled cases from memory. **In development, not adopted**: no scan, pre-push check or report uses it. Maintainer surface: `ousast learn`. | [Memory and detection](memory-and-detection.md), [Decision engine](decision-engine.md) |
 
+A code property graph is a graph of the program's syntax, control flow and data flow. Joern builds
+it.
+
 ## How they connect
 
 ```mermaid
@@ -33,31 +36,34 @@ flowchart LR
     engine -. "intended, not adopted: BLOCK / ADVISORY" .-> report
 ```
 
-- The **scanners** produce findings and, inside plane Runs, the `alerts` that become memory rows.
-- The **plane** runs model work (verify passes, roles) and model-free bookkeeping (facts, agreement,
-  feature records) per case; its `remember` task turns a case's artifacts into rows of the
-  [memory store](memory.md).
-- Memory feeds two consumers. `ousast improve --memory` turns stored rows into rule-status
-  proposals that pass through the same validator and gate as every other edit. The decision engine
-  retrieves labelled examples from the same store.
+- The **scanners** produce findings. Inside plane Runs, they also produce the `alerts` that become
+  memory rows.
+- The **plane** runs model work per case: verify passes and roles. It also runs model-free
+  bookkeeping per case: facts, agreement and feature records. Its `remember` task turns a case's
+  artifacts into rows of the [memory store](memory.md).
+- Memory feeds two consumers:
+  1. `ousast improve --memory` turns stored rows into rule-status proposals. They pass through the
+     same validator and gate as every other edit.
+  2. The decision engine retrieves labelled examples from the same store.
 - The decision engine's output (dotted line) is the intended path. Nothing user-facing reads it
   today.
 
 ## Where to go next
 
-- The current state in one page, every figure with its record: capability against the M4 gates, the
-  step-by-step approach, ax on the server, token economics before and after, and the open items:
-  [Where we stand](where-we-stand.md).
+- The current state in one page, every figure with its record: [Where we stand](where-we-stand.md).
+  It covers capability against the M4 gates, the step-by-step approach, ax on the server, token
+  economics before and after, and the open items.
 - Run a scan: [Examples](examples.md), and the commands in [Scanning](scanning.md).
 - Understand a finding's evidence and score: [Architecture](architecture.md).
 - The program analysis under a scan, what each representation can see and what it misses:
   [Detection techniques](detection-techniques.md).
 - Operate the plane and its store: [ax on this host](ops/ax/README.md) and
   [RustFS setup](rustfs.md).
-- Where the plane runs today and what a separate Kubernetes cluster would take (planned, not
+- Where the plane runs today, and what a separate Kubernetes cluster would take (planned, not
   done): [Deployment](deployment.md).
 - Where the model spend goes, per Task at runtime and in the maintainers' own coding sessions,
   and what is reused instead of paid again: [Token ergonomics](token-ergonomics.md).
 - Trust boundaries and hardening: [Threat model](threat-model.md).
 
-Every number in these pages is quoted from a committed record whose path is given next to it.
+Every number in these pages is quoted from a committed record. The record's path is given next to
+the number.

docs/memory-and-detection.md

@@ -1,11 +1,12 @@
 # Memory and detection
 
-How what the system remembers is meant to turn into better decisions on code it has not seen.
-This is the design of the learned decision engine (`src/openultrasast/learn/`), which is **in
-development and not adopted**: no scan, pre-push check or report uses it today. Every user-facing
-decision is still the deterministic one described in [Scanning](scanning.md) and
+This page shows how the system's memory is meant to improve decisions on code it has not seen. It
+describes the design of the learned decision engine (`src/openultrasast/learn/`).
+
+The engine is **in development and not adopted**. No scan, pre-push check or report uses it today.
+Every user-facing decision is still the deterministic one described in [Scanning](scanning.md) and
 [Architecture](architecture.md). The engine's commands and status are on
-[Decision engine](decision-engine.md); the store itself is on [Memory](memory.md).
+[Decision engine](decision-engine.md). The store itself is on [Memory](memory.md).
 
 ## From instruments to a decision
 
@@ -25,89 +26,128 @@ flowchart TD
     op -- "otherwise" --> adv["ADVISORY"]
 ```
 
-The intended place for these decisions is changed code: `learn/features.py` builds feature records
-for a whole-repository scan or for a base..head delta (`build_for_scan` with `Delta`), the same
-delta `ousast pre-push` checks. Integration into scan and pre-push is still ahead.
+These decisions are meant for changed code. `learn/features.py` builds feature records in two ways:
+
+- for a whole-repository scan;
+- for a base..head delta (`build_for_scan` with `Delta`). This is the same delta `ousast pre-push` checks.
+
+Integration into scan and pre-push is still ahead.
 
 ## 1. Instruments produce signals, not verdicts
 
 Each instrument contributes one block to a candidate's feature record (`learn/schema.py`,
-`INSTRUMENTS`): `language`, `quick` (quick rules), `engine` (the Joern model layer), `facts` (repo
-facts, such as callers and function span), `source`, `entry_points`, `roles` (model source, sink
-and sanitizer roles), `delta`, `verify` and `agree` (the plane's verify passes and their
-agreement), and `model_sinks` (the archived stage-1 classifier, `benchmarks/archive/model_sinks.py`, whose
-logic now lives in `plane/tasks/roles.py`). A candidate is `(path, function, family)` of one repository and pin
-where at least one instrument emitted a signal of that family (`learn/features.py`).
+`INSTRUMENTS`). The blocks are:
 
-An instrument that did not run, had no coverage or failed records that state (`none`, `failed`,
-`not_applicable`), never a zero, so "silent" and "did not look" stay distinguishable. None of the
-instruments decides alone.
+- `language`;
+- `quick` (quick rules);
+- `engine` (the Joern model layer);
+- `facts` (repo facts, such as callers and function span);
+- `source`;
+- `entry_points`;
+- `roles` (model source, sink and sanitizer roles);
+- `delta`;
+- `verify` and `agree` (the plane's verify passes and their agreement);
+- `model_sinks` (the archived stage-1 classifier, `benchmarks/archive/model_sinks.py`). Its logic now
+  lives in `plane/tasks/roles.py`.
 
-**Why profile v1 withholds the engine and function length.** A leak audit
-(`ousast learn audit-leaks`, `benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`)
-found that both separate the vulnerable and fixed sides of a pair by construction: a fix removes
-a sink, and a guard lengthens the function. That holds on the pair corpus, not on unseen code, so
-`INPUT_PROFILES["v1"]` drops the `engine` block and `facts.function_lines` from prompts and from
-the retrieval distance.
+A candidate is `(path, function, family)` of one repository and pin. At least one instrument must have
+emitted a signal of that family there (`learn/features.py`).
+
+An instrument may not run, may have no coverage, or may fail. It then records that state (`none`,
+`failed`, `not_applicable`), never a zero. So "silent" and "did not look" stay distinguishable. No
+instrument decides alone.
+
+**Why profile v1 withholds the engine and function length.** A leak audit found that both separate the
+vulnerable and fixed sides of a pair by construction (`ousast learn audit-leaks`,
+`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`):
+
+- a fix removes a sink;
+- a guard lengthens the function.
+
+That holds on the pair corpus, not on unseen code. So `INPUT_PROFILES["v1"]` drops the `engine` block
+and `facts.function_lines` from prompts and from the retrieval distance.
 
 ## 2. Labels come from ground truth only
 
 `ousast learn labels` (`learn/labels.py`) reads only the sources on a fail-closed allow-list
-(`learn/sources.toml`): `population-v1`, `population-v2`, `dev-php` (reviewed recipe sites),
-`pairs`, `advisory-fixes`, `adjudications` and `assumed_benign`. A file the list does not name is
-refused before it is opened; the reserved population v3 is excluded unread.
+(`learn/sources.toml`). The sources are `population-v1`, `population-v2`, `dev-php` (reviewed recipe
+sites), `pairs`, `advisory-fixes`, `adjudications` and `assumed_benign`. A file the list does not name
+is refused before it is opened. The reserved population v3 is excluded unread.
 
-- **Positives** are declared sites and the vulnerable side of a fix; **negatives** are the same
-  function on the fixed side where the fix changed it, and recorded `False`/`FP` adjudications.
-  A vulnerable-pin candidate matching no site is unlabelled, never negative.
-- **Assumed benign** (maintainer decision 2026-09-30): ordinary non-security commits of the training
-  repositories, at least 90 days from any fix, with no later security commit touching the same
-  files. They are their own source with weight 0.5, conditional on a signal at the tip and none at
-  the base, and reported separately, never merged with verified negatives.
-- **Plane verdicts are features, not labels.** "Agreed", "rejected" or "disputed" is the output of
-  the verify instrument; it enters the record as the `verify` and `agree` blocks. Using it as a
-  label would teach the engine to reproduce the plane, mistakes included, so "rejected by the
-  plane" is never a negative (`learn/labels.py`: "A plane verdict ... is never a label").
+- **Positives** are declared sites and the vulnerable side of a fix.
+- **Negatives** are two kinds:
+  - the same function on the fixed side, where the fix changed it;
+  - recorded `False`/`FP` adjudications.
+- A vulnerable-pin candidate matching no site is unlabelled, never negative.
+- **Assumed benign** (maintainer decision 2026-09-30) covers ordinary non-security commits of the
+  training repositories. Each is at least 90 days from any fix. No later security commit touches the
+  same files.
+  - They are their own source with weight 0.5, conditional on a signal at the tip and none at the base.
+  - They are reported separately, never merged with verified negatives.
+- **Plane verdicts are features, not labels.** "Agreed", "rejected" or "disputed" is the output of the
+  verify instrument. It enters the record as the `verify` and `agree` blocks. Using it as a label would
+  teach the engine to reproduce the plane, mistakes included. So "rejected by the plane" is never a
+  negative (`learn/labels.py`: "A plane verdict ... is never a label").
 
 `ousast learn memory build` joins each label with the candidate's feature record and an excerpt of
-its function and stores one `example` row (no rationale) plus the excerpt blob.
+its function. It stores one `example` row (no rationale) plus the excerpt blob.
 
 ## 3. Retrieval of similar labelled cases
 
-`learn/retrieve.py`, no model:
+`learn/retrieve.py` uses no model. It works in three steps:
 
-1. **Signal profile.** A Gower distance over the allow-listed features of the active profile,
-   per instrument block, with missing values explicit (a different state counts as distance 1).
-   The nearest 40 examples of the candidate's family and the nearest 40 of other families go on.
-2. **Code embeddings.** A cosine re-rank of the excerpts' embeddings (OpenAI
-   `text-embedding-3-small` through OpenRouter, 1,536 dimensions, cached by excerpt hash):
-   `r = 0.5 (1 - D) + 0.5 cos`. Without the candidate's vector the retrieval is signals only.
-3. **Balance.** Six examples: at most half per label, at most 2 per repository group, at most 2
-   contrast examples from other families.
+1. **Signal profile.** It computes a Gower distance (a distance for mixed-type features) over the
+   allow-listed features of the active profile, per instrument block. Missing values are explicit: a
+   different state counts as distance 1. The nearest 40 examples of the candidate's family go on. So do
+   the nearest 40 of other families.
+2. **Code embeddings.** It re-ranks by cosine over the excerpts' embeddings. The embeddings come from
+   OpenAI `text-embedding-3-small` through OpenRouter (1,536 dimensions, cached by excerpt hash). The
+   formula is `r = 0.5 (1 - D) + 0.5 cos`. Without the candidate's vector, the retrieval uses signals only.
+3. **Balance.** It picks six examples:
+   - at most half per label;
+   - at most 2 per repository group;
+   - at most 2 contrast examples from other families.
 
-**The evaluation boundary.** `eligible()` is the only way an example enters a prompt, for retrieved
-examples and compiled demonstrations alike: never the candidate's own repository group, never a
-group the fold evaluates, never the held-out source or framework, and (outside deployment) never
-a near-duplicate of the candidate (cosine >= 0.98). The program repeats the check on the ids it
-actually rendered before any call.
+**The evaluation boundary.** `eligible()` is the only way an example enters a prompt. This holds for
+retrieved examples and compiled demonstrations alike. An eligible example is never:
+
+- from the candidate's own repository group;
+- from a group the fold evaluates;
+- from the held-out source or framework;
+- a near-duplicate of the candidate (cosine >= 0.98), outside deployment.
+
+The program repeats the check on the ids it actually rendered, before any call.
 
 ## 4. The AI classifier, compiled
 
-`learn/program.py` and `learn/compile.py`, DSPy-style and in-house. The prompt is the instruction
-plus the compiled demonstrations (a prefix identical for every candidate, so the provider's prefix
-cache hits), then the retrieved examples, then the candidate's excerpt, signals, roles and family.
-The model (`deepseek-flash`) answers `vulnerable`, `not_vulnerable` or `unsure` with a confidence
-and a rationale that must cite a line of the code. It is sampled `k` times; the score `s` is the
-mean over samples, and a majority `unsure` can never BLOCK. Compilation bootstraps demonstrations
-and searches instructions on a compile split of repository groups, scored by a pre-registered
-balanced Brier score (`learn/compile.toml`); the program is stored as `programs/<id>.json`.
+The classifier lives in `learn/program.py` and `learn/compile.py`. It is DSPy-style (DSPy is a
+framework that compiles prompts from examples) and built in-house.
+
+The prompt has this order:
+
+1. the instruction plus the compiled demonstrations. This prefix is identical for every candidate, so
+   the provider's prefix cache hits;
+2. the retrieved examples;
+3. the candidate's excerpt, signals, roles and family.
+
+The model (`deepseek-flash`) answers `vulnerable`, `not_vulnerable` or `unsure`. It adds a confidence
+and a rationale. The rationale must cite a line of the code. The model is sampled `k` times. The score
+`s` is the mean over samples. A majority `unsure` can never BLOCK.
+
+Compilation bootstraps demonstrations and searches instructions on a compile split of repository groups.
+A pre-registered balanced Brier score scores it (`learn/compile.toml`). The program is stored as
+`programs/<id>.json`.
 
 ## 5. Calibration and the two operating points
 
-`learn/calibrate.py`: Platt scaling of `s`, cross-fitted so that no map is checked on the fold it
-was fitted on. Calibration **holds** when the slope interval contains 1, the intercept interval
-contains 0 and ECE <= 0.05. **BLOCK** is offered only when calibration holds and a fold reaches the
-required precision; otherwise the engine offers **ADVISORY** only.
+`learn/calibrate.py` applies Platt scaling (a logistic map from score to probability) to `s`. It is
+cross-fitted, so no map is checked on the fold it was fitted on.
+
+Calibration **holds** when the slope interval contains 1, the intercept interval contains 0 and ECE
+(expected calibration error) <= 0.05.
+
+**BLOCK** is offered only when calibration holds and a fold reaches the required precision. Otherwise
+the engine offers **ADVISORY** only.
 
 ## The loop: memory improves decisions without retraining
 
@@ -125,25 +165,36 @@ flowchart LR
     gate -- "accepted round" --> ledger["Rule status in the ledger"]
 ```
 
-- **Decisions.** The compiled part of the program is its instruction and demonstrations; the
-  examples are retrieved from memory for each candidate. A new `example` row is therefore eligible
-  for retrieval in the next decision without recompiling the program, subject to the same
-  evaluation boundary. `ousast learn curve` measures the metric over memory size on a fixed
-  candidate subset.
-- **Rules.** `ousast improve --memory` reads the run rows (alerts, verdicts, coverage) and proposes
-  rule-status edits only (shadow a rule that is repeatedly false across repositories; re-enable a
-  shadow rule that keeps hitting agreed declared sites). Rows from holdout pairs, the gated
-  manifest's own cases and any `--qualify-population` are dropped first, and every proposal goes
-  through the same validator and gate as any other edit (`improve/memory.py`).
+- **Decisions.**
+  - The compiled part of the program is its instruction and demonstrations.
+  - The examples are retrieved from memory for each candidate.
+  - So a new `example` row is eligible for retrieval in the next decision without recompiling. The same
+    evaluation boundary applies.
+  - `ousast learn curve` measures the metric over memory size on a fixed candidate subset.
+- **Rules.** `ousast improve --memory` reads the run rows (alerts, verdicts, coverage). It proposes
+  rule-status edits only:
+  - shadow a rule that is repeatedly false across repositories;
+  - re-enable a shadow rule that keeps hitting agreed declared sites.
+
+  Some rows are dropped first: rows from holdout pairs, the gated manifest's own cases and any
+  `--qualify-population`. Every proposal goes through the same validator and gate as any other edit
+  (`improve/memory.py`).
 
 ## What the first slice measured
 
 Record: `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json` (counts
-only). Family **injection**; memory of 724 static examples from all families; input profile v1;
-framework priors off; out-of-repository evaluation (compile split never evaluated, outer grouped
-5-fold, calibration cross-fitted, operating points nested); 95% repository-cluster bootstrap
-intervals. The slice's spend ($0.003215 per candidate at evaluation, k = 5, and its replay at
-$0) is on [Token ergonomics](token-ergonomics.md#measured-costs).
+only). The setup:
+
+- family **injection**;
+- memory of 724 static examples from all families;
+- input profile v1;
+- framework priors off;
+- out-of-repository evaluation: compile split never evaluated, outer grouped 5-fold, calibration
+  cross-fitted, operating points nested;
+- 95% repository-cluster bootstrap intervals.
+
+The slice's spend is on [Token ergonomics](token-ergonomics.md#measured-costs). It was $0.003215 per
+candidate at evaluation, k = 5, and its replay cost $0.
 
 | Measure | Value | What it means |
 | --- | --- | --- |
@@ -152,9 +203,12 @@ $0) is on [Token ergonomics](token-ergonomics.md#measured-costs).
 | ADVISORY | recall 0.90 (0.82 - 0.97), precision 0.54 (0.44 - 0.65) | precision is at the positive share (0.53), so at this threshold ADVISORY is **not discriminating**: it flags nearly as indiscriminately as flagging everything would |
 | BLOCK | not offered | calibration does not hold (ECE 0.064 > 0.05), and every outer fold reported `precision_unreachable` |
 
-**The engine is not adopted.** A second slice (2026-10-02) evaluated the six evaluable families
-the same way: pooled AUC 0.77 to 0.96 per family, within-pair AUC 0.72 to 0.93, BLOCK offered for
-none, re-run agreement below 0.9 for three families
-([Decision engine](decision-engine.md#second-slice-six-families-2026-10-02)). Leave-one-source-out
-and leave-one-framework-out, integration into scan and pre-push, and the one-time check on
-population v3 are still ahead.
+**The engine is not adopted.** A second slice (2026-10-02) evaluated the six evaluable families the
+same way ([Decision engine](decision-engine.md#second-slice-six-families-2026-10-02)). The results: pooled AUC 0.77 to 0.96 per family, within-pair AUC 0.72 to 0.93, BLOCK offered for
+none. Re-run agreement was below 0.9 for three families.
+
+Still ahead:
+
+- leave-one-source-out and leave-one-framework-out;
+- integration into scan and pre-push;
+- the one-time check on population v3.

docs/memory.md

@@ -1,8 +1,8 @@
 # Memory
 
-One store, keyed by repository and pin, holds what plane runs and the decision engine learned.
-The code is `src/openultrasast/plane/memory.py`; the same layout serves both backends.
-How the stored rows feed detection is on [Memory and detection](memory-and-detection.md).
+One store holds what plane runs and the decision engine learned. It is keyed by repository and pin
+(a fixed commit). The code is `src/openultrasast/plane/memory.py`. The same layout serves both backends.
+[Memory and detection](memory-and-detection.md) shows how the stored rows feed detection.
 
 ## What is stored
 
@@ -17,7 +17,7 @@ programs/<program id>.json                 compiled decision-engine programs
 ```
 
 **Rows.** Every row carries `id`, `kind`, `repo`, `pin`, `run`, `task`, `population`, `split`
-and `image` (`ROW_FIELDS`); `id` is the sha256 of (kind, run, task, subject), so a repeated row
+and `image` (`ROW_FIELDS`). `id` is the sha256 of (kind, run, task, subject). So a repeated row
 replaces itself. The kinds (`KINDS`) and who writes them:
 
 | Kind | Written by | Holds |
@@ -35,12 +35,15 @@ replaces itself. The kinds (`KINDS`) and who writes them:
 | `experiment_result` | `ousast learn experiment analyse --record` | the paired estimates, the looks and the adoption verdict (counts and intervals only) |
 | `label`, `decision` | reserved for later decision-engine tasks | declared in `KINDS`; not written by the code on main yet |
 
-**Blobs** (`put_blob`) are content-addressed: a 64-hex name under a fixed prefix. Excerpts are
-written by `learn memory build` (`learn/examples.py`), embeddings by `learn memory embed`
-(`learn/embeddings.py`, keyed by excerpt sha so an excerpt is embedded once), responses by the
-program's caller (`learn/program.py`, keyed by the sha256 of model, parameters digest, messages,
-temperature, sample and format, so a rerun replays at no cost), and compiled programs by
-`learn compile` (`learn/compile.py`).
+**Blobs** (`put_blob`) are content-addressed: each has a 64-hex name under a fixed prefix.
+Content-addressed means the name is the hash of the content. Who writes each blob:
+
+- Excerpts: `learn memory build` (`learn/examples.py`).
+- Embeddings: `learn memory embed` (`learn/embeddings.py`). They are keyed by excerpt sha, so an
+  excerpt is embedded once.
+- Responses: the program's caller (`learn/program.py`). The key is the sha256 of model, parameters
+  digest, messages, temperature, sample and format. So a rerun replays at no cost.
+- Compiled programs: `learn compile` (`learn/compile.py`).
 
 ## Write path
 
@@ -60,13 +63,14 @@ flowchart LR
     put --> store[("Memory store")]
 ```
 
-- `ingest` validates every row first: a malformed file fails before anything is written.
-- Rows are merged into their repository + pin object by `id`. Every object is written whole and the
-  index last, so an interrupted ingest leaves nothing a repeat would not repair.
+- `ingest` validates every row first. A malformed file fails before anything is written.
+- Rows are merged by `id` into their repository + pin object.
+- Every object is written whole, and the index last. So a repeat repairs anything an interrupted
+  ingest left behind.
 - Each row object carries tags (`TAG_FIELDS`: `repo`, `pin`, `kind`, `family`, `run`,
-  `population`, `split`) listing the distinct values of its rows (`*` when they exceed 256
-  characters).
-- The plane's store failures never change a run's result: an ingest can be repeated by hand
+  `population`, `split`). The tags list the distinct values of its rows. A tag is `*` when the
+  values exceed 256 characters.
+- The plane's store failures never change a run's result. An ingest can be repeated by hand
   (`cli.py`, `_plane_memory`).
 
 ## Read path
@@ -85,14 +89,14 @@ flowchart LR
     rules --> gate{"Unchanged validator and gate"}
 ```
 
-- **Seed** (`memory.seed`): a Run task whose `openultrasast.io/memory-key` matches stored facts gets
-  them and is marked done; a changed pin, candidate set or runner image finds nothing and
-  recomputes. The loop's `memory-snapshot` is seeded the same way, because a sandboxed task cannot
-  read the store.
+- **Seed** (`memory.seed`).
+  - A Run task gets stored facts when its `openultrasast.io/memory-key` matches them. It is then marked done.
+  - A changed pin, candidate set or runner image finds nothing and recomputes.
+  - The loop's `memory-snapshot` is seeded the same way. A sandboxed task cannot read the store.
 - **Retrieval** (`learn/retrieve.py`): see
   [Memory and detection](memory-and-detection.md#3-retrieval-of-similar-labelled-cases).
-- **`improve --memory`** (`improve/memory.py`): rows from holdout pairs, from the gated manifest's
-  own cases and from any `--qualify-population` are dropped before any rule sees them. See
+- **`improve --memory`** (`improve/memory.py`). Some rows are dropped before any rule sees them:
+  rows from holdout pairs, from the gated manifest's own cases, and from any `--qualify-population`. See
   [Architecture](architecture.md#proposals-from-plane-memory).
 
 ## Backends: FileStore and the S3 store
@@ -109,26 +113,32 @@ flowchart LR
 
 ### Verify at startup, no fallback
 
-The S3 store never configures its bucket. Each time an `S3Store` is opened for a bucket and
-prefix (once per process), `verify_bucket()` checks, using reads plus one small probe object at
-`<prefix>/_probe/select.jsonl`:
+The S3 store never configures its bucket. It only checks it. An `S3Store` is opened for a bucket and
+prefix once per process. Each time, `verify_bucket()` runs these checks. It uses reads plus one small
+probe object at `<prefix>/_probe/select.jsonl`.
 
-1. versioning is `Enabled` (provenance cites object versions);
-2. an enabled lifecycle rule expires `<prefix>/runs/` after a number of days, as a plain prefix
-   filter, and no enabled rule expires the whole store;
-3. S3 Select answers a probe query over JSON Lines;
-4. the probe's object tags are readable.
+1. Versioning is `Enabled`. Provenance cites object versions.
+2. An enabled lifecycle rule expires `<prefix>/runs/` after a number of days. It uses a plain prefix
+   filter. No enabled rule expires the whole store.
+3. S3 Select (a server-side query over JSON objects) answers a probe query over JSON Lines.
+4. The probe's object tags are readable.
 
-Anything missing is named, with the admin command that fixes it, in one `MemoryStoreError`, and
-the store does not start. There is no local or fetch-and-filter fallback: a filtered read that
-S3 Select cannot answer raises a `MemoryStoreError` instead of returning "no rows". The exact
-messages are listed on [RustFS setup](rustfs.md#what-the-store-verifies-at-startup).
+If anything is missing, the store does not start. One `MemoryStoreError` names each problem and the
+admin command that fixes it.
+
+There is no local or fetch-and-filter fallback. A filtered read that S3 Select cannot answer raises a
+`MemoryStoreError`. It does not return "no rows". [RustFS setup](rustfs.md#what-the-store-verifies-at-startup)
+lists the exact messages.
 
 ### Queryable fields per kind
 
-RustFS's Select infers an object's JSON schema from its leading rows, so a `where` field missing
-there fails with `EvaluatorBindingDoesNotExist` even when a later row has it, and a column that is
-null in the first 1000 rows breaks Select on the whole object (measured 2026-10-02; `memory.py`,
-`S3Store._select`). The store therefore writes every row with its kind's full set of queryable
-fields (`QUERY_FIELDS`), using `""` where a field does not apply, and refuses a `where` on an
-undeclared field. Existing rows are rewritten once with `ousast plane memory-normalise`.
+RustFS's Select infers an object's JSON schema from its leading rows. This causes two failures
+(measured 2026-10-02; `memory.py`, `S3Store._select`):
+
+- A `where` field missing from the leading rows fails with `EvaluatorBindingDoesNotExist`. This
+  happens even when a later row has the field.
+- A column that is null in the first 1000 rows breaks Select on the whole object.
+
+So the store writes every row with its kind's full set of queryable fields (`QUERY_FIELDS`). It uses
+`""` where a field does not apply. It refuses a `where` on an undeclared field. Existing rows are
+rewritten once with `ousast plane memory-normalise`.

docs/plane.md

@@ -1,23 +1,34 @@
 # The plane on ax
 
-Model-driven work over whole repositories runs on a separate agentic plane: a **Run**
-(`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG of steps, and each step is
-an ax **Task** (`plane/tasks/`) that binds its Workspaces (`plane/workspaces/`), at most one ax
-**Model** (`plane/models/`: `deepseek-flash` for chat, `openrouter-embedding` for embeddings) and
-its own `budget: {usd, calls}`. ax (google/ax over Agent Substrate, in a single-node kind cluster
-on the maintainer's host) is the only executor; there is no local subprocess path. Bring-up and
-the host's lessons are in [ax on this host](ops/ax/README.md); where each part runs today (the
-reconciler and the receiver on the host, the tasks in gVisor on Agent Substrate) and what a
-separate Kubernetes cluster would take (planned) are on [Deployment](deployment.md); this page
-draws the flow.
+Model-driven work over whole repositories runs on a separate agentic plane. The plane has these parts:
+
+- A **Run** (`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG (a graph of steps
+  with no cycles).
+- Each step is an ax **Task** (`plane/tasks/`).
+- A Task binds its Workspaces (`plane/workspaces/`).
+- A Task binds at most one ax **Model** (`plane/models/`). `deepseek-flash` serves chat.
+  `openrouter-embedding` serves embeddings.
+- A Task carries its own `budget: {usd, calls}`.
+
+ax is the only executor. It is google/ax, Google's controller that runs agent tasks, over Agent Substrate (its
+sandboxed actor runtime). It runs in a single-node kind cluster (Kubernetes in Docker) on the maintainer's host.
+There is no local subprocess path.
+
+Where to read more:
+
+- Bring-up and the host's lessons are in [ax on this host](ops/ax/README.md).
+- [Deployment](deployment.md) shows where each part runs today. The reconciler and the receiver run on the host.
+  The tasks run in gVisor (a sandboxing kernel) on Agent Substrate. It also covers what a separate Kubernetes
+  cluster would take (planned).
+- This page draws the flow.
 
 Sources: `src/openultrasast/plane/reconciler.py`, `router.py`, `egress.py`, `runner.py`,
 `generate.py`, `memory.py`, and `src/openultrasast/plane/tasks/`.
 
 ## One task, end to end
 
-`ousast plane run RUN.yaml` (the reconciler) does this for every ready task; `--workers` bounds
-the tasks in flight, and a rerun skips tasks already done.
+`ousast plane run RUN.yaml` (the reconciler) does this for every ready task. `--workers` bounds the
+tasks in flight. A rerun skips tasks already done.
 
 ```mermaid
 sequenceDiagram
@@ -48,44 +59,55 @@ sequenceDiagram
 
 What each step guarantees:
 
-- **Credentials** (`router.py`, `runner.py`). The reconciler reads the variable the Model's
-  `secretKey.key` names from the operator's environment or `.env` (which never overrides an
-  exported variable). The key exists only in the start request; the runner keeps it in memory,
-  passes it to the command's environment, redacts it from echoed stderr and never writes it.
-- **Egress** (`egress.py`). One EgressPolicy per actor, deny by default, hostnames only: plain
-  HTTP to the artifact receiver, TLS passthrough to the source hosts of the bound Workspaces and
-  to the hosts the bound Model declares (`openultrasast.io/egress-hosts`). It is written between
-  `ax apply` and the resume, and deleted with the actor.
-- **Inputs** (`reconciler.py`). Artifacts other tasks produced are never rendered into the
-  manifest; the runner fetches them from the receiver after the start, and the receiver serves a
-  running task its declared inputs only.
-- **Completion** (`runner.py`). ax has no Completed phase: the runner's delivery of the output tar
-  is the task's completion. A completion marker stops a resumed actor from repeating billed work;
-  when the marker says the delivery failed, a later boot retries the delivery alone.
-- **Budgets** (`plane/budget.py`). The metered client refuses the next call once a task's `usd` or
-  `calls` ceiling is reached; the task ends `unfinished` and resumes on a rerun with a larger
-  budget. Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
-- **Attribution.** `ousast plane status RUN` prints per task its status, `calls`, `prompt`,
-  `cache_hit` and `output` tokens and `usd` from the tasks' `summary.json`, and writes
-  `attribution.json` in the run directory (`$OUSAST_RESULTS/plane/RUN/`, default
-  `~/ousast-results/plane/`).
+- **Credentials** (`router.py`, `runner.py`).
+  - The reconciler reads the variable that the Model's `secretKey.key` names.
+  - It reads it from the operator's environment or `.env`. `.env` never overrides an exported variable.
+  - The key exists only in the start request.
+  - The runner keeps it in memory and passes it to the command's environment.
+  - The runner redacts it from echoed stderr and never writes it.
+- **Egress** (`egress.py`). Egress means outbound network traffic.
+  - Each actor gets one EgressPolicy. It denies by default and names hostnames only.
+  - It allows plain HTTP to the artifact receiver.
+  - It allows TLS passthrough to the source hosts of the bound Workspaces.
+  - It allows TLS passthrough to the hosts the bound Model declares (`openultrasast.io/egress-hosts`).
+  - The reconciler writes it between `ax apply` and the resume. It is deleted with the actor.
+- **Inputs** (`reconciler.py`).
+  - Artifacts that other tasks produced are never rendered into the manifest.
+  - The runner fetches them from the receiver after the start.
+  - The receiver serves a running task its declared inputs only.
+- **Completion** (`runner.py`).
+  - ax has no Completed phase. The runner's delivery of the output tar is the task's completion.
+  - A completion marker stops a resumed actor from repeating billed work.
+  - If the marker says the delivery failed, a later boot retries only the delivery.
+- **Budgets** (`plane/budget.py`).
+  - The metered client refuses the next call once a task reaches its `usd` or `calls` ceiling.
+  - The task then ends `unfinished`. It resumes on a rerun with a larger budget.
+  - Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
+- **Attribution.**
+  - `ousast plane status RUN` prints these per task: status, `calls`, `prompt`, `cache_hit` and
+    `output` tokens, and `usd`. It reads them from the tasks' `summary.json`.
+  - It writes `attribution.json` in the run directory (`$OUSAST_RESULTS/plane/RUN/`, default
+    `~/ousast-results/plane/`).
 
 !!! note "The golden snapshot"
-    Agent Substrate boots every new actor template once as a *golden* actor to snapshot it, with
-    the same image, Task and environment, and inside the sandbox nothing tells that boot from the
-    real one. So the runner's command never starts by itself: it waits for the start request,
-    which the reconciler sends through `atenet-router` to the task's actor, never to the golden
-    one. Workspaces are prepared only after the start, so the golden boot touches no network and a
-    started actor already runs under its egress policy (a clone at boot failed on the live cluster
-    while the actor was being restored, 2026-09-29; `runner.py`, `_run_once`). The first resume of
-    a new image can time out while the snapshot is built, which is why the reconciler retries it
+    Agent Substrate boots every new actor template once as a *golden* actor and snapshots it. That boot
+    uses the same image, Task and environment. Inside the sandbox, nothing tells that boot from the real one.
+
+    So the runner's command never starts by itself. It waits for the start request. The reconciler sends
+    that request through `atenet-router` to the task's actor, never to the golden one.
+
+    Workspaces are prepared only after the start. So the golden boot touches no network. A started actor
+    already runs under its egress policy. A clone at boot once failed on the live cluster while the actor
+    was being restored (2026-09-29; `runner.py`, `_run_once`).
+
+    The first resume of a new image can time out while the snapshot is built. So the reconciler retries it
     for up to `OUSAST_RESUME_TIMEOUT` seconds (900).
 
 ## The Runs: validation and loop
 
-`ousast plane workspaces POPULATION --validation-set ...` (`generate.py`) writes one chain per
-case; `--loop` adds the `alerts` step per case and the loop's singleton steps. Every step except
-the verify passes (and `roles` in harvest Runs) is model-free.
+`ousast plane workspaces POPULATION --validation-set ...` (`generate.py`) writes one chain per case.
+`--loop` adds the `alerts` step per case and the loop's singleton steps. Every step is model-free
+except the verify passes (and `roles` in harvest Runs).
 
 ### Per case
 
@@ -119,23 +141,27 @@ flowchart LR
     improve --> outcome["gate.json, journal.json, and rule_policy.json only if accepted"]
 ```
 
-- **Before the Run** `ousast plane run` seeds from the store (`memory.seed`): a task whose
-  `openultrasast.io/memory-key` (repository, pin, candidates digest, runner image digest) has
-  stored facts gets them and is marked done, and the sandboxed `memory-snapshot` task gets the
-  store's rows after the guard. **After the Run** it ingests every delivered `memory.jsonl`
-  (`ousast plane remember RUN` repeats this by hand). See [Memory](memory.md).
-- The loop's rules and its guard are described in
-  [Architecture](architecture.md#proposals-from-plane-memory). Adopting an accepted ledger stays a
-  maintainer commit (`src/openultrasast/plane/tasks/loop.py`).
+- **Before the Run**, `ousast plane run` seeds from the store (`memory.seed`).
+  - Each task has an `openultrasast.io/memory-key`: repository, pin, candidates digest and runner image digest.
+  - If the store has facts for that key, the task gets them and is marked done.
+  - The sandboxed `memory-snapshot` task gets the store's rows after the guard.
+- **After the Run**, it ingests every delivered `memory.jsonl`. `ousast plane remember RUN` repeats
+  this by hand. See [Memory](memory.md).
+- [Architecture](architecture.md#proposals-from-plane-memory) describes the loop's rules and its guard.
+  Adopting an accepted ledger stays a maintainer commit (`src/openultrasast/plane/tasks/loop.py`).
 - `ousast plane harvest` writes the decision engine's harvest Runs (verify a/b with agree, and
-  model `roles`); `ousast plane alerts-engine` produces a Run's `alerts` from the Joern engine
-  image on the host for PHP and other languages quick mode does not cover.
+  model `roles`).
+- `ousast plane alerts-engine` produces a Run's `alerts` from the Joern engine image on the host. It
+  covers PHP and other languages that quick mode does not cover.
 
 ## Where this runs in production
 
-ax is a Kubernetes application over Agent Substrate, every Task runs the one runner image as an actor that
-occupies a whole worker, and two profiles of this code base are the target: `kind` on a laptop (today) and `k3s`
-in production (planned), where the same CLI submits Runs remotely and the tasks deliver to the S3 store:
+ax is a Kubernetes application over Agent Substrate. Every Task runs the one runner image as an actor. Each actor
+occupies a whole worker. This code base targets two profiles:
+
+- `kind` on a server VM (today).
+- `k3s` in production (planned). There the same CLI submits Runs remotely, and the tasks deliver to the S3 store.
+
 [Production topology: ax on a Kubernetes cluster](deployment.md#production-topology-ax-on-a-kubernetes-cluster)
 draws it and sizes the pre-push hook at scale.
 

docs/rustfs.md

@@ -1,16 +1,22 @@
 # RustFS setup for the memory store
 
-The plane's memory store can live in an S3-compatible bucket: `OUSAST_MEMORY=s3://sast-memory`
-selects `S3Store` (`src/openultrasast/plane/memory.py`), which talks to the server through boto3.
-Any S3-compatible store with bucket versioning, lifecycle rules, object tags and S3 Select works;
-RustFS is the server the contract tests run against. The maintainer's server is RustFS at
-`files.because-security.com`, bucket `sast-memory`, store at the bucket root. What the store keeps
-there is described on [Memory](memory.md).
+The plane's memory store can live in an S3-compatible bucket. RustFS is an S3-compatible object
+storage server.
 
-**The store never configures its bucket.** An admin sets the bucket up once with admin
-credentials. The agent account the store runs as can read the bucket's settings but not change
-them, and reads and writes objects freely. Every time the store opens, it checks the setup and
-refuses to start if anything is missing.
+- `OUSAST_MEMORY=s3://sast-memory` selects `S3Store` (`src/openultrasast/plane/memory.py`).
+- `S3Store` talks to the server through boto3 (the AWS SDK for Python).
+- Any S3-compatible store works if it has bucket versioning, lifecycle rules, object tags and S3 Select
+  (a server-side query over JSON objects).
+- RustFS is the server the contract tests run against.
+- The maintainer's server is RustFS at `files.because-security.com`. The bucket is `sast-memory`. The
+  store sits at the bucket root.
+
+[Memory](memory.md) describes what the store keeps there.
+
+**The store never configures its bucket.** An admin sets the bucket up once with admin credentials.
+The store runs as an agent account. That account can read the bucket's settings but not change them.
+It reads and writes objects freely. Every time the store opens, it checks the setup. It refuses to start
+if anything is missing.
 
 ```mermaid
 flowchart LR
@@ -35,8 +41,8 @@ aws s3api put-bucket-versioning \
     --versioning-configuration Status=Enabled
 ```
 
-**2. Expire raw run outputs under `runs/` after 30 days.** Everything else (`repos/`, `facts/`,
-`index.jsonl` and the blobs) is kept, so the rule must be a plain prefix filter on `runs/`.
+**2. Expire raw run outputs under `runs/` after 30 days.** Everything else is kept: `repos/`, `facts/`,
+`index.jsonl` and the blobs. So the rule must be a plain prefix filter on `runs/`.
 
 ```bash
 aws s3api put-bucket-lifecycle-configuration \
@@ -46,9 +52,10 @@ aws s3api put-bucket-lifecycle-configuration \
 ```
 
 `put-bucket-lifecycle-configuration` **replaces every rule on the bucket**. If the bucket already
-has rules, merge this one into them first. The rule id `ousast-runs-expiry` and the 30 days are the
-store's defaults (`RUNS_RULE_ID`, `RUNS_EXPIRE_DAYS`); the store accepts any enabled, untagged
-prefix rule on `runs/` with a number of days.
+has rules, merge this one into them first.
+
+The rule id `ousast-runs-expiry` and the 30 days are the store's defaults (`RUNS_RULE_ID`,
+`RUNS_EXPIRE_DAYS`). The store accepts any enabled, untagged prefix rule on `runs/` with a number of days.
 
 To read the result back:
 
@@ -59,8 +66,8 @@ aws s3api get-bucket-lifecycle-configuration --endpoint-url https://files.becaus
 
 ## The agent policy
 
-Attach this policy to the agent account (the account whose keys go into `.env`). It is the
-complete policy: nothing else is needed, and it grants no `Put*` on versioning or lifecycle.
+Attach this policy to the agent account. That is the account whose keys go into `.env`. It is the
+complete policy, and nothing else is needed. It grants no `Put*` on versioning or lifecycle.
 
 ```json
 {
@@ -102,14 +109,14 @@ complete policy: nothing else is needed, and it grants no `Put*` on versioning o
 | `BucketSettingsReadOnly` | `arn:aws:s3:::sast-memory` | The agent can list the bucket and its versions and *read* its location, versioning and lifecycle settings, which is what the startup check needs; it cannot change them. |
 | `ObjectsReadWrite` | `arn:aws:s3:::sast-memory/*` | The agent reads, writes and deletes objects and their versions and reads and writes object tags; this covers row objects, blobs, the index, the Select probe, and the `kind` tag the store filters by. |
 
-So the agent writes objects freely but cannot change bucket settings: turning versioning off or
+So the agent writes objects freely but cannot change bucket settings. Turning versioning off or
 removing the expiry rule needs the admin.
 
 ## The `.env` settings
 
 The store reads its endpoint and credentials from `.env` in the working directory or from the
-environment (`s3_settings`), never from a manifest, and never prints them. The names are the ones
-the AWS SDKs and CLI use for credentials, so one `.env` serves both the store and `aws s3api`.
+environment (`s3_settings`). It never reads them from a manifest. It never prints them. The names are
+the ones the AWS SDKs and CLI use for credentials. So one `.env` serves both the store and `aws s3api`.
 
 | Variable | Value for this server | Notes |
 | --- | --- | --- |
@@ -121,26 +128,33 @@ the AWS SDKs and CLI use for credentials, so one `.env` serves both the store an
 | `S3_REGION` | optional (`us-east-1` when unset) | the SigV4 signing region; spares a GetBucketLocation call |
 | `S3_BUCKET` | `sast-memory` | the bucket `s3://` names when the URL has none; also the real-server tests' default bucket |
 
-The client signs with SigV4 and uses path-style addressing (`https://host/bucket/key`), which
-RustFS needs. The SDK's own checksum trailers are off, because not every S3-compatible server
-accepts them.
+The client signs with SigV4 (AWS's request-signing scheme). It uses path-style addressing
+(`https://host/bucket/key`), which RustFS needs. The SDK's own checksum trailers are off. Not every
+S3-compatible server accepts them.
 
 - **Never commit `.env`.** It is gitignored (`.gitignore`: `.env`, `.env.*`, except
   `.env.example`). `.env.example` lists these names with empty values.
 - **`.env` never overrides a variable already exported in your shell** (`config.load_dotenv`).
-  A stale `export AWS_SECRET_ACCESS_KEY=...` in your profile silently wins over the file; `unset`
+  A stale `export AWS_SECRET_ACCESS_KEY=...` in your profile silently wins over the file. `unset`
   it when a change in `.env` seems to have no effect.
-- The store needs the `s3` extra (`uv sync --extra s3`, which installs boto3); without it opening
-  the store fails with `OUSAST_MEMORY=s3://... needs boto3: install openultrasast[s3]`. A missing
-  variable fails with `OUSAST_MEMORY=s3://... needs S3_ENDPOINT, AWS_ACCESS_KEY_ID,
-  AWS_SECRET_ACCESS_KEY in .env or the environment` (naming only the missing ones); an endpoint
-  without a scheme fails with `S3_ENDPOINT must be a URL with a scheme (https://host[:port])`.
+- The store needs the `s3` extra (`uv sync --extra s3`, which installs boto3). These are the
+  startup errors:
+  - Without the extra, opening the store fails with
+    `OUSAST_MEMORY=s3://... needs boto3: install openultrasast[s3]`.
+  - A missing variable fails with `OUSAST_MEMORY=s3://... needs S3_ENDPOINT, AWS_ACCESS_KEY_ID,
+    AWS_SECRET_ACCESS_KEY in .env or the environment`. The message names only the missing ones.
+  - An endpoint without a scheme fails with
+    `S3_ENDPOINT must be a URL with a scheme (https://host[:port])`.
 
 ## What the store verifies at startup
 
-`S3Store.verify_bucket()` runs once per process for each bucket and prefix. It reads the
-versioning status and the lifecycle rules, writes a two-line probe object at
-`_probe/select.jsonl` (under the prefix, if any), queries it with S3 Select and reads its tags.
+`S3Store.verify_bucket()` runs once per process for each bucket and prefix. It does four things:
+
+1. It reads the versioning status.
+2. It reads the lifecycle rules.
+3. It writes a two-line probe object at `_probe/select.jsonl` (under the prefix, if any).
+4. It queries the probe with S3 Select and reads its tags.
+
 Every missing piece is collected into one error:
 
 ```text
@@ -148,8 +162,8 @@ memory s3://sast-memory: the bucket is not set up for the store. The store never
   - <one line per missing piece>
 ```
 
-The lines, exactly as the code writes them for this bucket at the root (`<failure>` is the
-server's error code and the first 200 characters of its message):
+The table lists the lines exactly as the code writes them for this bucket at the root. `<failure>` is
+the server's error code and the first 200 characters of its message.
 
 | Check | Refusal line |
 | --- | --- |
@@ -161,32 +175,81 @@ server's error code and the first 200 characters of its message):
 | tags unreadable | `object tags cannot be read on _probe/select.jsonl (<failure>): the store filters row objects by their kind tag, so the agent needs s3:GetObjectTagging on arn:aws:s3:::sast-memory/*` |
 | S3 Select missing | `S3 Select (SelectObjectContent over JSON Lines) does not answer on _probe/select.jsonl (<failure>). The store requires it and has no local fallback: use a server with S3 Select (RustFS has it; not every S3-compatible server does) and let the agent s3:PutObject and s3:GetObject` |
 
-The admin commands inside the messages use `--endpoint-url "$S3_ENDPOINT"`, which is the full
-URL from `.env`, so they run as written once `.env` is exported.
+The admin commands inside the messages use `--endpoint-url "$S3_ENDPOINT"`. That is the full URL from
+`.env`. So they run as written once `.env` is exported.
 
-There is **no fallback**: the store does not start in a degraded mode, does not drop to a local
-store, and does not fetch and filter objects itself.
+There is **no fallback**. The store does not:
+
+- start in a degraded mode;
+- drop to a local store;
+- fetch and filter objects itself.
 
 ## The RustFS Select limit and the fixed-schema rule
 
-RustFS's S3 Select infers an object's JSON schema from its leading rows. A `where` field that is
-absent from those rows is unbound, even if a later row has it: the query fails with
+RustFS's S3 Select infers an object's JSON schema from its leading rows. A `where` field absent from
+those rows is unbound, even if a later row has it. The query then fails with
 `EvaluatorBindingDoesNotExist` instead of matching (measured 2026-10-02 with the field first
-present at row 5000). The store does not read that as "no rows", since that answer could silently
-drop matches; it raises:
+present at row 5000).
+
+The store does not read that failure as "no rows". That answer could silently drop matches. It raises:
 
 ```text
 memory s3://sast-memory: S3 Select on <key> failed (EvaluatorBindingDoesNotExist: ...) (a field of [<fields>] is absent from the object's leading rows, which the server reads as its schema); the store has no local fallback
 ```
 
-The rule that follows: a field the store filters on is present in every row of its kind from
-the first row on, as `""` where it does not apply; each row kind has a fixed set of queryable
-fields, enforced at write time (see [Memory](memory.md#queryable-fields-per-kind)).
+The rule that follows has two parts (see [Memory](memory.md#queryable-fields-per-kind)):
+
+- A field the store filters on is present in every row of its kind from the first row on. It is `""`
+  where it does not apply.
+- Each row kind has a fixed set of queryable fields. The store enforces it at write time.
 
 ## Checks against the real server
 
-The contract tests that talk to a real server run only with `OUSAST_MEMORY_TEST_S3=1` (bucket
-`OUSAST_MEMORY_TEST_BUCKET`, else `S3_BUCKET`). They verify the bucket at its root, write only
-under a fresh `contract-<id>/` prefix, and remove every version and delete marker under it
-afterwards; they never configure the bucket. More on the host setup:
-[ax on this host](ops/ax/README.md).
+The contract tests that talk to a real server run only with `OUSAST_MEMORY_TEST_S3=1`. They use the
+bucket `OUSAST_MEMORY_TEST_BUCKET`, else `S3_BUCKET`. The tests:
+
+- verify the bucket at its root;
+- write only under a fresh `contract-<id>/` prefix;
+- remove every version and delete marker under it afterwards;
+- never configure the bucket.
+
+More on the host setup: [ax on this host](ops/ax/README.md).
+
+## Scoped account for engine workers
+
+Create a **new account** for the k3s engine Deployment. Attach only this policy.
+Do not give it the existing memory-store account's policies.
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "ListEnginePrefixes",
+      "Effect": "Allow",
+      "Action": "s3:ListBucket",
+      "Resource": "arn:aws:s3:::sast-memory",
+      "Condition": {
+        "StringLike": {
+          "s3:prefix": ["engine-queue/", "engine-queue/*", "engine-results/", "engine-results/*"]
+        }
+      }
+    },
+    {
+      "Sid": "EngineObjectsOnly",
+      "Effect": "Allow",
+      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
+      "Resource": [
+        "arn:aws:s3:::sast-memory/engine-queue/*",
+        "arn:aws:s3:::sast-memory/engine-results/*"
+      ]
+    }
+  ]
+}
+```
+
+The cluster can read and write only queued engine inputs, claims, and results.
+It cannot list or read the rest of the memory store. It cannot change bucket
+settings. The host queue dispatcher can use the same scoped account. Store its
+keys in the `ousast-engine-s3` Secret; no key values belong in manifests.
+Deployment and coordinator commands are in `ops/k8s/engine/README.md`.

docs/scanning.md

@@ -1,13 +1,21 @@
 # Scanning
 
-`ousast scan PATH --mode quick|standard|deep` scans a checkout; `ousast pre-push` checks only
-what a push (or an explicit `--base`/`--head` pair) changes. The stage-by-stage pipeline,
-evidence ladder and score are in [Architecture](architecture.md); this page shows the flows.
+This page shows the scan flows.
+
+- `ousast scan PATH --mode quick|standard|deep` scans a checkout.
+- `ousast pre-push` checks only what a push changes, or what an explicit `--base`/`--head` pair
+  changes.
+
+The stage-by-stage pipeline, the evidence ladder and the score are in
+[Architecture](architecture.md).
 
 ## The three scan modes
 
-The stages a mode requests are `MODE_STAGES` in `src/openultrasast/stages.py`: `quick` runs
-STATIC, `standard` adds MAP, `deep` adds REGRESS.
+The stages each mode requests are listed in `MODE_STAGES` in `src/openultrasast/stages.py`:
+
+- `quick` runs STATIC.
+- `standard` adds MAP.
+- `deep` adds REGRESS.
 
 ```mermaid
 flowchart TD
@@ -38,7 +46,7 @@ Sources: `README.md` (scan modes), `src/openultrasast/cli.py` (`run_stage` calls
 
 ## Quick rules per language
 
-Rules run only against files of their language. The bundled ruleset lives in
+Each rule runs only against files of its language. The bundled ruleset lives in
 `src/openultrasast/ruleset/<language>/rules.toml`.
 
 | Language | Classes (CWE) |
@@ -49,21 +57,31 @@ Rules run only against files of their language. The bundled ruleset lives in
 | Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
 | PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules |
 
-The detection gate (`python -m openultrasast.gate`) enforces at least 90% recall and under 10%
-false positives per language on the bundled corpora in `benchmarks/manifests/`. Those corpora are
-cheat-sheet fixtures: they catch regressions, they do not estimate recall on real code. **PHP is
-not in the gate**; its numbers are in-sample development numbers
-(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). See `README.md` for the
-gate's per-language output.
+The detection gate (`python -m openultrasast.gate`) enforces two limits per language on the
+bundled corpora in `benchmarks/manifests/`: at least 90% recall and under 10% false positives.
+
+- Those corpora are cheat-sheet fixtures. They catch regressions. They do not estimate recall on
+  real code.
+- **PHP is not in the gate**. Its numbers are in-sample development numbers
+  (`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
+
+See `README.md` for the gate's per-language output.
 
 ## The Joern engine
 
 The model layer (`src/openultrasast/model/`, `src/openultrasast/cpg/`) is the core of `standard`.
-It builds one code property graph per scan with Joern (`joern-parse`, then batched
-`joern --script` queries from `cpg/queries/*.sc`) and sends each question to the arbiter of its
-type: **taint** (source to sink with no discharging sanitizer), **dominance** (does a guard govern
-the obligated operation) and **configuration** (constant abstraction over a value). Taint facts
-ship for C, Java, JavaScript, PHP and Python (`src/openultrasast/ruleset/semantic/*.toml`).
+It builds one code property graph per scan with Joern. A code property graph is a graph of the
+program's syntax, control flow and data flow. The build runs `joern-parse`, then batched
+`joern --script` queries from `cpg/queries/*.sc`.
+
+The layer sends each question to the arbiter (the decider) of its type:
+
+- **taint**: a flow from source to sink with no discharging sanitizer;
+- **dominance**: does a guard govern the obligated operation;
+- **configuration**: constant abstraction over a value.
+
+Taint facts ship for C, Java, JavaScript, PHP and Python
+(`src/openultrasast/ruleset/semantic/*.toml`).
 
 ```mermaid
 flowchart LR
@@ -79,25 +97,33 @@ flowchart LR
     rung -- "residual question only" --> judge["LLM judge: confirm or contradict, never create a flow"]
 ```
 
-The Docker image ships Joern and `php-cli` (which the PHP frontend needs); without Joern the
-layer records `cpg_unavailable` and the scan is otherwise unchanged. Engine installation and the
-Joern features used are in [Engine and pre-push](ops/README.md).
-How the graph is built, queried and refined, with the records behind each refinement, is in
-[Detection techniques](detection-techniques.md).
+The Docker image ships Joern and `php-cli`. The PHP frontend needs `php-cli`. Without Joern, the
+layer records `cpg_unavailable`; the rest of the scan is unchanged.
+
+- Engine installation and the Joern features used: [Engine and pre-push](ops/README.md).
+- How the graph is built, queried and refined, with the records behind each refinement:
+  [Detection techniques](detection-techniques.md).
 
 ## Framework priors
 
-Rule and taint-fact entries that know a framework or library carry a `framework` or `library` tag
-naming a row of `src/openultrasast/ruleset/frameworks.toml` (for example `wordpress`, `flask`,
-`django`, `express`, `spring`). The loaders take `priors=`: `"all"` (the default and today's
-behaviour), `"off"` (language-level knowledge only) or a set of ids, so the contribution of
-framework knowledge can be measured and switched off. The decision engine's first slice ran with
-priors off ([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)).
+Some rule and taint-fact entries know a framework or library. They carry a `framework` or
+`library` tag. The tag names a row of `src/openultrasast/ruleset/frameworks.toml` (for example
+`wordpress`, `flask`, `django`, `express`, `spring`).
+
+The loaders take `priors=` with one of three values:
+
+- `"all"`: the default and today's behaviour;
+- `"off"`: language-level knowledge only;
+- a set of ids.
+
+So the contribution of framework knowledge can be measured and switched off. The decision
+engine's first slice ran with priors off
+([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)).
 
 ## Pre-push: the delta check (experimental)
 
 `ousast pre-push` analyses the commits a `git push` would publish, or an explicit
-`--base`/`--head` pair, and compares head with base so that only new or worsened defects are
+`--base`/`--head` pair. It compares head with base. Only new or worsened defects become
 candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
 [Engine and pre-push](ops/README.md#explicit-local-replay).
 
@@ -105,23 +131,288 @@ candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
 flowchart TD
     input["git push refs on stdin, or --base / --head"] --> resolve["Resolve both commit identities"]
     resolve --> snap["Materialise tracked bytes in private scratch storage"]
-    snap --> regions["Discover changed regions, up to --max-regions"]
-    regions --> engine["Evidence ranker and arbiter on head"]
+    snap --> quick["Quick rules on the changed head lines (advisory, under 1 s)"]
+    quick --> eng{"--engine"}
+    eng -- "off" --> art
+    eng -- "background" --> bg["Detached engine run; result shown on the next push"]
+    bg --> art
+    eng -- "inline, the default" --> engine["Engine on head: changed regions up to --max-regions, ranker, arbiter"]
     engine --> cmp["Targeted base comparison: new or worsened only"]
     cmp --> admit{"Admission policy: capability registry"}
-    admit -- "registry empty today" --> diag["Diagnostic candidates in the artifact"]
+    admit -- "registry empty today" --> diag["Advisory engine findings, not alerts"]
     admit -- "an enabled capability" --> alert["Normal alert"]
-    diag --> art["JSON artifact outside the repository"]
+    diag --> art["Terminal lines + JSON artifact outside the repository"]
     alert --> art
     art --> exit{"--mode"}
     exit -- "advisory, the default" --> allow["Exit 0: allow"]
     exit -- "blocking, explicit opt-in" --> block["May block the push"]
 ```
 
-- `--deadline` (30 s by default) bounds preparation, both revisions and reporting together;
-  `--incomplete-coverage block` opts into failing when coverage is incomplete.
-- The default capability registry is empty pending independent qualification, so no normal alert
-  is emitted today; every candidate stays diagnostic.
+- `--deadline` (30 s by default) bounds preparation, the quick rules, both revisions and reporting together.
+- The quick rules run first on the changed lines. Their matches are shown as advisory pattern
+  matches, never as alerts, and never block.
+- The default capability registry is empty until independent qualification passes. So no normal
+  alert is emitted today. An engine finding the push introduced is shown as an advisory engine
+  finding; every other candidate stays diagnostic in the artifact.
 - A model is called only with an explicit `--model-config`.
-- Hook installation (`ops/install-pre-push`) and the private cache (`--cache-dir`) are documented
-  in [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
+- The private cache (`--cache-dir`) and the manual install recipe are documented in
+  [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
+
+### First run: install the hook on any repository
+
+The hook is advisory. It never rejects a push unless you set `OUSAST_PUSH_MODE=blocking`. A check
+that cannot run says so in one line and lets the push through: a missing `ousast`, a missing
+Docker image, an invalid setting, a crash.
+
+1. Install the tool. The package is not on PyPI. Install it from the repository (or from a
+   release wheel on the GitHub releases page):
+
+    ```bash
+    pip install "git+https://github.com/norandom/OpenUltraSAST"
+    ```
+
+2. Install the hook in the repository you want checked. Git's effective hook directory is used,
+   including `core.hooksPath`, and that setting is never edited:
+
+    ```bash
+    cd /path/to/your/repository
+    ousast pre-push install            # refuses if a pre-push hook already exists
+    ousast pre-push install --force    # moves an existing hook to pre-push.before-ousast and chains it
+    ```
+
+3. Push as usual. Without the Joern engine, you get the quick rules and one line saying the
+   engine did not run. For the engine, choose one:
+
+    - a host install with `joern-parse` on the PATH (`pyinfra @local ops/joern.py`, see
+      [Engine and pre-push](ops/README.md));
+    - Docker: `docker pull ghcr.io/norandom/openultrasast:2.0.1`, then
+      `ousast pre-push install --docker`. The wrapper `ousast-docker` is installed beside the
+      hook and runs the check in the container (repository read-only, no network,
+      `--memory 3g`). The container runs the code of its image's version: the quick-rule
+      tier, the plain skip lines and `--engine` need an image built from a version that has them
+      (`docker build -t openultrasast:dev .`, then `OUSAST_IMAGE=openultrasast:dev`).
+
+4. Remove it with `ousast pre-push uninstall`. Only a hook this tool wrote is removed, and a
+   chained previous hook is restored.
+
+A push on the 2026-10-02 proof repository (serve-static, a new `debug.js` that evaluates
+`req.query.code`, no Joern on the host) prints:
+
+```text
+Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.
+- js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review
+- js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review
+No actionable defects reported; analysis is incomplete.
+Skipped: the changed code has no counterpart in the base revision to compare against
+Skipped: engine did not run: the code graph could not be built (is Joern installed? ...)
+Skipped: no base revision was analysed, so no engine finding can be called new or worsened
+Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.
+Details: /home/you/.cache/openultrasast/push/push-12345.json
+```
+
+The hook reads these settings from the environment of `git push`:
+
+| Variable | Default | Effect |
+| --- | --- | --- |
+| `OUSAST_PUSH_DEADLINE` | `30` | seconds for the whole check |
+| `OUSAST_PUSH_MODE` | `advisory` | `blocking` is the only way a result can reject a push |
+| `OUSAST_INCOMPLETE_COVERAGE` | `allow` | with `blocking`: `block` also rejects incomplete coverage |
+| `OUSAST_PUSH_ENGINE` | `inline` | `background`: return after the quick rules, run the engine detached; `off`: quick rules only |
+| `OUSAST_COMPARISON_BASE` | unset | a local ref to compare a new branch against |
+| `OUSAST_ARTIFACT_DIR` | `~/.cache/openultrasast/push` | where the JSON results go; outside the repository |
+| `OUSAST_COMMAND` | `ousast` | the command the hook runs, for example the Docker wrapper |
+| `OUSAST_IMAGE` | `ghcr.io/norandom/openultrasast:2.0.1` | the Docker wrapper's image |
+| `OUSAST_DOCKER_MEMORY` | `3g` | the Docker wrapper's memory limit |
+
+**A new branch.** Git gives the hook no remote base for a branch the remote does not have yet.
+The check says so ("new branch: the remote has no base to compare against"), runs the quick rules
+over the files the branch's last commit changed instead, and says that too. Set
+`OUSAST_COMPARISON_BASE=main` to compare a new branch against your local `main`.
+
+**The engine and the deadline.** The engine does not fit a 30 s deadline today. The 2026-10-02
+audit measured 260 s on a 26-file repository; graph reuse is the planned fix. When it cannot
+finish, one line says so. With `OUSAST_PUSH_ENGINE=background`, the hook returns after the quick
+rules and starts one detached engine run (`--background-deadline`, 900 s by default). The next push
+in the same repository prints `Engine result for <head> from the previous push: ...` once. Only
+one background run happens at a time. The Docker wrapper cannot run the engine in the background,
+because the container ends with the hook.
+
+**The artifact.** Every run writes one JSON file, and the terminal prints its path after
+`Details:`. It records the following:
+
+- `snapshots[].bytes_read`: proof that the input was read;
+- `quick_tier`: the quick-rule matches, with the files and bytes read;
+- `admission.coverage_reasons`: the raw skip reasons;
+- `admission.dispositions`: every engine candidate and its vetoes;
+- `engine_background`: background runs started or shown;
+- `timings`.
+
+Files accumulate, one per push; delete old ones when you like.
+
+### Skip reasons
+
+The terminal prints one `Skipped:` line per distinct reason in plain words. The artifact keeps the
+raw reason. Repository paths are hex-encoded there, and per-question reasons end in a hash.
+
+| Raw reason (artifact) | Terminal line says | What to do |
+| --- | --- | --- |
+| `cpg_build_failed`, `cpg_unavailable` | the engine did not run: the code graph could not be built | install Joern, or use the Docker wrapper |
+| `cpg_empty` | the engine read no code | check that the changed files are in a supported language |
+| `frontend_unsupported` | no engine frontend for a language in this push | none: that language gets the quick rules at most |
+| `language_not_covered:<lang>:<changed>:<total>` | language not covered: `<lang>` (n changed files, m in the repository) | none: neither the engine nor the quick rules cover it, so silence is not a clean result |
+| `deadline_exhausted` | the engine did not finish within the deadline | raise `OUSAST_PUSH_DEADLINE`, or set `OUSAST_PUSH_ENGINE=background` |
+| `change_context_deadline_exhausted` | the deadline ran out while relating the change to the code | as above |
+| `report_deadline_exhausted` | the deadline ran out while saving the details | as above |
+| `engine_off`, `engine_background` | the engine was switched off, or runs after the push | none |
+| `resolution:missing_base` | new branch: the remote has no base to compare against | set `OUSAST_COMPARISON_BASE` |
+| `resolution:unsupported_base`, `resolution:no_merge_base`, `resolution:ambiguous_base`, `resolution:missing_target`, `resolution:unsupported_target`, `resolution:resolution_error` | the pushed ref or its remote base could not be resolved locally | fetch the remote, then push again |
+| `base_comparison_unavailable`, `base_unavailable` | no base revision was analysed, so nothing can be called new | see the line before it for the cause |
+| `base_counterpart_unresolved` | the changed code has no counterpart in the base to compare against | none: expected for new files |
+| `<path>:ambiguous_line_correspondence`, `<path>:line_correspondence_unavailable`, `<path>:ambiguous_path_rename_correspondence` | changed lines or a renamed file could not be matched to the base | none: findings there stay diagnostic |
+| `unmatched_added_deleted_path_correspondence` | files were both added and deleted, so a move cannot be told from a new file | push the move separately from other edits |
+| `declaration_paths_unavailable` | project declaration files could not be listed | rerun |
+| `capability_unavailable`, `capability_disabled`, `capability_unevaluated`, `capability_ambiguous` | engine findings are not qualified as alerts yet | none: they show as advisory engine findings |
+| `eligibility:<reason>` | the capability registry does not match this build | reinstall a consistent build |
+| `comparison_unknown`, `base_incomplete` and the other novelty reasons | whether this push introduced some engine findings could not be established | none: they stay diagnostic |
+| `dynamic_external_or_depth_context_unresolved:...` | n engine checks depend on dynamic or external code the engine cannot follow | none |
+| `query_failed`, `query_budget_reserved`, `query_too_expensive`, `regions_truncated`, `files_unparsed` | some engine questions were not answered | raise the deadline or `--max-regions` |
+| `cross_partition_semantics_unresolved` | flows between languages are not followed | none |
+| `dependency_unresolved`, `change_unsupported` | engine findings depend on unresolved code or have no supported link to the change | none |
+| `snapshot:<reason>` | some files could not be materialised for analysis | see the reason (symlinks, submodules, LFS) |
+| `<stage>_failed:<Error>`, `push_input_failed:<Error>` | the named step failed | rerun; report it if it repeats |
+
+A reason not in this table is still printed as its own words, followed by "see the details file".
+
+### In CI
+
+The same check runs on a pushed range in CI. This GitHub Actions job is advisory: it never fails
+the build, and it uploads the artifact.
+
+```yaml
+name: ousast pre-push (advisory)
+on: push
+jobs:
+  pre-push:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install "git+https://github.com/norandom/OpenUltraSAST"
+      - name: Check what this push changed
+        if: github.event.before != '0000000000000000000000000000000000000000'
+        run: |
+          ousast pre-push . --base ${{ github.event.before }} --head ${{ github.sha }} \
+            --artifact "$RUNNER_TEMP/ousast-pre-push.json" --engine off --deadline 120 || true
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: ousast-pre-push
+          path: ${{ runner.temp }}/ousast-pre-push.json
+```
+
+A branch's first push has no `github.event.before` to compare against, and the job skips it. For
+the engine in CI, run the job in the container image instead of `pip install`, and drop
+`--engine off`.
+
+### `ousast pre-push --help`
+
+```text
+usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]
+                       [--comparison-base COMPARISON_BASE] [--model-config MODEL_CONFIG]
+                       [--prior-hook PRIOR_HOOK] --artifact ARTIFACT [--cache-dir CACHE_DIR]
+                       [--deadline DEADLINE] [--cancellation-allowance CANCELLATION_ALLOWANCE]
+                       [--max-regions MAX_REGIONS] [--mode {advisory,blocking}]
+                       [--incomplete-coverage {allow,block}] [--engine {inline,background,off}]
+                       [--background-deadline BACKGROUND_DEADLINE]
+                       [--experimental-declarations EXPERIMENTAL_DECLARATIONS]
+                       [--experimental-record-vetoes]
+                       path
+
+Check what a push changes: quick rules on the changed lines, then the Joern engine on head and
+base within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with
+'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall
+[REPO]'.
+
+positional arguments:
+  path                  the repository to check (the hook passes '.')
+
+options:
+  -h, --help            show this help message and exit
+  --base BASE           explicit replay: the base revision (with --head; instead of --remote)
+  --head HEAD           explicit replay: the head revision to check against --base
+  --remote NAME URL     Git hook mode: the remote name and URL Git passes to pre-push; the push
+                        input is read from stdin
+  --comparison-base COMPARISON_BASE
+                        hook mode: a local ref to compare a NEW branch against (the remote has no
+                        base for it); the hook reads OUSAST_COMPARISON_BASE
+  --model-config MODEL_CONFIG
+                        explicit optional witness selection endpoint configuration
+  --prior-hook PRIOR_HOOK
+                        explicitly chain an existing hook with the same input and arguments
+  --artifact ARTIFACT   where to write the complete JSON result; must be outside the repository
+  --cache-dir CACHE_DIR
+                        reuse compatible local artifacts in a private directory outside the
+                        repository
+  --deadline DEADLINE   seconds for the whole check: resolution, quick rules, engine and reporting
+                        (default 30; hook: OUSAST_PUSH_DEADLINE)
+  --cancellation-allowance CANCELLATION_ALLOWANCE
+                        seconds after the deadline to stop work and save the result (default 2)
+  --max-regions MAX_REGIONS
+                        most code regions the engine examines, highest-ranked first (default 500)
+  --mode {advisory,blocking}
+                        advisory (default) never blocks; blocking is the explicit opt-in that
+                        rejects a push for an admitted alert (hook: OUSAST_PUSH_MODE)
+  --incomplete-coverage {allow,block}
+                        with --mode blocking only: also block when coverage is incomplete (hook:
+                        OUSAST_INCOMPLETE_COVERAGE)
+  --engine {inline,background,off}
+                        inline (default): run the Joern engine inside the deadline; background:
+                        return after the quick rules and run the engine detached, showing its
+                        result on the next run; off: quick rules only
+  --background-deadline BACKGROUND_DEADLINE
+                        seconds the detached engine run of --engine background may take (default
+                        900)
+  --experimental-declarations EXPERIMENTAL_DECLARATIONS
+                        explicit --base/--head replay only: unreviewed capability declarations
+                        used to render the experimental evaluation explanation; never consulted by
+                        admission and cannot enable any capability
+  --experimental-record-vetoes
+                        explicit --base/--head replay only: record each comparison and admission
+                        veto per raw finding instead of applying it; output is labeled
+                        experimental, changes no result and enables no hook capability
+```
+
+```text
+usage: ousast pre-push install [-h] [--force] [--docker] [repository]
+
+Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory
+(core.hooksPath is honoured, never edited).
+
+positional arguments:
+  repository  the repository (default: current directory)
+
+options:
+  -h, --help  show this help message and exit
+  --force     install: replace our hook, or move a foreign hook to pre-push.before-ousast and
+              chain it; uninstall: remove a hook this tool did not write
+  --docker    also install the ousast-docker wrapper beside the hook, so the check runs in the
+              container image
+```
+
+```text
+usage: ousast pre-push uninstall [-h] [--force] [repository]
+
+Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.
+
+positional arguments:
+  repository  the repository (default: current directory)
+
+options:
+  -h, --help  show this help message and exit
+  --force     install: replace our hook, or move a foreign hook to pre-push.before-ousast and
+              chain it; uninstall: remove a hook this tool did not write
+```