unseen-eval: sourcing memory -- compact journal records, bounded caches, streamed history (4.6 GB -> 107 MB RSS)

The sourcing run was killed four times (exit 143/137) after growing to 4.6 GB
RSS on the 7.7 GB VM. Live after the fix: flat 107 MB through 3,029
advisories. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

.agents/skills/kiro-debug/SKILL.md

@@ -55,6 +55,14 @@ Extract:
 - The command that produced the failure
 - Whether the failure is deterministic or intermittent
 
+### 1b. Verify the Instrument Before You Believe the Measurement
+Every wrong diagnosis recorded in this project had the same shape: the input was unreadable and the tool reported a plausible zero (`file_exists()` false, the parser wrote nothing, the frontend exited 0), and "no errors, 0 findings" read exactly like a clean result. Before trusting any measurement in the failure report:
+- **Prove the tool read its input.** Make it open one file and print the size. A zero from an unread tree and a zero from clean code are indistinguishable afterwards.
+- **Distrust a number that is too good.** A Joern script "finishing" in 0.5 s when a JVM takes 15 s to start is a failed launch. A harness must fail loudly with the exit code.
+- **Prefer the shipped environment** (the Docker image has a native `php`); local wrappers that proxy stdio are not valid instruments for `php2cpg`.
+- **When a result contradicts a working baseline, suspect the instrument first.**
+A plausible zero is a failed instrument until proven otherwise; classify it as `CONFIG_GAP` or `RUNTIME_MISMATCH`, not as `LOGIC_ERROR` in the code under test.
+
 ### 2. Inspect Local Runtime and Repository State
 Inspect the repository for local evidence:
 - `package.json`, `pyproject.toml`, `go.mod`, `Makefile`, `README*`
@@ -134,6 +142,7 @@ If the issue is fixable by repo changes inside the current task plan, do not esc
 | “Let’s try a few fixes” | Multi-fix guessing hides root cause. |
 | “The spec is probably wrong, I’ll adapt it” | Spec conflicts must be surfaced explicitly. |
 | “The docs search is optional” | For runtime/dependency issues, docs and version issues often contain the shortest path to root cause. |
+| “It ran clean: no errors, zero findings” | A tool that never read its input reports the same. Prove the read (one file, its size) before believing the zero. |
 
 ## Output Format
 

.agents/skills/kiro-discovery/SKILL.md

@@ -20,7 +20,7 @@ description: Entry point for new work. Determines the best action path or work d
 Gather **only metadata** to determine the action path. Do NOT read full file contents yet.
 
 - **Specs inventory**: Scan `.kiro/specs/*/spec.json` for `name`, `phase` fields and `approvals` status. Note feature names and their current status.
-- **Steering existence**: Check which files exist in `.kiro/steering/` (product.md, tech.md, structure.md, roadmap.md). Do NOT read their contents yet.
+- **Steering existence**: Check which files exist in `.kiro/steering/` (core: overview.md, roadmap.md, safety-net.md; custom files may exist too). Do NOT read their contents yet.
 - **Roadmap check**: If `.kiro/steering/roadmap.md` exists, read it. This contains project-level context (approach, scope, constraints, spec list) from a previous discovery session. Use it to restore project context.
 - **Top-level structure**: List the project root directory to note key directories and files. Do NOT recurse into subdirectories.
 
@@ -60,7 +60,7 @@ For Path A/B, recommend the next action and stop.
 **Only for Path C, D, and E.** Now load the context needed for discovery.
 
 **In main context** (essential for dialogue with user):
-- **Steering documents**: Read product.md and tech.md (if they exist) for project goals, constraints, and tech stack
+- **Steering documents**: Read overview.md and safety-net.md (if they exist) for project goals, constraints, and the current state of the stack
 - **Relevant specs**: If the request is adjacent to an existing spec, read that spec's requirements.md to understand boundaries and avoid overlap
 
 **Delegate to sub-agent** (keeps exploration out of main context):
@@ -198,7 +198,7 @@ Use this roadmap structure:
 - [ ] feature-c -- [one-line description]. Dependencies: feature-a, feature-b
 ```
 
-Then write `.kiro/specs/<feature>/brief.md` for **every** feature listed under `## Specs (dependency order)` using the Path C brief format. This enables parallel spec creation via `/kiro-spec-batch`.
+Then write `.kiro/specs/<feature>/brief.md` for **every** feature listed under `## Specs (dependency order)` using the Path C brief format. Each spec is then created one at a time with `/kiro-spec-init`, in dependency order.
 
 **For Path E (mixed decomposition)**:
 
@@ -219,7 +219,7 @@ Use the same roadmap structure as Path D, plus these additional sections:
 ```
 
 Path E rules:
-- Keep `## Specs (dependency order)` reserved for **new specs only** so `/kiro-spec-batch` can still parse it unchanged
+- Keep `## Specs (dependency order)` reserved for **new specs only** so the list stays a plain, parseable creation order
 - Record existing-spec extensions under `## Existing Spec Updates`
 - Record true no-spec work under `## Direct Implementation Candidates`
 - Write `brief.md` only for the **new specs** listed under `## Specs (dependency order)`
@@ -235,15 +235,12 @@ Suggest the next command and stop. Do NOT automatically run downstream spec gene
 
 - Path A: `/kiro-spec-requirements {feature}` to update the existing spec
 - Path B: Recommend direct implementation without creating a spec
-- Path C: Default to `/kiro-spec-init <feature-name>`
+- Path C: `/kiro-spec-init <feature-name>`, then the step-by-step phases (requirements, design, tasks), each human-reviewed
-  - Optional fast path: `/kiro-spec-quick <feature-name>` when the user explicitly wants to continue immediately
+- Path D: `/kiro-spec-init <first-feature-name>` for the first spec in roadmap.md dependency order; validate that slice before the next one
-- Path D: Default to `/kiro-spec-batch` (creates all specs in parallel based on roadmap.md dependency order)
-  - Optional cautious path: `/kiro-spec-init <first-feature-name>` when the user wants to validate the first slice before batching the rest
 - Path E: Choose the next command based on the new-spec portion of the decomposition
-  - If there is exactly one new spec: `/kiro-spec-init <new-feature-name>`
+  - `/kiro-spec-init <new-feature-name>` for the first (or only) new spec, the rest in dependency order afterwards
-  - If there are multiple new specs: `/kiro-spec-batch`
   - Also note which existing specs should be revisited with `/kiro-spec-requirements <feature>`
-- Re-entry: `/kiro-spec-init <next-feature-name>` or `/kiro-spec-batch` if multiple specs remain
+- Re-entry: `/kiro-spec-init <next-feature-name>` for the next spec that remains
 
 If the decomposition contains only existing-spec updates plus direct implementation candidates, do NOT use Path E. Prefer Path A when one existing spec is the clear home, or recommend the existing-spec update plus direct implementation work without creating roadmap entries.
 

.agents/skills/kiro-impl/SKILL.md

@@ -26,7 +26,7 @@ You operate in two modes:
 If steering/spec context is already available from conversation, skip redundant file reads.
 Otherwise, load all necessary context:
 - `.kiro/specs/{feature}/spec.json`, `requirements.md`, `design.md`, `tasks.md`
-- Core steering context: `product.md`, `tech.md`, `structure.md`
+- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
 - Additional steering files only when directly relevant to the selected task's boundary, runtime prerequisites, integrations, domain rules, security/performance constraints, or team conventions that affect implementation or validation
 - Relevant local agent skills or playbooks only when they clearly match the task's host environment or use case; read the specific artifact(s) you need, not entire directories
 

.agents/skills/kiro-spec-design/SKILL.md

@@ -1,13 +1,18 @@
 ---
 name: kiro-spec-design
-description: Create comprehensive technical design for a specification
+description: Create comprehensive technical design for a specification; optionally run the implementation gap analysis first (--gap) or an interactive design review of an existing design (--review)
 metadata:
-  shared-rules: "design-principles.md, design-discovery-full.md, design-discovery-light.md, design-synthesis.md, design-review-gate.md"
+  shared-rules: "design-principles.md, design-discovery-full.md, design-discovery-light.md, design-synthesis.md, design-review-gate.md, gap-analysis.md, design-review.md"
 ---
 
 
 # Technical Design Generator
 
+**Arguments**: `$1` is the feature. `$2` may be `-y` (auto-approve requirements), `--gap` (run the
+optional gap analysis of Step 1b before designing) or `--review` (run only the optional interactive
+design review of Step 7 on an existing `design.md`). The design review gate of Step 5 always runs
+when a design is generated; the optional steps never replace it.
+
 <background_information>
 - **Success Criteria**:
   - All requirements mapped to technical components with clear interfaces
@@ -24,8 +29,8 @@ metadata:
 
 **Read all necessary context**:
 - `.kiro/specs/$1/spec.json`, `requirements.md`, `design.md` (if exists)
-- `.kiro/specs/$1/research.md` (if exists, contains gap analysis from `/kiro-validate-gap`)
+- `.kiro/specs/$1/research.md` (if exists, contains the gap analysis from Step 1b or an earlier run)
-- Core steering context: `product.md`, `tech.md`, `structure.md`
+- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (this repository's core steering; custom steering files may also exist in `.kiro/steering/`)
 - Additional steering files only when directly relevant to requirement coverage, architecture boundaries, integrations, runtime prerequisites, security/performance constraints, or team conventions that affect implementation readiness
 - `.kiro/settings/templates/specs/design.md` for document structure
 - Read `rules/design-principles.md` from this skill's directory for design principles
@@ -35,6 +40,16 @@ metadata:
 - If `-y` flag provided ($2 == "-y"): Auto-approve requirements in spec.json
 - Otherwise: Verify approval status (stop if unapproved, see Safety & Fallback)
 
+### Step 1b (optional, `--gap`): Implementation Gap Analysis
+
+Recommended for brownfield features; skip for greenfield. Analyze the gap between the approved requirements and the existing codebase to inform the design, following `rules/gap-analysis.md` from this skill's directory.
+
+1. Read `rules/gap-analysis.md` for the analysis framework.
+2. Research in parallel where possible (sub-agents if available, otherwise sequentially): existing implementations, architecture patterns, integration points and extension possibilities; external dependency compatibility and version constraints when needed; the requirements, core steering, task-relevant extra steering and relevant local agent skills or playbooks.
+3. Execute the gap analysis: Grep and Read the codebase, use WebSearch/WebFetch for external dependencies if needed, evaluate multiple implementation approaches (extend / new / hybrid) with trade-offs, flag areas needing further research. Provide information and options, not final implementation choices.
+4. Write the analysis to `.kiro/specs/$1/research.md` (append after a `---` rule if the file exists; never overwrite earlier research) and read it back to verify the write. Use the language specified in spec.json.
+5. Unapproved requirements are a warning here, not a stop: gap analysis can inform requirement revisions. Then continue with Step 2, or stop and report if the user asked for the gap analysis alone.
+
 ### Step 2: Discovery & Analysis
 
 **Critical: This phase ensures design is based on complete, accurate information.**
@@ -137,6 +152,15 @@ After all findings return, synthesize in main context before proceeding.
    - Set `approvals.requirements.approved: true`
    - Update `updated_at` timestamp
 
+### Step 7 (optional, `--review`): Interactive Design Review
+
+A human-paced quality review of an existing `.kiro/specs/$1/design.md`, in addition to the automatic gate of Step 5. Follow `rules/design-review.md` from this skill's directory.
+
+1. Gather context: spec.json, requirements.md, design.md, core steering, review-relevant extra steering and local playbooks; survey existing codebase patterns, naming conventions and component structure for reference (parallel sub-agents if available).
+2. Review: Analysis -> Critical Issues -> Strengths -> GO/NO-GO. Limit to the 3 most important concerns; recognize 1-2 strengths; engage interactively (clarifying questions, alternatives); leverage conversation history for requirements context and user intent. Quality assurance, not perfection seeking; accept acceptable risk; every suggestion must be implementable.
+3. Decide: a clear GO/NO-GO with rationale. GO: `/kiro-spec-tasks $1` (or `-y`). NO-GO: address the critical issues, re-run `/kiro-spec-design $1`, then `/kiro-spec-design $1 --review` again.
+4. If design.md does not exist, stop: "Run `/kiro-spec-design $1` first to generate design document". If the design phase is not marked generated in spec.json, warn but proceed.
+
 ## Critical Constraints
  - **Type Safety**:
    - Enforce strong typing aligned with the project's technology stack.
@@ -199,7 +223,7 @@ Provide brief summary in the language specified in spec.json:
 
 **If Design Approved**:
 - Review generated design at `.kiro/specs/$1/design.md`
-- **Optional**: Run `/kiro-validate-design $1` for interactive quality review
+- **Optional**: Run `/kiro-spec-design $1 --review` for the interactive quality review (Step 7)
 - Then `/kiro-spec-tasks $1 -y` to generate implementation tasks
 
 **If Modifications Needed**:

.agents/skills/kiro-spec-requirements/SKILL.md

@@ -24,7 +24,7 @@ metadata:
    - Read `.kiro/specs/$1/spec.json` for language and metadata
    - Read `.kiro/specs/$1/brief.md` if it exists (discovery context: problem, approach, scope decisions, boundary candidates)
    - Read `.kiro/specs/$1/requirements.md` for project description
-   - Core steering context: `product.md`, `tech.md`, `structure.md`
+   - Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
    - Additional steering files only when directly relevant to feature scope, user personas, business/domain rules, compliance/security constraints, operational constraints, or existing product boundaries
    - Relevant local agent skills or playbooks only when they clearly match the feature's host environment or use case and contain domain terminology or workflow rules that shape user-observable requirements
 
@@ -40,7 +40,7 @@ The following research areas are independent. Decide the optimal decomposition b
 **In main context** (essential for requirements generation):
 - Spec files: spec.json, brief.md, requirements.md (project description)
 - EARS format rules, requirements review gate, requirements template
-- Core steering: product.md, tech.md (directly inform scope and constraints)
+- Core steering: overview.md, safety-net.md (directly inform scope and constraints)
 
 **Delegate to sub-agent** (keeps exploration out of main context):
 - **Codebase hints** (brownfield projects): Spawn a sub-agent to explore existing implementations that inform requirement scope. Ask it to summarize: (1) what already exists, (2) relevant interfaces/APIs, (3) patterns that new requirements should align with. Return a summary under 150 lines.
@@ -133,7 +133,7 @@ Provide output in the language specified in spec.json with:
 **If Requirements Approved**:
 - Review generated requirements at `.kiro/specs/$1/requirements.md`
 - **Optional Gap Analysis** (for existing codebases):
-  - Run `/kiro-validate-gap $1` to analyze implementation gap with current code
+  - Run `/kiro-spec-design $1 --gap` to analyze the implementation gap with the current code before designing
   - Identifies existing components, integration points, and implementation strategy
   - Recommended for brownfield projects; skip for greenfield
 - Then `/kiro-spec-design $1 -y` to proceed to design phase

.agents/skills/kiro-spec-tasks/SKILL.md

@@ -25,7 +25,7 @@ metadata:
 **Read all necessary context**:
 - `.kiro/specs/$1/spec.json`, `requirements.md`, `design.md`
 - `.kiro/specs/$1/tasks.md` (if exists, for merge mode)
-- Core steering context: `product.md`, `tech.md`, `structure.md`
+- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
 - Additional steering files only when directly relevant to requirements coverage, design boundaries, runtime prerequisites, or team conventions that affect task executability
 
 **Validate approvals**:

.agents/skills/kiro-steering/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: kiro-steering
-description: Manage .kiro/steering/ as persistent project knowledge
+description: Manage .kiro/steering/ as persistent project knowledge, including custom steering documents for specialized contexts
 metadata:
   shared-rules: "steering-principles.md"
 ---
@@ -11,9 +11,12 @@ metadata:
 <background_information>
 **Role**: Maintain `.kiro/steering/` as persistent project memory.
 
+**Core steering in this repository**: `overview.md` (what the project is and what is actually true of it, dated current state), `roadmap.md` (milestones and status), `safety-net.md` (product direction). Custom files may exist beside them (for example a domain or standards file); all `.kiro/steering/*.md` are loaded as project memory. The generic bootstrap templates under `.kiro/settings/templates/steering/` are named `product.md`, `tech.md`, `structure.md`; they are starting points for a fresh project, not this repository's file names.
+
 **Mission**:
 - Bootstrap: Generate core steering from codebase (first-time)
 - Sync: Keep steering and codebase aligned (maintenance)
+- Custom: Create specialized steering documents beyond the core files (optional, see below)
 - Preserve: User customizations are sacred, updates are additive
 
 **Success Criteria**:
@@ -27,8 +30,9 @@ metadata:
 
 Check `.kiro/steering/` status:
 
-**Bootstrap Mode**: Empty OR missing core files (product.md, tech.md, structure.md)  
+**Bootstrap Mode**: Empty OR missing core files (here `overview.md`, `roadmap.md`, `safety-net.md`; in a fresh project the template names `product.md`, `tech.md`, `structure.md`)  
 **Sync Mode**: All core files exist  
+**Custom Mode**: The user asks for a specialized steering document (see Custom Steering below)
 
 ---
 
@@ -75,6 +79,20 @@ After all parallel research completes, synthesize patterns for steering files.
 
 ---
 
+## Custom Steering (optional)
+
+Create a specialized steering document for one domain beyond the core files.
+
+1. **Ask the user** for the domain/topic (e.g. "API standards", "testing approach") and the specific patterns to document
+2. **Check for a template** in `.kiro/settings/templates/steering-custom/`: `api-standards.md`, `testing.md`, `security.md`, `database.md`, `error-handling.md`, `authentication.md`, `deployment.md`. Load the matching one as a starting point; without a template, generate from domain knowledge
+3. **Analyze the codebase** (JIT) for the domain's patterns with Glob/Grep/Read (template and principles in parallel with the domain analysis when multi-agent is available)
+4. **Generate** the document: follow the template structure if any, apply `rules/steering-principles.md`, patterns not exhaustive lists, one domain per file, 100-200 lines (a 2-3 minute read), never secrets
+5. **Create** `.kiro/steering/{name}.md`; ensure it does not duplicate core steering content
+
+Custom files are loaded as project memory like the core files and carry equal weight.
+
+---
+
 ## Granularity Principle
 
 From `rules/steering-principles.md` (in this skill's directory):
@@ -106,9 +124,9 @@ Chat summary only (files updated directly).
 ✅ Steering Created
 
 ## Generated:
-- product.md: [Brief description]
+- overview.md: [What it is, current state]
-- tech.md: [Key stack]
+- roadmap.md: [Milestones, status]
-- structure.md: [Organization]
+- safety-net.md: [Product direction]
 
 Review and approve as Source of Truth.
 ```
@@ -118,8 +136,8 @@ Review and approve as Source of Truth.
 ✅ Steering Updated
 
 ## Changes:
-- tech.md: React 18 → 19
+- overview.md: Current state dated, retired component removed
-- structure.md: Added API pattern
+- roadmap.md: Milestone status updated
 
 ## Code Drift:
 - Components not following import conventions
@@ -128,6 +146,20 @@ Review and approve as Source of Truth.
 - Consider api-standards.md
 ```
 
+### Custom:
+```
+✅ Custom Steering Created
+
+## Created:
+- .kiro/steering/api-standards.md
+
+## Based On:
+- Template: api-standards.md
+- Analyzed: src/api/ directory patterns
+
+Review and customize as needed.
+```
+
 ## Examples
 
 ### Bootstrap
@@ -136,7 +168,12 @@ Review and approve as Source of Truth.
 
 ### Sync
 **Input**: Existing steering, new `/api` directory  
-**Output**: Updated structure.md, flagged non-compliant files, suggested api-standards.md
+**Output**: Updated overview.md, flagged non-compliant files, suggested api-standards.md
+
+### Custom
+**Input**: "Create API standards steering"  
+**Action**: Load template, analyze src/api/, extract patterns  
+**Output**: api-standards.md with project-specific REST conventions
 
 ## Safety & Fallback
 

.agents/skills/kiro-steering/rules/steering-principles.md

@@ -84,7 +84,11 @@ Never include:
 
 ## File-Specific Focus
 
-- **product.md**: Purpose, value, business context (not exhaustive features)
+This repository's core steering files:
-- **tech.md**: Key frameworks, standards, conventions (not all dependencies)
+
-- **structure.md**: Organization patterns, naming rules (not directory trees)
+- **overview.md**: What the project is and what is actually true of it; a dated current-state section wins over older history
-- **Custom files**: Specialized patterns (API, testing, security, etc.)
+- **roadmap.md**: Milestones, their measured status, and the accepted order of work (not a task list)
+- **safety-net.md**: Product direction and acceptance criteria (not feature catalogs)
+- **Custom files**: Specialized patterns (API, testing, security, etc.); they may exist beside the core files and carry equal weight
+
+(The generic bootstrap templates `product.md`, `tech.md`, `structure.md` under `.kiro/settings/templates/steering/` cover purpose, key frameworks and organization patterns for a fresh project.)

.agents/skills/kiro-validate-impl/SKILL.md

@@ -68,10 +68,16 @@ For each detected feature:
 - Read `.kiro/specs/<feature>/requirements.md` for requirements
 - Read `.kiro/specs/<feature>/design.md` for design structure
 - Read `.kiro/specs/<feature>/tasks.md` for task list and Implementation Notes
-- Core steering context: `product.md`, `tech.md`, `structure.md`
+- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
 - Additional steering files only when directly relevant to the validated boundaries, runtime prerequisites, integrations, domain rules, security/performance constraints, or team conventions that affect the GO/NO-GO call
 
-**Discover canonical validation commands**:
+**This repository's gates** (`.github/workflows/ci.yml`; run them as written, judge each by its own exit code):
+- `TEST_COMMANDS`: `pytest -q -p no:cacheprovider < /dev/null; echo "exit=$?"` -- judge ONLY by the printed exit code, never by a summary line; `ruff check src tests`; `ruff format --check src tests`; `mypy src/openultrasast`
+- Detection gates: `python -m openultrasast.gate` (>=90% recall, <10% FP per language; hard, score-independent), `python -m openultrasast.map_gate`, and the pair gate `python -m openultrasast.pair_gate` (local vuln vs fix). A ruleset change that fails a gate is NO-GO.
+- Docs: `mkdocs build --strict` when `docs/` or `mkdocs.yml` changed
+- `SMOKE_COMMANDS`: `ousast --help`, and for a scan change `ousast scan <fixture> --mode quick` whose `manifest.json` names the stages that ran and the files preprocess kept. A plausible zero (0 findings, 0 degradations) from a tool that never read its input is a failed instrument, not a pass: prove the tool read its input first.
+
+**Discover canonical validation commands** (for other repositories, or anything the list above does not cover):
 - Inspect repository-local sources of truth in this order: project scripts/manifests (`package.json`, `pyproject.toml`, `go.mod`, `Cargo.toml`, app manifests), task runners (`Makefile`, `justfile`), CI/workflow files, existing e2e/integration configs, then `README*`
 - Derive a feature-level validation set for this repo: `TEST_COMMANDS`, `BUILD_COMMANDS`, and `SMOKE_COMMANDS`
 - Prefer commands already used by repo automation over ad hoc shell pipelines

.agents/skills/kiro-verify-completion/SKILL.md

@@ -61,7 +61,8 @@ Use the language specified in `spec.json`.
 4. Check exit code, failure count, skipped scope, and missing coverage.
 5. Reject claims that are broader than the evidence.
 6. If mandatory validation cannot be completed, return `MANUAL_VERIFY_REQUIRED`.
-7. Only then allow the claim.
+7. Verify the instrument before you believe the measurement: prove the tool read its input (it opened one file and printed the size; the stage list names what ran). A plausible zero -- "0 findings, 0 errors, exit 0" from a tree the tool never read, or a Joern script that "finished" before a JVM could start -- is a failed instrument, not evidence. For pytest, judge only by the printed exit code (`pytest -q -p no:cacheprovider < /dev/null; echo "exit=$?"`), never by a summary line.
+8. Only then allow the claim.
 
 ## Claim-Specific Rules
 
@@ -116,6 +117,7 @@ Return `NOT_VERIFIED` when:
 | “Build should be fine because lint passed” | Lint does not prove build success. |
 | “Tests passed and build succeeded, so it must run” | Type erasure, module loading, native ABI, and boot-time config issues can still fail at runtime. |
 | “The feature is done because all tasks are checked off” | `FEATURE_GO` also requires coverage, integration, and design alignment. |
+| “The scan found nothing, so the code is clean” | An unread input and clean code both report zero. Prove the tool read its input first. |
 
 ## Output Format
 

.agents/skills/openultrasast-scan/SKILL.md

@@ -0,0 +1,123 @@
+---
+name: openultrasast-scan
+description: OpenUltraSAST scan workflows - `ousast scan` in quick, standard or deep mode, the evidence ladder, `--fail-on` exit codes, recorded degradations, the run directory and `ousast mcp`. Use when running or planning a scan, reading a run directory, or wiring OpenUltraSAST into an agent.
+---
+
+# OpenUltraSAST Scan
+
+Use this skill to run `ousast scan PATH --mode quick|standard|deep` from an agent and to read what
+it wrote.
+
+## Workflow
+
+1. Confirm the target is a local checkout (a path, not a URL).
+2. Run through the project entry point: `uv run ousast ...` inside the repository, or `ousast` from an
+   environment where it is installed. Never activate a virtualenv by hand.
+3. Start with `--mode quick` unless the task needs the Joern model layer (`standard`) or sandboxed
+   regression snippets (`deep`). Check the prerequisites below first; a mode whose engine is missing
+   still exits 0 and records a degradation.
+4. Read the printed `scan_id=` and `run_dir=` lines; the run directory is `.openultrasast/runs/<scan-id>/`.
+5. Report the finding count, how many the verifier accepted, and **every** recorded degradation
+   (`manifest.json`, key `degradations`). No findings plus a degradation is not a clean scan: the scan
+   answered a smaller question than it was asked.
+6. Never call a finding verified from its title or severity. `verification.json` holds the status.
+
+## Commands
+
+```bash
+uv run ousast scan <repo> --mode quick
+uv run ousast scan <repo> --mode standard --fail-on verified
+uv run ousast scan <repo> --mode deep --fail-on worth-fixing
+uv run ousast scan <repo> --mode quick --config openultrasast.toml
+uv run ousast mcp                 # the MCP server over stdio, for agent hosts
+```
+
+## Modes and prerequisites
+
+| Mode | Adds | Needs | Without it |
+| --- | --- | --- | --- |
+| `quick` | language-scoped pattern rules, entry-point hints, ranking, verification, score; deterministic | nothing | - |
+| `standard` | the MAP stage: complexity map, semantic overlay, authorization obligations, the Joern model layer (one code property graph per repository; taint, guard-dominance and configuration questions), deterministic fusion | Joern (shipped in the Docker image, with `php-cli` for PHP); a provider key for the residual LLM question | `cpg_unavailable`; without a key the LLM question is not asked and only the graph's entailed findings are reported |
+| `deep` | REGRESS: promoted candidates run as snippets in a Docker sandbox (no network, read-only source, non-root) and get a `triggerable` / `not_triggerable` / ... verdict in `verdicts.json` | a working `docker` | `sandbox_unavailable` |
+
+Model stages read `DEEPSEEK_API_KEY` (every LLM call); with only `OPENROUTER_API_KEY` set, chat
+calls fall back to OpenRouter. An exported shell variable wins over `.env` silently. `[models]
+hunter` in the config enables the tool hunter in `standard`; it is off by default.
+
+## Exit code: `--fail-on`
+
+| Policy | Exit 1 when |
+| --- | --- |
+| `never` (default) | never |
+| `findings` | any finding was reported |
+| `verified` | any verification result is `verified` (accepted at `static_corroboration` or above) |
+| `worth-fixing` | a deep-mode sandbox verdict is marked `worth_fixing` |
+
+A degradation never changes the exit code. Read `manifest.json` for it.
+
+## Recorded degradations
+
+Each entry in `manifest.json` `degradations` is `{"stage": ..., "reason": ...}`. The ones to expect:
+
+- `cpg_unavailable` (stage `model`): no Joern on the host; the model layer ran nothing.
+- `sandbox_unavailable` (stage `regress`): no working `docker`; deep mode fell back to standard.
+- `learning_endpoint_unavailable` (stage `model`): no provider key resolved. The graph's entailed
+  findings are still reported; the residual question is skipped. This is recorded in `manifest.json`
+  but has no line in `report.md`'s "What could not be analysed" section.
+- `hunter_model_unavailable` (stage `map`): `[models] hunter` is unset, so the tool hunter did not run.
+  Recorded on every standard scan without that setting; it is expected, not an error.
+- coverage facts from the graph, rendered in `report.md` under "What could not be analysed":
+  `cpg_empty`, `cpg_build_failed`, `cpg_sharded`, `files_unparsed`, `query_failed`,
+  `query_budget_reserved`, `query_too_expensive`, `budget_exhausted`, `regions_truncated`,
+  `region_failed`; plus `max_findings_exceeded` and `facts_unavailable` (overlay, obligations).
+
+Prove the instrument read its input before believing a zero: `manifest.json` names the stages that
+ran and the files the preprocess step kept (`preprocess/file_targets.json`).
+
+## Evidence ladder
+
+`src/openultrasast/verification.py` defines the levels in this order. A level moves one rung at a
+time and never backward.
+
+| Level | Meaning | Produced today by |
+| --- | --- | --- |
+| `suspicion` | a rule, model or weak signal raised a hypothesis | the Joern model layer (its rung, `model_entailed` or `suspicion`, travels in the finding's tags), the tool hunter, authorization-obligation checks |
+| `static_corroboration` | deterministic code evidence at the line | quick rules, the semantic overlay |
+| `crash_reproduced` | an artifact-backed reproducer | nothing: there is no crash reproducer. Deep-mode sandbox verdicts (`verdicts.json`, `worth_fixing`) do not raise the level |
+| `root_cause_explained` | independent verifier reasoning | nothing: the LLM judge was retired in v2.0.0 |
+| `exploit_demonstrated` | a demonstrated exploit | nothing |
+| `patch_validated` | relevant checks re-run after a patch | nothing: there is no patch stage |
+
+The verifier's status per finding (`verification.json`): below `static_corroboration` is
+`needs_evidence`; `static_corroboration` with reachability `unknown` is `rejected` (the tie-breaker
+asks for call-graph, route, CLI, parser or dynamic evidence); `static_corroboration` with
+function-level reachability is `accepted`. `verified` means accepted at `static_corroboration` or
+above, and that is what `--fail-on verified` counts. There is no configurable report threshold: the
+`[evidence]` config section was retired in v2.0.0 and is ignored.
+
+## Run artifacts
+
+Under `.openultrasast/runs/<scan-id>/`:
+
+- `report.md`, `report.sarif` (SARIF 2.1.0 for code hosts and IDEs), `manifest.json` (stages, degradations, model payload)
+- `findings.json`, `verification.json`, `score.json`, `shadow_findings.json` (rules shipped as `shadow`)
+- `harness.json`: the resolved harness configuration the run used
+- standard: `fusion.json`, `complexity_map.json`, `overlay.json`; deep: `verdicts.json`
+- `calibration/false_positive_learnings.json`, `rank/ranking.json`, `mapping/`, `preprocess/`, `trace/events.jsonl`
+
+## Changed code only
+
+`ousast pre-push --base <rev> --head <rev> --artifact <file> <repo>` replays a revision pair and
+reports only new or worsened defects (experimental; `--mode advisory` is the default and never
+blocks). The capability registry is empty, so every candidate is diagnostic.
+
+## MCP
+
+`ousast mcp` serves a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools:
+`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`,
+`explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a shell, Docker or a
+free-form command.
+
+```jsonc
+{ "command": "uv", "args": ["run", "ousast", "mcp"] }
+```

.agents/skills/openultrasast-triage/SKILL.md

@@ -0,0 +1,75 @@
+---
+name: openultrasast-triage
+description: OpenUltraSAST finding triage - false-positive adjudication with the fixed reason taxonomy, the evidence ladder, what a verdict may feed (labels, never rule edits), and fixing then re-scanning. Use when reviewing findings, reducing false positives, deciding whether evidence is enough, or auditing a fix for a finding.
+---
+
+# OpenUltraSAST Triage
+
+Use this skill when triaging OpenUltraSAST findings or auditing a fix for one.
+
+## Triage Loop
+
+1. Identify the finding: id, path, rule, CWE and policy severity, confidence, evidence level and
+   verification status (`findings.json`, `verification.json` in the run directory).
+2. Separate deterministic evidence (rule match, graph entailment, attached reachability) from model
+   assumptions.
+3. Check attacker control, reachability, sanitizer behaviour, duplicates, impact, and contradictory
+   evidence.
+4. If rejected or unresolved, record the false-positive reason and its scope.
+5. Prefer scoped demotion over deleting a vulnerability class globally: a rejection in `auth/`
+   demotes `auth/...`, never the class (`calibration.py`).
+6. Record the adjudication. Do not edit a rule because of one finding (see below).
+
+## False-Positive Reasons
+
+The taxonomy is `FalsePositiveReason` in `src/openultrasast/calibration.py`; use these values exactly:
+
+- `unreachable_path`
+- `missing_attacker_control`
+- `sanitizer_disproved`
+- `static_rule_mismatch`
+- `incorrect_model_assumption`
+- `duplicate`
+- `insufficient_impact`
+- `unsupported`
+- `contradicted`
+- `unverified`
+
+## Decision Standard
+
+If evidence is uncertain, mark the finding `needs_evidence` and name the next evidence instead of
+escalating severity. Fusion (`fusion.json`, standard mode) is deterministic: it adjudicates between
+the quick-rule and MAP panels by fixed triggers and is not a second opinion from a model. Independent
+model agreement exists only in the plane's verify passes a and b, with pass c on disputes, and the
+`agree` step; those verdicts arrive as features of the decision engine, never as a verification
+status.
+
+## What an Adjudication Feeds
+
+- Recorded adjudications (a per-finding verdict with a reason) become decision-engine labels through
+  the fail-closed source list `src/openultrasast/learn/sources.toml` (`[[source]] id =
+  "adjudications"`). Aggregate counts label nothing; only per-finding verdicts do.
+- Plane verdicts (`agreed`, `rejected`, `disputed`) are features, never labels. "Rejected by the
+  plane" is not a negative.
+- Rule changes go through the ledger: `ousast improve` (the validator and the detection gate, holdout
+  pairs per round) or a registered experiment (`ousast learn experiment register|units|run|analyse`).
+  Never a hand edit of `rules.toml` per miss or per false positive.
+
+## Fix and Re-scan
+
+1. Intake: restate the finding id, evidence level, affected path, and the acceptance criterion.
+2. Plan: the smallest defensive change that addresses the root cause.
+3. Implement: modify only the files the finding requires.
+4. Adversarial review: look for bypasses, regressions, false assumptions, and unrelated edits. Use a
+   differential review for changes in auth, crypto, config, parsers, and sandbox behaviour.
+5. Reconcile: fix each review finding or document why it does not apply.
+6. Verify: a re-scan of the checkout in the same mode, or a replay of the change with
+   `ousast pre-push --base <before> --head <after> --artifact <file> <repo>`, shows the finding gone
+   and no new findings. That re-scan is the evidence. Nothing in the tool records `patch_validated`;
+   do not claim it.
+
+Guardrails:
+
+- Do not rewrite modules when a local fix is enough.
+- Do not change public APIs unless the vulnerability requires it.
+- Do not call a finding fixed from model reasoning alone; the re-scan or replay decides.

.env.example

@@ -6,9 +6,8 @@
 DEEPSEEK_API_KEY=
 # DEEPSEEK_BASE_URL=https://api.deepseek.com
 
-# Embeddings only: OpenAI text-embedding-3-small through OpenRouter.
+# Embeddings only (`ousast learn memory embed`, default openai/text-embedding-3-small through OpenRouter).
 OPENROUTER_API_KEY=
-OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small
 
 # Plane memory store. Default is a local FileStore under ~/ousast-results/plane/memory.
 # For an S3-compatible server (RustFS is the tested one): OUSAST_MEMORY=s3://<bucket>[/<prefix>], or

.github/workflows/docs.yml

@@ -0,0 +1,43 @@
+name: Docs
+
+# Builds the MkDocs site and publishes it to GitHub Pages on every push to main that touches it.
+on:
+  push:
+    branches: [main]
+    paths: ["docs/**", "mkdocs.yml", "README.md", ".github/workflows/docs.yml"]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+      - uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+      - uses: actions/setup-python@v6
+        with:
+          python-version-file: .python-version
+      - run: uv sync --extra docs
+      - run: uv run mkdocs build --strict
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: site
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - id: deployment
+        uses: actions/deploy-pages@v4

.github/workflows/engine-image.yml

@@ -0,0 +1,69 @@
+name: Engine image
+on:
+  workflow_dispatch:
+  push:
+    branches: [main]
+    paths:
+      - 'src/**'
+      - 'benchmarks/learn/engine_trace*.py'
+      - 'benchmarks/learn/engine_task_entry.py'
+      - 'benchmarks/unseen/replay_entry.py'
+      - 'benchmarks/learn/engine_worker_service.py'
+      - 'Dockerfile'
+      - 'plane/Dockerfile.engine-task'
+      - 'ops/frontend-retention/install.py'
+      - 'pyproject.toml'
+      - '.github/workflows/engine-image.yml'
+permissions:
+  contents: read
+  packages: write
+concurrency:
+  group: engine-image-${{ github.ref }}
+  cancel-in-progress: false
+jobs:
+  image:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - package: openultrasast
+            dockerfile: Dockerfile
+          - package: ousast-engine-task
+            dockerfile: plane/Dockerfile.engine-task
+            target: engine
+          - package: ousast-replay-task
+            dockerfile: plane/Dockerfile.engine-task
+            target: replay
+    steps:
+      - uses: actions/checkout@v5
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - id: names
+        env:
+          OWNER: ${{ github.repository_owner }}
+          PACKAGE: ${{ matrix.package }}
+        run: |
+          echo "image=ghcr.io/${OWNER,,}/$PACKAGE" >> "$GITHUB_OUTPUT"
+          echo "sha=sha-${GITHUB_SHA:0:7}" >> "$GITHUB_OUTPUT"
+      - id: build
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: ${{ matrix.dockerfile }}
+          target: ${{ matrix.target }}
+          platforms: linux/amd64
+          push: true
+          tags: |
+            ${{ steps.names.outputs.image }}:${{ steps.names.outputs.sha }}
+            ${{ steps.names.outputs.image }}:main
+      - env:
+          IMAGE: ${{ steps.names.outputs.image }}
+          DIGEST: ${{ steps.build.outputs.digest }}
+        run: |
+          echo "$IMAGE@$DIGEST"
+          echo "$IMAGE@$DIGEST" >> "$GITHUB_STEP_SUMMARY"

.github/workflows/release.yml

@@ -34,3 +34,45 @@ jobs:
           generate_release_notes: true
           prerelease: ${{ contains(github.ref_name, '-') }}
           make_latest: ${{ !contains(github.ref_name, '-') }}
+
+  images:
+    # The scan image (Joern + php-cli, `docker run ... scan`) and the plane's task runner, pushed to GHCR under the
+    # tag's version. Pin them by the digests this job prints, not by tag: Agent Substrate rejects tags.
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - name: openultrasast
+            dockerfile: Dockerfile
+          - name: openultrasast-runner
+            dockerfile: plane/Dockerfile.runner
+    steps:
+      - uses: actions/checkout@v5
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository_owner }}/${{ matrix.name }}
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}},enable=${{ !contains(github.ref_name, '-') }}
+            type=raw,value=latest,enable=${{ !contains(github.ref_name, '-') }}
+      - id: build
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: ${{ matrix.dockerfile }}
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+      - run: |
+          echo "ghcr.io/${{ github.repository_owner }}/${{ matrix.name }}@${{ steps.build.outputs.digest }}" >> "$GITHUB_STEP_SUMMARY"

.gitignore

@@ -36,3 +36,6 @@ workspace/
 
 # MkDocs build output (mkdocs build)
 site/
+
+# Unseen evaluation pointers for non-permissive repositories.
+benchmarks/unseen/private/

.kiro/specs/ai-service-plane/spec.json

@@ -1,9 +1,9 @@
 {
   "feature_name": "ai-service-plane",
   "created_at": "2026-09-29T15:40:00Z",
-  "updated_at": "2026-09-29T17:55:00Z",
+  "updated_at": "2026-10-02T00:00:00Z",
   "language": "en",
-  "phase": "implementation",
+  "phase": "implementation-complete",
   "approvals": {
     "requirements": {
       "generated": true,

.kiro/specs/contributor-scan/requirements.md

@@ -1,5 +1,7 @@
 # Requirements Document: contributor-scan
 
+> **Status 2026-10-02:** paused since 2026-09-23 (its last change). 27 of 51 tasks done: the MAP integration was delivered and `v1.2.0-alpha.1` rolled out NO-GO (`census-repair-verification.md`); the remaining tasks wait on `pre-push-safety-net` M2/M3 and nothing here moved after the ax plane landed. Not active.
+
 ## Introduction
 
 `model-grounded-detection` built an arbiter and measured it: 41.3% of the corpus reaches `model_entailed`

.kiro/specs/finding-feedback-loop/requirements.md

@@ -1,5 +1,7 @@
 # Requirements Document: finding-feedback-loop
 
+> **Status 2026-10-02:** dormant. Requirements approved and design generated on 2026-09-09; the design was never approved and no tasks exist. It depends on `contributor-scan`, which is paused; reopen it with that spec.
+
 ## Introduction
 
 Nine engine defects were found in one session against two repositories. **Three were the same bug in three

.kiro/specs/learned-decision-engine/fix-mechanisms-plan.md

@@ -0,0 +1,80 @@
+# Plan: learn fix mechanisms into the engine (draft for review, 2026-10-04)
+
+Status: approved 2026-10-04 by the maintainer. Increment 1 = discharge facts only; coverage additions (Req 7) come later.
+Decisions: minimum 2 proposing repositories before a candidate is tried; admission per language (not pooled by callee name).
+
+## Why
+
+The Joern trace run over exp-005's 501 units shows the engine cannot tell a vulnerable function from its
+fix. In the 105 pairs where both sides were analysed, a taint path appears on both sides in 12, on the
+vulnerable side only in 2, on the fixed side only in 0, and on neither in 91
+(`benchmarks/measurements/2026-10-04-joern-slice-leak-audit/record.json`).
+
+So the fix is invisible to the engine. Either the flow survives and the engine does not know the fix's
+mechanism (an `allowed_classes` argument, a path normalisation check, an escaping call), or the engine's
+sources and sinks miss the function on both sides. A classifier fed this evidence cannot improve on it.
+
+The corpus now holds 710 complete fix pairs (`2026-10-04-decision-engine-harvest-3/record.json`). Each fix
+diff shows what was added. That is the material the engine lacks.
+
+## Goal
+
+Grow the engine's discharge knowledge (sanitizers, guards, safe-argument forms) and, second, its source and
+sink coverage, from the fix diffs. Admit a fact only when it improves held-out separation. Spend no model
+money.
+
+## Requirements (proposed)
+
+1. **Mining.** From each fix pair, extract what the fixed function adds on the vulnerable flow: new calls,
+   new conditions guarding the sink, new or changed sink arguments. Record each as a candidate fact with its
+   language, family, form (sanitizer call, guard predicate, safe argument, sink restriction) and the pairs
+   that propose it. Mining is deterministic (tree-sitter diff of the two function bodies). No model calls.
+2. **Generalisation is explicit.** A candidate is a pattern, not a repository name: a callee name, an
+   argument key, a predicate shape. Candidates naming project-specific identifiers are rejected (Req 8: no
+   application-specific trees).
+3. **Admission by held-out separation.** Folds are by repository, as in the decision engine. A candidate is
+   admitted for a fold only if, on that fold's held-out pairs, adding it to the fact tables:
+   - increases pairs where the vulnerable side keeps a path and the fixed side loses it or becomes
+     sanitized;
+   - and does not remove any held-out vulnerable-side path (no recall loss).
+   A candidate proposed by fewer than 2 distinct repositories in the training folds is not tried.
+4. **No train-on-test.** Mining and admission never read held-out pairs; the closed-loop leak recorded in
+   memory (`closed-loop-train-on-test-leak`) is the failure this rule exists to prevent. Population v3 is
+   never read.
+5. **Measurement by the trace runner.** Each admission round re-runs the engine on the affected pins only
+   (the runner's resume and selection already support this; on k3s the worker Deployment runs them in
+   parallel). The outcome is the pair-asymmetry table per family, before and after, on held-out folds.
+6. **Gate before production.** An admitted fact enters `src/openultrasast/ruleset/semantic/*.toml` only after
+   the detection gates (`gate`, `map_gate`, `pair_gate`) show no regression, and only as a registered
+   experiment whose decision rule was fixed before the run.
+7. **Coverage second.** After discharge facts, the same loop proposes source and sink additions from pairs
+   where neither side has a path (91 of 105 today), admitted under the same held-out rule.
+
+## How it relates to what exists
+
+- `corpus-seeded-mechanisms` already has a closed loop that admits and retracts mechanisms by leave-one-out.
+  This plan reuses its lever and store, and points it at discharge facts mined from real fix diffs instead
+  of variant search.
+- `benchmarks/pairs/mechanisms.toml` (23 mechanisms) labels what kind of bug a pair is. The new candidates
+  describe what a fix does. They live in the fact tables, not in the mechanism vocabulary.
+
+## Measures of success
+
+- Pair asymmetry on held-out folds: vulnerable-only paths rise from 2 of 105 toward a substantial share,
+  with zero fixed-only paths.
+- No held-out recall loss on vulnerable sides.
+- Then the decision engine is re-run with the new engine evidence (exp-005 design), and the pooled block gate
+  (exp-004 design) is re-measured.
+
+## Cost and time
+
+- Model cost: $0 (mining and admission are deterministic; the classifier re-run afterwards is a separate,
+  approved spend).
+- Engine time: one pass over affected pins per admission round; about 2 hours per full pass on k3s plus the
+  VM, about 6 hours on the VM alone.
+
+## Open questions for the maintainer
+
+1. Should coverage additions (Req 7) be in the same increment or a later one?
+2. Minimum proposing repositories: 2 is the proposal. Higher is safer, lower finds more.
+3. Admission per family and language, or pooled across languages for the same callee name?

.kiro/specs/learned-decision-engine/requirements.md

@@ -119,6 +119,19 @@ rule hits.
 2. Without a trained model available the tools fall back to today's behaviour and say so in the report.
 3. Hand-written rules remain as signal producers; changing their enabled/shadow status no longer changes what is
    reported unless the engine's decision changes.
+4. *(Amended 2026-10-03, maintainer decision; refines the BLOCK rule of 7.3 and 7.4. Tested by exp-004.)* BLOCK
+   precision is demonstrated pooled across families, not per family:
+   - **Normalisation**, declared before any run: each family's raw score is put on one scale by a per-family Platt
+     map (`learn.calibrate.fit_map` with its floors) fitted on training folds only. A family the training folds
+     cannot map does not block on that fold.
+   - **t_block**: one threshold on the normalised score for all families, chosen on training folds as the lowest
+     one whose Wilson 95% lower bound on precision, over the training flags of all families together, is >= 0.95.
+     It is reported by the Wilson 95% lower bound over all held-out flags of all families, before the floor.
+   - **Family floor**: a family may block only if its own held-out flags have zero errors and number at least F
+     (F = 10, reasoned in the exp-004 registration); otherwise that family stays ADVISORY.
+   - **t_quiet**: per family, the highest threshold keeping >= 90% of the paired positives at or above it (chosen
+     on training folds, its held-out recall reported); findings below it are silent.
+   - No held-out label moves a threshold or a normalisation map; candidates whose verdict is `unsure` never block.
 
 ### Requirement 7: Built for unseen repositories -- a repo safety net
 
@@ -136,7 +149,8 @@ rule hits.
    ADVISORY, trading precision for recall; both reported with out-of-repository intervals. The decision unit is a
    candidate in changed code (`pre-push` delta), with whole-repository scans as a secondary mode.
 4. Calibration is checked on held-out repositories; where it does not hold, only ADVISORY is offered for that
-   family.
+   family. *(Amended 2026-10-03: once exp-004 adopts the pooled gate of 6.4, BLOCK precision is shown by its held-out
+   bound and family floor, and calibration is reported, not required.)*
 5. Any adaptation to a user's own repository (facts reuse, their dismissals) is opt-in, separate from the
    generalisation model, and never contributes to its reported numbers.
 

.kiro/specs/learned-decision-engine/spec.json

@@ -1,7 +1,7 @@
 {
   "feature_name": "learned-decision-engine",
   "created_at": "2026-09-30T14:00:00Z",
-  "updated_at": "2026-10-01T10:20:00Z",
+  "updated_at": "2026-10-04T18:00:00Z",
   "language": "en",
   "phase": "implementation",
   "approvals": {

.kiro/specs/learned-decision-engine/tasks.md

@@ -8,6 +8,10 @@ memory, compiled DSPy-style in-house); tasks 1-5 are unaffected, tasks 6-10 are
 against the provider balance read just before it, and none starts with a ceiling above that balance. No test makes
 a model or embedding call.
 
+Direction 2026-10-04: `.kiro/steering/roadmap.md` (section of 2026-10-04, phases G0-G5) is the authority. Every
+change is judged on the unseen-repository pool (G0, spec `unseen-repo-evaluation`) by catch rate and false-alarm
+rate per push. Within-pair AUC stays a development signal. Tasks 11-14 below follow that roadmap.
+
 - [x] 1. Schema and features
   - `learn/schema.py` (closed, typed allow-list; no identities; missing != zero), `learn/features.py` (host
     builder), new memory `KINDS`, `plane/tasks/features.py` and the `remember` input; tests enforcing the
@@ -45,6 +49,11 @@ a model or embedding call.
   - _Requirements: 1.1_
 
 - [ ] 6. Program, local memory and compilation (design sections 4-5)
+  - Status 2026-10-04: 6.1-6.6 are done. 6.7 is partial: leave-one-source-out, the 0/25/50% memory-size points and
+    the source-count curve are not run. Around this task the experiment machinery grew: rule arms
+    (`precision_bound_block`, `pooled_block_gate` with a family floor) in `learn/experiments.py`, the response-cache
+    fill with a hard ceiling and a dry-run estimate in `learn/fill.py`, and the in-function slice in `learn/slice.py`.
+    These ran exp-003, exp-004 and exp-005 (7.5-7.7). Memory now holds 710 complete fix pairs (7.9).
   - [x] 6.1 Excerpts and the example store (no model)
     - `learn/excerpt.py` (function span at the label's pin, bounds 80/4,000 for candidates and 40/2,000 for
       examples, numbered lines, no path/repository/commit, advisory ids and security wording in comments
@@ -116,12 +125,63 @@ a model or embedding call.
     - _Requirements: 5.1, 5.2, 8.3_
 
 - [ ] 7. Experiments (exp-001 and program variants need budget go-ahead)
-  - `learn/experiments.py`, `register`, `experiment-outcome`, Run annotation checks, `OUSAST_VERIFY_CALLERS`;
+  - [x] 7.1 Machinery (no model in tests): `learn/experiments.py` -- YAML manifests under `plane/experiments/<id>.yaml`
-    exp-001 (known-callers line on/off, ~$7.7 expected, ceiling $15) registered, run and recorded -- an
+    (hypothesis, arms as program-spec overrides or plane `task_env`, unit, pairing, repository cluster, metrics with
-    inconclusive result is pre-registered as likely.
+    `[label=]` selectors and targets, MDE, two looks with O'Brien-Fleming boundaries, budget ceiling, families, folds,
-  - Program-variant experiments (paired on candidates, repository cluster, arm A replayed from cache at $0): the
+    frozen units file), `register` (`experiment` row with both digests; refuses an uncommitted, modified or re-declared
-    machinery and its test; the first variant (`k = 5` vs `k = 1`, or the confidence source) **(budget go-ahead:
+    manifest), `units` (freeze the families' evaluation candidates, paired by candidate across pins; no model), `run`
-    ~$0.9-1.35 per arm-B pass of 300 candidates, ceiling $2 each)**.
+    (paired arms in a seeded repository order, a seeded coin per unit for which arm goes first, `arm_outcome` rows,
+    resumable, arm A replay-only at $0, canary replicates under a fresh cache salt, the first look during the run),
+    `analyse` (paired repository bootstrap, exact McNemar, the looks, adopt / reject / equivalent / inconclusive,
+    `experiment_result` row + `benchmarks/experiments/<id>/result.json`), `ousast learn experiment
+    register|units|run|analyse`; `OUSAST_VERIFY_CALLERS=off` in `plane/tasks/verify.py`; the retrieval-ensemble
+    sampling variant (`ProgramSpec.sampling = retrieval_ensemble`: `k` samples at temperature 0 over `k` disjoint
+    retrieved example sets; default unchanged). Tests: `test_learn_experiments.py` (18, scripted clients, FileStore).
+  - [x] 7.2 Registered, not run: `exp-001-known-callers` (plane arms, `flag_rate[label=1]`, MDE 0.15, ceiling $15;
+    units `pending` -- model-free candidate generation on the 41 development repositories is not built, and plane Run
+    annotation checks / the `experiment-outcome` task are not written) and `exp-002-retrieval-ensemble` (arm A today's
+    k = 5 temperature sampling, arm B the retrieval ensemble; primary canary agreement (target 0.90) and within-pair
+    AUC, secondary usd per candidate, AUC, ADVISORY recall/precision; families injection and access_control on the
+    slice-2 candidates; ceiling $4: A $1, B $3). Both registered in `s3://sast-memory` at fbc3220
+    (`benchmarks/experiments/<id>/registration.json`). exp-002's units are `pending` too: on 2026-10-02 every example
+    row in the store carried null plane-only instruments (`agree`, `model_sinks`, `verify`) and `load_examples` raised
+    on them, so `ousast learn experiment units` could not freeze the candidates.
+  - [ ] 7.3 exp-001 run and recorded (~$7.7 expected, ceiling $15) -- an inconclusive result is pre-registered as
+    likely; needs the frozen units file first.
+  - [x] 7.4 exp-002 run and recorded 2026-10-02 (budget go-ahead ~$2 expected, ceiling $4): **reject B**. 160 paired
+    units (injection 119, access_control 41; 56 repositories; 108 units in 54 candidate pairs), both arms, two
+    replicates each, no early stop. Primary: canary agreement A 0.875 -> B 0.906 (diff +0.03, CI [-0.03, +0.10],
+    spans; B meets the 0.90 target), within-pair AUC A 0.815 -> B 0.741 (diff -0.07, CI [-0.13, -0.02], **against B**).
+    Secondary: AUC -0.02 (CI [-0.04, -0.003], against), ADVISORY recall -0.06 (spans; McNemar 7/2, p = 0.18),
+    precision +0.01 (spans), usd per candidate A $0.0021 -> B $0.0051 (CI [+0.0027, +0.0032], against: the five
+    disjoint example sets share only the prompt prefix in DeepSeek's cache). Priced usage A $0.68 (1206 paid calls,
+    395 replayed), B $1.63 (1350 paid, 250 replayed). Result: `benchmarks/experiments/exp-002-retrieval-ensemble/result.json`,
+    `benchmarks/measurements/2026-10-02-exp-002/record.json`. Three repairs on the way (each its own commit): example
+    rows' null plane-only features (1f525ff), the 788 embedding vectors absent from RustFS (copied from the file
+    store), the result row's column-type collision under S3 Select (780caaa). Further program variants (`k = 5` vs
+    `k = 1`, the confidence source) as new manifests.
+  - [x] 7.5 exp-003 run and recorded 2026-10-03 ($0, replay only): **inconclusive**. Selective blocking (a per-family
+    raw-score threshold chosen on training folds by its Wilson lower bound) offers no BLOCK on held-out paired units
+    in any family; the incumbent stays. 211 of 353 frozen units evaluated (142 replay misses). Proving 0.95 needs at
+    least 73 clean flags. Result: `benchmarks/experiments/exp-003-precision-bound-blocking/result.json`.
+  - [x] 7.6 exp-004 run and recorded 2026-10-03: **inconclusive**. One pooled BLOCK threshold over all families with a
+    family floor (Req 6.4). No outer fold found a threshold, pooled or per family; B offers no BLOCK. 501 units,
+    replay at $0 after a cache fill of $1.24 (ceiling $2; the dry-run estimate over-priced 4.5x). Result and record:
+    `benchmarks/experiments/exp-004-pooled-block-gate/result.json`, `record.json`.
+  - [x] 7.7 exp-005 run and recorded 2026-10-03: **inconclusive**. The in-function def-use slice in the prompt of the
+    six slice-2 programs. Within-pair AUC 0.762 -> 0.746 (CI [-0.063, 0.032], spans 0); unsure rate 0.29 -> 0.24
+    (favours B); usd per candidate slightly higher. A $0 leak audit came first
+    (`benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json`). Result:
+    `benchmarks/experiments/exp-005-graph-slice/result.json`.
+  - [x] 7.8 Negatives harvest 2026-10-03 ($0.0014 embeddings, no chat-model call): 98 real fix pairs whose function
+    survives the fix, from reviewed advisories, as pointer-pair corpus `advisory-fixes-2`; memory 937 -> 1133
+    examples. Record: `benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest/record.json`.
+  - [x] 7.9 Harvest 3, 2026-10-04 ($0.003 embeddings, no chat-model call): 251 pairs as `advisory-fixes-3`; complete
+    pairs in memory 459 -> 710. Record: `benchmarks/measurements/2026-10-04-decision-engine-harvest-3/record.json`.
+  - [x] 7.10 Joern trace run over exp-005's 501 units and its leak audit, 2026-10-04 ($0): no leak flagged. In the 105
+    pairs where both sides were asked, a path appears on both sides in 12, on the vulnerable side only in 2, on the
+    fixed side only in 0, and on neither in 91. The engine cannot see what a fix changed. Record:
+    `benchmarks/measurements/2026-10-04-joern-slice-leak-audit/record.json`.
   - _Requirements: 3.6, 4.1-4.4_
 
 - [ ] 8. Integration
@@ -147,3 +207,50 @@ a model or embedding call.
     `result-v3-engine.json` is committed. Nothing between the two commits may touch the program, memory snapshot,
     features, priors or thresholds. This spec's code never opens v3 files.
   - _Requirements: 5.3_
+
+- [ ] 11. Fix mechanisms (plan approved 2026-10-04; see `fix-mechanisms-plan.md`)
+  - Roadmap G2 arm (`.kiro/steering/roadmap.md`, section of 2026-10-04): one candidate improvement, not the spine.
+    It is judged on the unseen-repository pool (G0, spec `unseen-repo-evaluation`) by catch rate and false-alarm
+    rate per push. **Paused after 11.1 until the G0 pool exists.**
+  - [ ] 11.1 Mine candidate discharge facts from the 710 fix diffs (sanitizer calls, guarding conditions, safe sink
+    arguments, safe replacements); deterministic, $0. In progress.
+  - [ ] 11.2 Admission by held-out separation per outer fold: a candidate turns "path on both sides" into "path on the
+    vulnerable side only" on folds it was not mined from, removes no held-out vulnerable path, has at least 2
+    proposing repositories, and is admitted per language.
+  - [ ] 11.3 Engine re-run on the affected pins (kube-ax lanes plus the VM lane).
+  - [ ] 11.4 Registered experiment, with the detection gates green, before `ruleset/semantic/*.toml` changes.
+  - [ ] 11.5 Coverage increment: source and sink additions for pairs where neither side has a path (later).
+  - [ ] 11.6 TypeScript admission in the engine (branch `engine-typescript`).
+  - _Requirements: 4.1-4.4, 7.1, 8.1_
+
+- [ ] 12. exp-006: Joern witness slice in the prompt **(budget go-ahead, about $2-4)**
+  - Planned, roadmap G1/G2. Against the current prompt, after a $0 leak audit, on re-frozen units. Judged on the
+    unseen-repository pool, not on within-pair AUC alone.
+  - _Requirements: 4.1-4.4, 7.1_
+
+- [ ] 13. exp-007: the combiner ($0 replay)
+  - Planned, roadmap G1/G2. Three or four weights (score, engine entailment, neighbour agreement) over cached
+    responses. Judged on the unseen-repository pool.
+  - _Requirements: 4.1-4.4, 7.1_
+
+- [ ] 14. Re-run the pooled block gate on re-frozen units
+  - Planned, roadmap G3. The exp-004 design with the family floor (Req 6.4), together with the pool's false-alarm
+    rate per family.
+  - _Requirements: 6.4, 7.3_
+
+- [ ] 15. Trained classifier arm (maintainer decision 2026-10-04: train a model, not hand-set profiles; judged on the
+  unseen-repo pool, roadmap G1)
+  - [ ] 15.1 Features: code embedding of the excerpt (stored vectors) + every allow-listed signal; no hand-picked
+    profile. Automated screening: a signal that separates pair sides by itself on training folds above the leak
+    threshold is dropped for that fold (the leak audit as a filter, fitted per fold).
+  - [ ] 15.2 Models: regularised logistic regression and gradient-boosted trees, fitted per repository-held-out outer
+    fold with inner-fold hyper-parameter search; per-family and pooled variants.
+  - [ ] 15.3 Scores on held-out folds: within-pair AUC, pooled AUC, top-k precision, Wilson bound at the pooled
+    threshold; compared with the current LLM programs on the same units.
+  - [ ] 15.4 Scored on the unseen-repo pool when G0 exists: catch rate at fixed false-alarm budgets.
+  - _Requirements: 3 (revised 2026-10-04 by maintainer decision), 6, 7_
+
+- [ ] 16. Jointly optimised LLM program arm (judged on the unseen-repo pool)
+  - Search input fields, retrieval parameters, k, instruction and demonstrations together (DSPy MIPRO-style) on
+    training folds; no hand-set input profile. Budget go-ahead before paid search.
+  - _Requirements: 3, 6, 7_

.kiro/specs/plane-on-kubernetes/brief.md

@@ -0,0 +1,33 @@
+# Brief: the plane on a Kubernetes cluster
+
+Maintainer, 2026-10-02: "the plan is to move the whole to a kube system soon. what is in the way?" and earlier
+"later i will deploy this in a separate kube env."
+
+## Where we are
+
+The plane already runs on google/ax over Agent Substrate, on a single-node kind cluster on the maintainer's
+laptop (`ops/ax/`). Its manifests are ax's own kinds, every task is one digest-pinned runner image, the memory
+store is external (S3-compatible RustFS), and provider credentials travel only in the start request. What still
+assumes the laptop is small and listed in `docs/deployment.md` and `ops/ax/README.md`:
+
+1. the reconciler is a host CLI process (`ousast plane run`);
+2. the artifact receiver is a thread of that process, reached through a Service + EndpointSlice to the kind
+   gateway; the router is reached through a kubectl port-forward;
+3. the kube context `kind-ousast` and the registry `localhost:5001` are hard-coded in `doctor.py`, `egress.py`,
+   `router.py`;
+4. images come from the kind registry;
+5. ax's deploy manifest points `AX_SNAPSHOTS_BUCKET` at the ax authors' bucket;
+6. the Joern engine runs as a host Docker container (2.8 GB image, 3 GB memory), not as an ax task;
+7. the worker pool is two 1 CPU / 1.5 GiB workers with no autoscaling.
+
+## Direction
+
+Make the plane deployable to any Kubernetes cluster that runs Agent Substrate and ax, prove every change on the
+kind cluster first (it already behaves like the target), and leave nothing that only works on this host. The
+production use (a pre-push safety net fed by CI) needs, in addition, a delta Run command and Workspaces from a
+URL and commit; those are in scope as the last increment because they define what the cluster is for.
+
+## Not in scope
+
+Provisioning the maintainer's cluster, Substrate's and ax's own deployment procedures (followed, not changed),
+and any detection change.

.kiro/specs/plane-on-kubernetes/design.md

@@ -0,0 +1,426 @@
+# Design Document
+
+> **Status 2026-10-04.** Groups 1-2 are done on kind (records `benchmarks/measurements/2026-10-02-k8s-01-profile/`
+> and `2026-10-02-k8s-02-store-delivery/`). Task 2.6 is paused. The production target changed. The operator runs a
+> kube-ax cluster (k3s, Agent Substrate v0.3.0, AX v0.3.1, 2 nodes). There we may only submit AX Tasks: no k8s
+> object creation, no secrets, no access to `ate-system`. The engine now runs there as AX Tasks through
+> `benchmarks/learn/engine_trace_ax.py`. The engine-task image (`plane/Dockerfile.engine-task`) is built FROM
+> `ghcr.io/norandom/ax-task-runner:v0.3.1`. Tasks read and write through presigned HTTPS links, are named
+> `ousast-engine-*`, and run a 1.7 GB heap with `ActiveProcessorCount=2` and one resident Joern session per pin. A
+> pin whose sandbox dies is retried once under a new name, then handed to the VM lane (2.5 GB heap). Egress is
+> operator-managed: HTTPS 443 to public hosts is open; LAN and cluster-internal addresses are blocked. Archive
+> names (`.tar`, `.gz`, `.zip`, `.bin`) failed through the store host's Cloudflare cache until the operator added a
+> cache-bypass rule; presigned objects are staged as `.dat` as a safeguard. Groups 3-8 are **to re-plan** against
+> that cluster under roadmap G4 (`.kiro/steering/roadmap.md`, section of 2026-10-04). Their text below is kept
+> as written. G4 (detection on push for any repository: `ousast plane scan --base --head`, engine pins on kube-ax,
+> model calls on the VM or CI) is the new target for group 7, sized by what G1-G3 keep.
+
+## Overview
+
+The plane keeps its shape: `ousast plane run` is the reconciler, ax is the only executor, every task is one
+digest-pinned image under the runner contract, memory is the S3 store. What changes is where the reconciler's
+peers live and how they are named. Three facts from the code and the upstream checkouts decide the design:
+
+- **Every host assumption is an address or a process in four places.** `kind-` + `KIND_CLUSTER_NAME` is built in
+  `doctor.py:39`, `egress.py:115`/`:148` and `router.py:145`; `localhost:5001` in `doctor.py:45-47` and
+  `up.sh:10`; the receiver is a thread of the CLI (`reconciler.py:350-352`) reached by an EndpointSlice to the kind
+  gateway (`receiver-service.yaml.tmpl`, `up.sh:55-57`); the router and ax-server are reached by `kubectl
+  port-forward` (`router.py:146-148`; ax's own CLI does the same, `ax/internal/tunnel/tunnel.go:180-238`).
+- **The store is already the one thing every profile shares.** `S3Store` is verified at every open
+  (`memory.py:608-640`), its `runs/` prefix expires after 30 days (`RUNS_RULE_ID`, `memory.py:589-591`), and no
+  task reaches it today. A presigned URL is a credential, and the plane already has a channel for credentials that
+  never touches a manifest or the golden boot: the start request (`router.py`, `start_task`, `runner.py:219-275`).
+- **ax-server authenticates nobody; Kubernetes does.** `deploy/ax-server.yaml` runs `--addr=:8080` with no auth
+  flag; the `ax` CLI tunnels to `svc/ax-server` through kubectl; `kubectl-ate` port-forwards to `ate-api-server`
+  and sends a Kubernetes ServiceAccount token (`cmd/kubectl-ate/internal/cmd/root.go:59-60`). So the only
+  authenticated door into either control plane is the Kubernetes API with a kubeconfig.
+
+Hence (maintainer, 2026-10-02): the reconciler stays the CLI and gains an execution mode. `local` is the kind
+profile as today; `remote` is the same `ousast plane run` (later `ousast plane scan`) from a laptop or a CI job
+against ax in k3s, with task artifacts delivered to the S3 store through presigned URLs minted by the CLI and
+carried in the start request, and the CLI polling the store for completion. No in-cluster reconciler is needed
+for the move; the receiver thread goes in both profiles once the store path passes its kind proof.
+
+### Goals
+
+- Two profiles of one code base, `kind` and `k3s`; every change proven on kind first (Req 2.4, 7.1).
+- No module names a host (`kind-ousast`, `localhost:5001`, `172.19.0.1`); a test enforces it (Req 1.3).
+- One delivery path (the store) in both profiles; one credential channel (the start request).
+- The Joern engine as a plane task on its own pool (Req 4); a delta Run for a push (Req 6).
+
+### Non-Goals
+
+Provisioning k3s; changing Substrate or ax (their manifests are applied with our values, not edited in place);
+any detection, scoring or population change; an unattended in-cluster reconciler (deferred, section 2.4).
+
+## Architecture
+
+```mermaid
+flowchart LR
+    subgraph laptop ["kind profile: laptop"]
+        cli1["ousast plane run --exec local"]
+        env1[".env: provider + S3 keys"]
+    end
+    subgraph kind ["kind cluster (same host)"]
+        ax1["ax-server (ax-system)"]
+        sub1["Substrate: atenet-router, atenet-egress, ate-api"]
+        pool1["ousast-pool: 2 gVisor workers"]
+        epool1["engine-pool: 1 worker, 4 GiB"]
+        reg1["kind registry: images by digest"]
+    end
+    s3[("S3 store (RustFS, external): runs/, repos/, facts/")]
+    cli1 -- "ax CLI tunnel; kubectl-ate; router port-forward" --> ax1
+    cli1 --> sub1
+    sub1 --> pool1
+    sub1 --> epool1
+    pool1 -- "presigned PUT/GET via egress (TLS 443)" --> s3
+    epool1 --> s3
+    cli1 -- "mint URLs, poll, ingest" --> s3
+    reg1 -.-> pool1
+```
+
+```mermaid
+flowchart LR
+    subgraph ci ["k3s profile: laptop or CI job"]
+        cli2["ousast plane run|scan --exec remote"]
+        kc["kubeconfig: SA ousast-ci token"]
+        sec["CI secrets: provider + S3 keys"]
+    end
+    subgraph k3s ["k3s cluster"]
+        api["Kubernetes API (RBAC: portforward in ax-system, ate-system)"]
+        ax2["ax-server"]
+        sub2["Substrate router / egress / ate-api"]
+        pool2["ousast-pool + HPA"]
+        epool2["engine-pool + HPA"]
+    end
+    ghcr["GHCR ghcr.io/norandom: runner + engine images by digest"]
+    s3b[("S3 store (RustFS)")]
+    cli2 -- "port-forward tunnels only" --> api
+    api --> ax2
+    api --> sub2
+    sub2 --> pool2
+    sub2 --> epool2
+    pool2 -- "presigned PUT/GET" --> s3b
+    epool2 --> s3b
+    cli2 --> s3b
+    ghcr -. "imagePullSecret if private" .-> pool2
+```
+
+The two diagrams differ in the kubeconfig principal, the registry, the pool sizes and the HPA. Nothing else.
+
+New code: `plane/profile.py` (the profile), `plane/delivery.py` (presign, poll, extract), `plane/k8s.py`
+(manifest rendering per profile), `plane/tasks/engine.py`, `plane/scan.py` (Req 6). `reconciler.py` loses the
+receiver wiring and gains a `Delivery` dependency; it stays under the 500 lines `test_plane_reconciler.py:803`
+enforces because every new concern is a sibling module, as `router.py` and `egress.py` already are.
+
+## Components and Interfaces
+
+### 1. Configuration surface: `PlaneProfile` (Req 1)
+
+**Decision.** One frozen dataclass `PlaneProfile` in `plane/profile.py`, loaded by `load_profile()` from a TOML
+file named by `OUSAST_PLANE_PROFILE` (a name resolved under `ops/k8s/profiles/<name>.toml`, or a path; default
+`kind`), with every field overridable by an environment variable of the same upper-cased name
+(`OUSAST_KUBE_CONTEXT`, `OUSAST_REGISTRY`, ...). Profiles hold no secret: keys stay in the environment or
+`.env`, as today.
+
+| Field | `kind.toml` | `k3s.toml` | Replaces |
+| --- | --- | --- | --- |
+| `exec` | `local` | `remote` | the implicit host mode |
+| `kube_context` | `kind-ousast` | `ousast-k3s` (the maintainer's context name) | `"kind-" + KIND_CLUSTER_NAME` in three modules |
+| `registry` | `localhost:5001` | `ghcr.io/norandom` | `doctor.py:45`, `up.sh:10` |
+| `images` | `kind-images.json` | `images.json` (from the release, section 8) | the `image:` literals in `plane/tasks/*.yaml` |
+| `router_url` | `""` (port-forward) | `""` (port-forward) | `OUSAST_ROUTER_URL` stays as the test override |
+| `ax_server` | `""` (ax CLI tunnel) | `""` | `$AX_SERVER` stays as ax's own override |
+| `memory` | `s3://sast-memory` | `s3://sast-memory` | `OUSAST_MEMORY` default `file://` is removed for the plane (no fallback) |
+| `atespace` | `default` | `default` | `generate.ATESPACE` |
+| `snapshots_bucket` | `s3://ax-snapshots` | `s3://ax-snapshots` | ax's `gs://dberkov-gke-dev3/ate-env/` (rendered into our ax-server patch) |
+| `pools.default`, `pools.engine` | `{replicas: 2, cpu: 1, memory: 1536Mi}`, `{1, 2, 4Gi}` | documented production values | `workerpool.yaml.tmpl` |
+| `image_pull_secret` | `""` | `ghcr-pull` | none today |
+
+**Why a file plus env, not env only.** Eleven values change together per cluster; env-only has no reviewable
+record of what "k3s" is (the `ops/ax/README.md` table at lines 193-204 already struggles to list them), and a
+CI job would carry eleven exports. The env override keeps the existing tests (`OUSAST_ROUTER_URL`,
+`OUSAST_ARTIFACT_HOST`) and one-off experiments working. **Rejected:** a `[plane]` section in the scan config
+TOML (the scan config is a user's tool configuration; the plane is a maintainer surface and must not leak into
+`ousast scan`); a Kubernetes ConfigMap (there is no in-cluster reconciler to read it).
+
+`doctor()` takes the profile: it checks `kube_context` reachability, the two namespaces, that the `images` file's
+runner reference resolves (`docker manifest inspect` for GHCR, the `/v2/` catalog for a local registry), that
+`memory` opens (`open_store` verifies the bucket), and prints each configured address (Req 1.2). The literal scan
+(section 9) covers `src/`, `ops/`, `plane/` and `docs/` except the profile files themselves and
+`ops/ax/up.sh`, which may name kind because it creates it.
+
+### 2. Execution mode: the CLI with `--exec local|remote` (Req 2)
+
+**Decision.** `ousast plane run RUN.yaml [--exec local|remote]` (default from the profile). Both modes run the
+same `reconciler.run`: render, `ax apply`, egress policy, resume, start, await, delete. What differs is
+addressing and principal, and none of it is new client code:
+
+| Call | Through | Principal in `remote` |
+| --- | --- | --- |
+| `ax apply/resume/get/delete` (`reconciler.Ax`) | ax's own tunnel: `kubectl port-forward -n ax-system svc/ax-server` (`tunnel.go:203-238`), context from the profile | kubeconfig for SA `ousast-ci` |
+| `kubectl ate create/get/update egress-policy` (`egress.Egress`) | kubectl-ate's own port-forward to `ate-api-server`, bearer = the SA token (`root.go:59-60`) | same SA; Substrate's authorization of that token for `egress-policy` is an open question (below) |
+| `POST /ousast/v1/start` (`router.open_router`) | `kubectl port-forward svc/atenet-router` in `ate-system`, as today | same SA |
+| artifacts, inputs, state | the S3 store (section 3) | the store's agent key from the CI secrets |
+
+**Auth: kubeconfig with a ServiceAccount token, not an ingress.** ax-server has no authentication
+(`deploy/ax-server.yaml:71-79`: `--addr=:8080`, nothing else) and Substrate's router routes on a header. Exposing
+either through Traefik would publish an unauthenticated control plane behind whatever middleware we bolt on;
+tunnelling through the Kubernetes API keeps one authenticated, audited door and uses the clients as they ship.
+**Rejected:** Traefik IngressRoute for ax-server and the router (unauthenticated upstreams, a second auth layer to
+design, and ax's CLI would still tunnel); a bespoke ax HTTP client (Req boundary: no replacement of the `ax` CLI
+beyond what addressing needs, and the tunnel already handles addressing).
+
+**RBAC (`ops/k8s/base/rbac-ci.yaml`).** ServiceAccount `ousast-ci` in `ax-system`; Role in `ax-system` and in
+`ate-system`: `pods` get/list, `services` get, `pods/portforward` create; nothing cluster-wide, no verbs on
+Substrate's or ax's CRDs (both are reached through their servers, not through the Kubernetes API). The token is a
+bound TokenRequest (`kubectl create token ousast-ci --duration=...`) stored as the CI secret `KUBECONFIG`. Secrets
+for the provider and the store stay where the reconciler runs: `.env` on a laptop, repository secrets in CI,
+Kubernetes Secrets only on the day an in-cluster reconciler exists (2.4). The start-request path is unchanged
+(Req 2.3).
+
+**Run state of record moves to the store.** `state.json` is written locally under `$OUSAST_RESULTS/plane/<run>/`
+as today and mirrored to `runs/<run>/state.json` on the store after each change; a rerun from another machine
+seeds its local copy from the store. This is what lets CI job N+1 resume CI job N's Run (Runs are long,
+resumable and artifact-driven: `reconciler.py:272-312`).
+
+**Kind keeps `local` (Req 2.4).** Same command, same manifests, local registry, laptop as reconciler; the only
+differences are the profile's values. The kind proof of each increment is run in `local`, and increment 3 runs
+`remote` against kind from a container that holds only the kubeconfig and the secrets (section 7).
+
+#### 2.4 Deferred: an in-cluster reconciler
+
+Not needed for the move. It would add: survival of a laptop sleeping or a CI job timing out mid-Run, scheduled
+Runs, and Secrets injected by Kubernetes. It would cost: a workload (Job per Run is the natural shape, keeping
+the CLI path identical), a PVC or store-only state, a submit path, and RBAC for ax's and Substrate's Services from
+inside the cluster. Revisit when unattended Runs are wanted; the store-backed state above is its prerequisite.
+
+### 3. Delivery: presigned URLs in the start request, both profiles (Req 3)
+
+**Decision.** The runner PUTs `output.tar` to a presigned S3 URL and GETs each declared input from a presigned
+URL; the CLI mints them and polls the store. The URLs travel in the start body, never in env:
+
+```json
+{"run": "...", "task": "...", "credentials": {"DEEPSEEK_API_KEY": "..."},
+ "delivery": {"put": "<presigned PUT runs/<run>/<task>/output.tar>",
+              "inputs": {"CANDIDATES": "<presigned GET runs/<run>/<producer>/<artifact>>"}}}
+```
+
+- **Why the start body.** A presigned URL is a bearer capability, so the credentials-only-in-the-start-request
+  rule applies; env would bake it into the ActorTemplate and hand it to the golden boot; and the body has no
+  32 KB ceiling (Substrate rejects env values over 32768 characters, `reconciler.py:141`). `StartGate.offer`
+  (`runner.py:250-275`) validates `delivery` the way it validates `credentials`: `https` URLs only, hostnames,
+  no echo. Expiry = `OUSAST_TASK_TIMEOUT` (7200 s) + 600 s; a re-start after a re-resume re-mints.
+- **Why presigned rather than a scoped delivery credential.** A per-task S3 key needs a per-task policy on RustFS,
+  gives the sandbox `ListBucket` and reads beyond its inputs, and would travel in the same body. Presigned URLs
+  name exactly the objects a task may write and read, which is what the receiver enforces today
+  (`Receiver.input_file`, `egress.py:230-247`).
+- **Why not an in-cluster receiver Service.** It is a server to run, holds state in a process, and is reachable
+  only from a pod; that is the gap remote mode has. **Rejected.**
+- **Objects.** `runs/<run>/<task>/output.tar` (the runner's PUT); after the CLI extracts it, the declared
+  `outputs[]` are re-put as `runs/<run>/<task>/<artifact>` so a consumer's presigned GET names one artifact, and
+  `runs/<run>/<task>/done.json` carries the summary's status. The 30-day `runs/` expiry already applies; memory
+  rows are ingested from the local extraction by `remember` as today (f: annotations and `memory_key` seeding
+  unchanged, `generate.py:_chain_inputs`, `memory.seed`).
+- **Completion.** `_await`'s `delivered()` becomes `Delivery.delivered(task)`: a HEAD on `output.tar`, polled at
+  `OUSAST_POLL_SECONDS` (3 s). The runner's completion marker logic (`runner.py`, `_run_once`) is unchanged.
+- **Inputs no task produces** (`candidates.json`, `functions.json`, `case.json`) may also be store objects
+  declared as inputs, which lifts the ~20 KB inline-files limit (`ops/ax/README.md:117`); Workspace `files`
+  stay for the smoke Task and small cases.
+
+**One path for both profiles (d).** The store is required and verified in both; keeping the receiver for kind
+would keep `receiver_address`, `OUSAST_ARTIFACT_DIAL`, the `Host`-dial trick and the EndpointSlice alive for one
+profile and leave the kind proof proving a path k3s never runs. The receiver thread, `receiver-service.yaml.tmpl`,
+`receiver_cluster_ip` and the `up.sh` step go after increment 2's proof (the `harnessx-removal` sequencing rule:
+nothing deleted before its replacement passes). Tests that used `OUSAST_ARTIFACT_HOST` move to a local HTTP PUT
+target that accepts presigned-shaped URLs.
+
+**Egress per task (c).** `policy_for` drops the `http` rule and emits one `tls_passthrough` rule on 443 for the
+sorted union of the store endpoint's hostname (`urlsplit(S3_ENDPOINT).hostname`, IPs rejected as today), the bound
+Workspaces' Git hosts (`git_hosts`) and the bound Model's `openultrasast.io/egress-hosts`. A model-free task's
+policy names the store and its Git hosts only. The gateway already passes TLS to the Model host by hostname, so
+the store host is the same mechanism (`egress.py:7-16`). Open question: the store endpoint must be a DNS name
+the gateway resolves; `files.because-security.com` is; an in-cluster RustFS would need a resolvable Service name.
+
+### 4. The engine as a plane task (Req 4)
+
+**Decision.** `plane/tasks/engine.py` runs in the Joern image under the runner contract, on a second WorkerPool
+`engine-pool`. The root `Dockerfile` gains the two lines `Dockerfile.runner:25-29` has (the `ax-task-runner`
+shim and `/workspace`), so one *contract* in two images: `ousast-runner` (slim, PyYAML) and `ousast-engine`
+(JRE + Joern + php-cli, ~2.8 GB). **Rejected:** Joern inside the single runner image (every task would pull
+2.8 GB and every golden snapshot carry it; the engine runs for a minority of cases); Docker-in-pod (no socket on
+containerd; `Dockerfile:1-7` already argues against it).
+
+- **Task.** Template `plane/tasks/engine.yaml`: `command: ["engine"]`, env `OUSAST_WORKSPACE_DIR`,
+  `OUSAST_FIXED_DIR`, `OUSAST_INPUT_CASE`, pins and `OUSAST_FAMILIES`, bound to the same two Workspaces the
+  `alerts` task binds (`generate.py:342-353`). The module lifts `benchmarks/push/finding_dump.py`'s scan into
+  `openultrasast.cpg.dump` (the image ships `src/`, not `benchmarks/`) and reuses `engine_alerts.engine_rows`,
+  `read_record` and the `alerts` helpers (`scan`, `coverage`, `write_rows`, `write_summary`). It emits the same
+  `alerts.jsonl` rows (`rule_id: engine:<family>`, `source: "engine"`) and `summary.json` (coverage with the
+  engine's languages, files and bytes read, questions, seconds, degradations per pin) as `alerts-engine`, with
+  `host: false`. The instrument rule stays: no `read N files` line or `questions == 0` fails the task loudly.
+- **Sizing from the record** (`benchmarks/measurements/2026-09-30-php-engine-alerts/record.json`): container
+  peak 2.129 GiB under a 3 GiB limit, 789 s for both pins of froxlor, deadline 1800 s per pin. Task resources:
+  requests 3 Gi / 1 CPU, limits 4 Gi / 2 CPU; `engine-pool` workers 4 Gi / 2 CPU, replicas 1 on kind (the 7 GB
+  host holds it only with `ousast-pool` at one worker), production replicas in `k3s.toml`. Task timeout: two pins
+  times 1800 s plus grace fits `OUSAST_TASK_TIMEOUT` 7200.
+- **Placement is the open question of this increment.** ax's templates set no worker selector
+  (`workerpool.yaml.tmpl:1-3`), so Substrate picks any free worker of the sandbox class. The kind proof must show
+  a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a
+  separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays
+  `alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such.
+  **Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.**
+  With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor
+  requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS
+  1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so
+  group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether
+  Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the
+  k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path.
+- **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and
+  `docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose
+  `engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`.
+
+### 5. `ops/k8s/` layout (Req 5)
+
+```
+ops/k8s/
+  profiles/kind.toml, k3s.toml          # section 1; k3s values documented, not secrets
+  base/
+    workerpool-default.yaml.tmpl         # replicas, limits, nodeSelector from the profile (render: plane/k8s.py)
+    workerpool-engine.yaml.tmpl          # engine-pool, same template, engine values
+    ax-server-snapshots.yaml             # a strategic-merge patch setting AX_SNAPSHOTS_BUCKET (applied over ax's deploy)
+    rbac-ci.yaml                         # SA ousast-ci; Roles + RoleBindings in ax-system and ate-system (section 2)
+    secrets.example.env                  # the variable names only: DEEPSEEK_API_KEY, OPENROUTER_API_KEY, S3_*, AWS_*
+  k3s/
+    imagepullsecret.yaml.tmpl            # ghcr-pull from a GitHub PAT, referenced by the pools' pod template
+    hpa-default.yaml, hpa-engine.yaml    # External metric ate_workerpool_workers{state=at_capacity}, averageValue 0.7
+    prometheus-adapter.yaml              # from substrate/demos/autoscaled-workerpool, namespace ax-system
+  render.sh                              # ousast plane manifests --profile <p> --out DIR; kubectl apply -f DIR
+```
+
+`ops/ax/up.sh` keeps creating kind and calls `render.sh kind` for the pools; its receiver step is deleted in
+increment 2. **gVisor on k3s is a correction to the requirements text:** Substrate does not use a Kubernetes
+`RuntimeClass`. The `ateom-gvisor` worker (`workerImage: ko://.../cmd/ateom-gvisor`) fetches the gvisor release
+tarball named by `manifests/ate-install/sandboxconfig-gvisor.yaml` (`gs://gvisor/releases/nightly/...`) and runs
+`runsc` itself inside the worker pod (`cmd/ateom-gvisor/actorpath.go:35-45`). The node checklist is therefore:
+PodSecurity in `ax-system` admits the worker pod's security context (recorded from kind in increment 6), the
+node reaches the tarball URL or a mirror named in a copied SandboxConfig, and `/var/lib/ate` sits on local-path
+storage with room for golden actors (24 MB each, `ops/ax/README.md:112`). Autoscaling is k3s-only: the HPA
+needs Prometheus and the adapter (the Substrate demo README says kind-only for the demo; we take its manifests),
+and the 7 GB host cannot scale anyway. The queueing consequence of one actor per worker is recorded in the
+runbook: a Run with `--workers N` beyond the pool's replicas queues in ax as Suspended until a worker frees.
+
+### 6. `ousast plane scan <url> --base <c> --head <c>` (Req 6)
+
+`plane/scan.py`, reusing `generate.py`'s render path with a new case source:
+
+1. **Workspace generation** (`generate.scan_cases(url, base, head, cache)`): a shallow clone into the case cache
+   (`runner.find_cached_checkout` layout), two pinned Workspaces `<repo_dir>-<head7>` and `<repo_dir>-<base7>`
+   (`workspaces.py` shape, pins in `openultrasast.io/git-commits`), repository key by `memory.repo_key`.
+2. **Candidates from changed functions**: `generate.fix_ranges(repo, base, head, side="new")` gives the head-side
+   hunks (the `push/` delta primitive `_diff_records` is the same `git diff -U0`), `repo_facts.enclosing` names
+   the function per hunk line, filtered to `repo_facts.product_files`. Quick rules on the changed files
+   (`tasks.alerts.scan`) add alerted functions; `roles` runs once per repository when memory has no `roles` rows
+   for it; `repo-facts` supplies callers. `candidates.json` and `functions.json` are store inputs (section 3).
+3. **Run composition**: per changed-function set `facts -> va, vb -> agree -> vc -> final -> features ->
+   remember`, plus `engine` when a changed file's language is engine-covered and not quick-covered, or PHP
+   (`ENGINE_ALONGSIDE_QUICK`). Budgets from `generate.verify_budget` (0.022 USD per candidate, headroom 1.5);
+   `--ceiling` caps the sum. `--all` (whole repository, secondary) takes every product function.
+4. **Results** land in memory under `repos/<host>__<owner>__<name>/<head>.jsonl` through `remember` and
+   `ousast plane remember` (rows already carry `repo` and `pin`, `memory.py:77`); `runs/<run>/` holds the raw
+   artifacts for 30 days. The report prints the attribution table and the plane's agreement (`final`); the
+   decision engine's verdict only when a model is adopted (learned-decision-engine Req 6 unchanged).
+5. **Latency split (e).** `ousast pre-push` stays the blocking hook: quick rules and the delta engine inside
+   `OUSAST_PUSH_DEADLINE` (30 s, `ops/README.md:322`). `ousast plane scan --exec remote --no-wait` submits and
+   prints the Run name in the hook's report; `--wait SECONDS` blocks for CI. `ousast plane status <run>` reports
+   later. The wrapper `ops/pre-push` gains an opt-in `OUSAST_PLANE_SCAN=1`; it never blocks on the plane.
+
+### 7. Increment order, kind proof and measurement
+
+| # | Increment | Kind proof (`local` unless stated) | Recorded under `benchmarks/measurements/` |
+| --- | --- | --- | --- |
+| 1 | `PlaneProfile`, literals replaced, `doctor` by profile | `ousast plane doctor` passes with `kind.toml`; literal scan 0 | `<date>-k8s-01-profile/record.json`: doctor lines per profile, scan count |
+| 2 | Store delivery + receiver-less egress; receiver removed after | `ops/ax/smoke-run.sh` delivers `facts.json` via the store; policy has one TLS rule | `-02-store-delivery/`: bytes, PUT-to-poll seconds, rendered policies |
+| 3 | `--exec remote` from a CI-shaped container (kubeconfig + secrets only) against kind | smoke Run completes; audit log lists only the Role's verbs | `-03-remote-exec/`: kube audit verbs vs Role, run status |
+| 4 | Images on GHCR, digests, repin (section 8) | kind pulls `ghcr.io/norandom/ousast-runner@sha256` (pull secret if private); smoke Run | `-04-ghcr-images/`: sizes, pull seconds per image |
+| 5 | `engine` task + `engine-pool` | froxlor case as `engine`: rows equal the 2026-09-30 host record (6/6), placement on engine-pool shown | `-05-engine-task/`: rows diff, seconds, peak memory, placement |
+| 6 | Pools per profile, HPA manifests, node checklist | kind pools rendered and applied; worker securityContext recorded; HPA renders (k3s-only live) | `-06-pools/`: footprint table, securityContext |
+| 7 | `plane scan` on a real pair (a PHP CVE pair from the dev corpus) | Run completes; rows under `repos/<repo>/<head>` | `-07-plane-scan/`: candidates, cost, agreement, rows |
+| 8 | Runbook (`docs/deployment.md`), checklist mapped to R1-R7 | n/a (documentation; every "planned" label removed only with a recorded run) | the checklist links the records above |
+
+### 8. GitHub Actions and re-pinning
+
+`.github/workflows/images.yml` on tags `v*` (separate from `release.yml`, which builds the wheel): build
+`plane/Dockerfile.runner` as `ghcr.io/norandom/ousast-runner` and the root `Dockerfile` as
+`ghcr.io/norandom/ousast-engine` with `docker/build-push-action`, tag by version, and write the two digests to a
+release asset `images.json` (`{"runner": "ghcr.io/...@sha256:...", "engine": "..."}`) and to job outputs.
+`generate.repin_templates` (`generate.py:484-497`, one image for all templates) becomes `repin(plane, images)`:
+`engine.yaml` gets `images["engine"]`, every other template `images["runner"]`; `ousast plane repin images.json`
+exposes it and `--runner-image FILE` keeps working for a single-image file. Private packages: the pools' pod
+template names `imagePullSecret` from the profile.
+
+### 9. Tests
+
+- `tests/test_plane_literals.py`: scans `src/`, `plane/`, `ops/` (minus `ops/ax/up.sh`, `ops/k8s/profiles/`) and
+  `docs/` for `kind-ousast`, `localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py`
+  and the ax tunnel note; built at runtime like `test_removed_plane_references.py`.
+- `tests/test_plane_profile.py`: defaults, file + env precedence, unknown key rejected, no secret field, both
+  committed profiles load.
+- `tests/test_plane_k8s.py`: render both profiles; kind has no HPA and no pull secret; k3s has both; engine pool
+  limits >= the engine task's limits; every image reference is digest-pinned.
+- `tests/test_plane_rbac.py`: the rendered Role's verbs equal the set of Kubernetes API calls the CLI makes
+  (fake `kubectl`/`ax`/`kubectl-ate` record their argv; the test maps them to `pods/portforward`, `pods`,
+  `services`) and nothing more.
+- `tests/test_plane_egress.py`: for every committed Task template, `policy_for` yields one TLS rule whose hosts are
+  exactly the store host, the bound Git hosts and the Model hosts; no `http` rule; an IP endpoint is refused.
+- `tests/test_plane_delivery.py`: start body with `delivery` accepted; URLs never logged; expiry; PUT and GET
+  against a local server; `delivered()` polling; declared-only inputs.
+- `tests/test_plane_engine_task.py`: fixture record -> rows and summary equal `engine_alerts.produce`'s shape;
+  the instrument failures.
+- `tests/test_plane_scan.py`: candidates from a two-commit fixture repository; Run composition; budgets.
+- **Only live on kind:** placement on `engine-pool`; Substrate accepting the SA token for `egress-policy`;
+  the gateway passing TLS to the store host; GHCR pulls; the HPA (k3s only). These are the kind proofs above.
+
+## Data Models
+
+Store layout additions: `runs/<run>/state.json`, `runs/<run>/<task>/output.tar`, `runs/<run>/<task>/<artifact>`,
+`runs/<run>/<task>/done.json`, `runs/<run>/inputs/<name>`; all under the existing 30-day `runs/` rule. Start body:
+`credentials` as today plus optional `delivery {put, inputs{name: url}}`. `PlaneProfile` as in section 1.
+`images.json` as in section 8.
+
+## Error Handling
+
+A profile naming an unreadable context, a bucket that fails `verify_bucket`, or an images file without a digest
+fails before `ax apply` with the field named. A presigned PUT that fails three times ends the task `failed` with
+the HTTP status (`runner.deliver` semantics kept); an expired URL after a re-resume is re-minted by the next
+start. A store HEAD error during polling is retried with the resume backoff and reported after ten consecutive
+failures, like `ax get` (`reconciler.py`, `_await`). A `delivery` URL that is not `https` to a hostname is a 400
+from `StartGate.offer`. The engine task fails loudly on an unread input, as `read_record` does.
+
+## Requirement Traceability
+
+| Req | Components | Tests | Kind proof / record |
+| --- | --- | --- | --- |
+| R1 nothing assumes this host | `profile.py`, `doctor.py`, `egress.py`, `router.py`, `ops/k8s/profiles/` | literals, profile | increment 1 |
+| R2 reconciler in the cluster -> CLI with `--exec remote`; kind first-class | `reconciler.py` (`Delivery`), `Ax` with context, `rbac-ci.yaml`, store-backed state | rbac, profile | increments 3, 1 |
+| R3 receiver-less artifacts and egress | `delivery.py`, `runner.py` start body and PUT/GET, `egress.policy_for` | delivery, egress | increment 2 |
+| R4 engine as a task | `tasks/engine.py`, `cpg/dump.py`, `Dockerfile` shim, `engine.yaml`, `engine-pool` | engine task, k8s | increment 5 |
+| R5 sizing and autoscaling | `k8s.py`, `ops/k8s/base`, `ops/k8s/k3s` | k8s | increment 6 (HPA live only on k3s) |
+| R6 delta Run | `scan.py`, `generate.scan_cases`, `ops/pre-push` opt-in | scan | increment 7 |
+| R7 proven on kind, documented | every increment's record; `docs/deployment.md` | n/a | increment 8 |
+
+## Open Questions
+
+1. **Substrate authorization for the CI token.** `kubectl-ate` sends a ServiceAccount token; which audience and
+   which Substrate-side grant let `ousast-ci` create `egress-policy` in `default` is not in the checkouts read
+   here. Proven or refuted in increment 3.
+2. **Engine placement.** Whether Substrate places a 3 Gi actor only on workers that fit it (section 4). The
+   fallback is named.
+3. **gVisor tarball reachability on k3s** (`gs://gvisor/...` in the SandboxConfig): mirror or egress needed.
+4. **Store endpoint hostname** for any future in-cluster RustFS; today's external endpoint is fine.
+5. **Two images under one contract** versus the maintainer's "one runner image for all tasks": the design
+   reads it as one contract; confirm.
+6. **GHCR visibility** (public packages need no pull secret) and whether Substrate's and ax's `ko` images go to
+   the same registry namespace.
+7. **Reconciler size**: remote mode and `Delivery` must stay under the 500-line test; if they do not, the test's
+   bound moves with a reason, not silently.

.kiro/specs/plane-on-kubernetes/requirements.md

@@ -0,0 +1,135 @@
+# Requirements Document
+
+## Introduction
+
+The plane moves from this laptop's kind cluster to a Kubernetes cluster of the maintainer's choosing without
+changing what runs: the same ax manifests, runner image, memory store and credential path. Every host-bound
+piece becomes a cluster resource or a configuration value, each change is proven on kind first, and the
+production entry point (a delta Run for a push) is added. Context: `brief.md`, `docs/deployment.md`.
+
+## Boundary Context
+
+- **In scope**: two profiles of one code base, `kind` (local, kept) and `k3s` (production); a remote execution mode of the CLI; artifacts delivered to the store; configurable
+  cluster context, registry and image pins; our own snapshot and memory buckets; the Joern engine as a plane task
+  on a larger worker pool; worker pool sizing and autoscaling; a delta Run command and Workspaces from URL +
+  commit; a deployment runbook with a proof on kind and a checklist for the target cluster.
+- **Out of scope**: provisioning or operating the target cluster; modifying Agent Substrate or ax; any change to
+  detection, scoring, populations or the decision engine; replacing the `ax` CLI with a bespoke client beyond what
+  remote submission needs.
+- **Adjacent expectations**: `ai-service-plane` requirements (one Model + budget per task, attribution,
+  credentials only in the start request, egress policies, verify-at-startup memory store) keep holding;
+  `harnessx-removal` Req 6 (memory on the S3 store) keeps holding; nothing is deleted before its replacement
+  passes its test (the `harnessx-removal` sequencing rule).
+
+## Target Environment (maintainer, 2026-10-02)
+
+- **Cluster: k3s.** Consequences: containerd (no Docker socket; the engine's host-Docker path does not exist there),
+  gVisor needs `runsc` and a `RuntimeClass` installed on the nodes Substrate's workers run on, k3s ships Traefik
+  and local-path storage, and single-node k3s is close enough to kind that the kind proof transfers.
+- **Images and source: GitHub.** Images are published to GitHub Container Registry (`ghcr.io/norandom/...`),
+  built by GitHub Actions from tagged commits, digest-pinned in the manifests; private packages need an
+  `imagePullSecret` on the worker ServiceAccounts and the CI principal. Substrate's and ax's own images are built with
+  `ko` from their checkouts and pushed to the same registry.
+
+## Requirements
+
+### Requirement 1: Nothing assumes this host
+
+**User Story:** As the maintainer, I want every host-bound assumption replaced by a cluster resource or a
+configuration value, so the plane runs the same on kind and on the target cluster.
+
+#### Acceptance Criteria
+
+1. The kube context, the registry (GHCR in production), the ax-server address, the router address, the receiver address and the
+   snapshot and memory buckets are configuration (environment or one config file), with the kind values as the
+   documented defaults for development; no module names `kind-ousast`, `localhost:5001` or `172.19.0.1`.
+2. `ousast plane doctor` checks the configured cluster, not kind by name, and reports each configured address.
+3. A test enumerates the source tree for host-only literals and fails on any.
+
+### Requirement 2: The reconciler stays the CLI, with a remote execution mode
+
+**User Story:** As the maintainer, I want to trigger the checks in the cluster from the CLI and get the results
+fed back, so that remote execution is a mode of the same tool, not a separate service.
+
+#### Acceptance Criteria
+
+1. `ousast plane run` (and the later `ousast plane scan`) takes an execution mode, `local` or `remote`, from the
+   profile or a flag; `remote` submits the Run to ax-server in the target cluster (over the cluster's ingress or a
+   kubeconfig), `local` is today's kind path. The manifests, the memory-key seeding, the budgets and the
+   attribution table are identical in both modes.
+2. In remote mode the CLI needs no inbound path: it polls the memory store for task completion and artifacts and
+   prints the same status, attribution table and verdicts; `ousast plane status <run>` reports a Run started
+   earlier from any machine with the store credentials.
+3. Provider and store credentials come from the caller's environment (a laptop `.env`, a CI secret), travel only
+   in the start request, and never appear in a manifest, an ActorTemplate or a log.
+4. The principal the CLI uses against the cluster has the least privilege the ax API needs and nothing against
+   Agent Substrate directly; a test compares the rendered role with the calls the CLI makes.
+5. kind stays a first-class local execution mode, not a stepping stone: `ousast plane run` from a laptop
+   against the kind cluster keeps working with the development profile (same code, same manifests, local
+   registry), and every release is proven on it before anything is applied to k3s.
+6. An unattended in-cluster reconciler (a Deployment or Job that runs without a caller) is a later option, not
+   part of this spec.
+
+### Requirement 3: Artifacts land in the store, not on a host
+
+**User Story:** As the maintainer, I want task artifacts delivered to the memory store so that no host has to
+be reachable from the cluster.
+
+#### Acceptance Criteria
+
+1. Tasks deliver their artifacts to the S3 memory store (presigned URLs minted by the CLI and carried in the
+   start request, or a scoped delivery credential; the design chooses one and states why) and the CLI reads
+   completion from the store. The receiver thread and the kind gateway EndpointSlice remain only if the design
+   shows the local profile needs them; one delivery path for both profiles is preferred.
+2. Egress policies grant each task exactly the hosts it needs (the store's endpoint, git hosts, the bound
+   Model's host) and nothing else; a test renders the policy for every committed Task.
+
+### Requirement 4: The engine is a plane task
+
+**User Story:** As the maintainer, I want Joern to run where the other tasks run, on a worker sized for it.
+
+#### Acceptance Criteria
+
+1. A second WorkerPool sized for the engine (memory and CPU from a measured run) exists alongside the default
+   pool; an `engine` task module wraps today's host engine run and emits the same `alerts.jsonl` rows and
+   coverage as `alerts-engine`.
+2. The engine image is digest-pinned in the registry like the runner image; its size and pull time are recorded.
+3. `alerts-engine` on the host remains available for development and is marked as such.
+
+### Requirement 5: Sizing and autoscaling
+
+**User Story:** As the maintainer, I want the pool to follow the load of pushes without manual resizing.
+
+#### Acceptance Criteria
+
+1. Worker pool replicas, limits and the autoscaling rule (Substrate's assigned-worker signal) are manifests under
+   `ops/k8s/` with the kind values as a small profile and a documented production profile.
+2. The runbook records the measured footprint per task type and the queueing consequence of one actor per
+   worker.
+
+### Requirement 6: A delta Run for a push
+
+**User Story:** As a user with a repository the project has never seen, I want a push checked by the plane.
+
+#### Acceptance Criteria
+
+1. `ousast plane scan <repo-url> --base <commit> --head <commit>` generates Workspaces from the URL and commits,
+   derives candidates from the changed functions (quick rules, inferred roles, repository facts), and submits a
+   Run of facts, verify a/b, tie-break and agree with per-task budgets; a whole-repository mode is secondary.
+2. Results are written to the memory store under the repository and head commit and reported with the
+   attribution table; the decision engine's verdict is reported only when a model is adopted, else the plane's
+   agreement (ai-service-plane Req 6 and learned-decision-engine Req 6 unchanged).
+3. The latency split is explicit: a blocking pre-push carries quick rules and the delta engine within its
+   deadline; plane verdicts arrive asynchronously.
+
+### Requirement 7: Proven on kind, then documented for the target
+
+**User Story:** As the maintainer, I want each step shown working on kind before I touch the real cluster.
+
+#### Acceptance Criteria
+
+1. Every requirement above is exercised end to end on the kind cluster (remote mode included, with the kind cluster as the remote target), with
+   a recorded run under `benchmarks/measurements/`.
+2. `docs/deployment.md` becomes the runbook: prerequisites for the target cluster (Substrate, ax, registry,
+   buckets, Secrets, node requirements for gVisor workers), the profile switch, and a checklist whose every item
+   maps to a requirement here; planned items are labelled planned until their run is recorded.

.kiro/specs/plane-on-kubernetes/spec.json

@@ -0,0 +1,22 @@
+{
+  "feature_name": "plane-on-kubernetes",
+  "created_at": "2026-10-02T20:00:00Z",
+  "updated_at": "2026-10-04T18:00:00Z",
+  "language": "en",
+  "phase": "implementation",
+  "approvals": {
+    "requirements": {
+      "generated": true,
+      "approved": true
+    },
+    "design": {
+      "generated": true,
+      "approved": true
+    },
+    "tasks": {
+      "generated": true,
+      "approved": true
+    }
+  },
+  "ready_for_implementation": true
+}

.kiro/specs/plane-on-kubernetes/tasks.md

@@ -0,0 +1,295 @@
+# Implementation Plan
+
+> **Status 2026-10-04.** Groups 1-2 are done on kind (records `benchmarks/measurements/2026-10-02-k8s-01-profile/`
+> and `2026-10-02-k8s-02-store-delivery/`). Task 2.6 is paused. The production target changed. The operator runs a
+> kube-ax cluster (k3s, Agent Substrate v0.3.0, AX v0.3.1, 2 nodes). There we may only submit AX Tasks: no k8s
+> object creation, no secrets, no access to `ate-system`. The engine now runs there as AX Tasks through
+> `benchmarks/learn/engine_trace_ax.py`. The engine-task image (`plane/Dockerfile.engine-task`) is built FROM
+> `ghcr.io/norandom/ax-task-runner:v0.3.1`. Tasks read and write through presigned HTTPS links, are named
+> `ousast-engine-*`, and run a 1.7 GB heap with `ActiveProcessorCount=2` and one resident Joern session per pin. A
+> pin whose sandbox dies is retried once under a new name, then handed to the VM lane (2.5 GB heap). Egress is
+> operator-managed: HTTPS 443 to public hosts is open; LAN and cluster-internal addresses are blocked. Archive
+> names (`.tar`, `.gz`, `.zip`, `.bin`) failed through the store host's Cloudflare cache until the operator added a
+> cache-bypass rule; presigned objects are staged as `.dat` as a safeguard. Groups 3-8 are **to re-plan** against
+> that cluster under roadmap G4 (`.kiro/steering/roadmap.md`, section of 2026-10-04). Their text below is kept
+> as written. G4 (detection on push for any repository: `ousast plane scan --base --head`, engine pins on kube-ax,
+> model calls on the VM or CI) is the new target for group 7, sized by what G1-G3 keep.
+
+One task group per increment of `design.md` section 7, in that order; a later group never starts before the
+earlier group's kind proof is recorded. Every task ships with its tests; a task is done when its tests pass in
+the host suite, ruff and mypy are clean, and nothing in `src/`, `plane/`, `ops/` (minus `ops/ax/up.sh` and
+`ops/k8s/profiles/`) or `docs/` names `kind-ousast`, `localhost:5001`, `172.19.` or a port-forward address
+(task 1.3 makes that a test). Nothing is deleted before its replacement passes its proof. Each group ends with
+its kind proof and the record it writes under `benchmarks/measurements/<date>-k8s-NN-<slug>/record.json`
+(commit, images, the instrument checks, the numbers). Steps that call a model are marked **paid: needs a budget
+go-ahead** with a ceiling; steps that need a hand on a cluster that does not exist yet are marked **manual**.
+
+- [x] 1. Profile and literals (design section 1; open-question checks 2, 3, 7)
+- [x] 1.1 `PlaneProfile`
+  - `src/openultrasast/plane/profile.py`: frozen dataclass with the fields of design section 1; `load_profile()`
+    reads `ops/k8s/profiles/<OUSAST_PLANE_PROFILE>.toml` (a name or a path; default `kind`), then applies
+    `OUSAST_<FIELD>` overrides; rejects an unknown key, a secret-looking key (`*_KEY`, `*_SECRET`), an `exec`
+    outside `local|remote`, an `images` file without digest pins. `ops/k8s/profiles/kind.toml` and `k3s.toml`
+    committed with the design's values; `ops/k8s/profiles/kind-images.json` from `~/.cache/ousast/ax-src/runner-image`.
+  - `tests/test_plane_profile.py`: defaults; file then env precedence; each rejection names the field; both
+    committed profiles load; no field holds a value matching a credential pattern.
+  - _Requirements: 1.1_
+- [x] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
+  - `doctor(profile)`: context from `profile.kube_context`; the registry check resolves the `images` file's runner
+    reference (`docker manifest inspect` for a remote registry, `/v2/_catalog` for a local one); `open_store`
+    verifies `profile.memory`; prints every configured address (Req 1.2). `Egress(context=...)`,
+    `open_router(context=...)` and `receiver_cluster_ip(context)` take the profile's context; `generate.ATESPACE`
+    becomes `profile.atespace`. `cli.py`: `ousast plane doctor|run|workspaces|harvest` accept `--profile`.
+  - `tests/test_plane_reconciler.py`, `test_plane_router.py`, `test_plane_egress.py`: the fakes receive the
+    profile's context, never a built one.
+  - Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile.
+  - _Requirements: 1.1, 1.2_
+- [x] 1.3 Host-literal scan test
+  - `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does)
+    over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`,
+    `localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged
+    `ax-tunnel`; fails listing `path:line`.
+  - Evidence: the test passes on this commit and fails when a literal is reinserted (shown once in the record).
+  - _Requirements: 1.3_
+- [x] 1.4 `up.sh` by profile
+  - `ops/ax/up.sh`: `KO_DOCKER_REPO` and the context from `kind.toml` (read with `python -m
+    openultrasast.plane.profile --print registry|kube_context`); writes `kind-images.json` instead of
+    `runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3).
+  - Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes.
+  - _Requirements: 1.1, 2.4_
+- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any`
+  - `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's
+    engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records
+    with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after.
+  - Evidence: `record.json` says whether Substrate places by resource fit (`placement: fit|any`). `any` selects
+    the design's fallback for group 5 and is written into `design.md` section 4 before group 5 starts.
+  - _Requirements: 4.1_
+- [x] 1.6 Reconciler bound (open question 7) -- 489 lines after 1.2; projected 514 during group 2: the bound moves to 550 in 2.4
+  - Record `wc -l src/openultrasast/plane/reconciler.py` (477 at 17ac7ad) and the lines task 2.4 may add
+    (receiver wiring removed: `-12`, `Delivery` calls: `+8` estimated). If the estimate crosses 500, the bound
+    in `tests/test_plane_reconciler.py:803` moves to 550 in task 2.4 with the reason in the assertion message.
+  - Evidence: the numbers in `record.json`.
+  - _Requirements: 2.1_
+- [x] 1.7 gvisor tarball reachability (open question 3) **manual** -- documented as planned in `docs/deployment.md` B.1; the live check waits for k3s
+  - `docs/deployment.md` (B.1, planned): the k3s node must fetch
+    `gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd` (`substrate/manifests/ate-install/sandboxconfig-gvisor.yaml`)
+    or a mirrored copy named in a copied SandboxConfig; the check is `kubectl -n ate-system logs` of the worker
+    pod's atelet fetch line, run by hand on the day k3s exists. Labelled planned until then.
+  - _Requirements: 7.2_
+- [x] 1.8 Kind proof and record 01
+  - `ousast plane doctor --profile kind` passes every line; `ousast plane doctor --profile k3s` fails only on the
+    unreachable context (the expected line); `test_plane_literals` 0 offenders; the probe of 1.5; the bound of 1.6.
+  - `benchmarks/measurements/<date>-k8s-01-profile/record.json`.
+  - _Requirements: 1.1, 1.2, 1.3, 7.1_
+
+- [ ] 2. Store delivery, receiver-less egress (design section 3)
+- [x] 2.1 Presigned URLs and the store run directory
+  - `src/openultrasast/plane/delivery.py`: `Delivery(store, run)` mints presigned PUT for
+    `runs/<run>/<task>/output.tar` and GET for `runs/<run>/<producer>/<artifact>` (boto3 `generate_presigned_url`
+    on `S3Client.sdk`, expiry `OUSAST_TASK_TIMEOUT + 600`); `delivered(task)` HEADs the tar; `collect(task)`
+    downloads, extracts under the run dir with the member checks `Receiver.accept` has, re-puts declared
+    `outputs[]` as single objects, writes `done.json`; `put_input(name, path)` for inputs no task produces;
+    `state.json` mirrored to `runs/<run>/state.json` and seeded from it on a rerun.
+  - `tests/test_plane_delivery.py`: against a local S3-shaped fake (PUT/GET/HEAD by key); URLs never appear in
+    logs; expiry; member rejection; declared-only re-put; state seeding.
+  - _Requirements: 3.1, 2.1_
+- [x] 2.2 Runner: `delivery` in the start body
+  - `src/openultrasast/plane/runner.py`: `StartGate.offer` validates an optional `delivery {put, inputs}` (https,
+    hostname, no echo); `_deliver_output` PUTs the tar to `delivery.put` (same retry and `DeliveryError`
+    semantics); `fetch_inputs` GETs from `delivery.inputs[name]`; `OUSAST_ARTIFACT_URL` and `OUSAST_ARTIFACT_DIAL`
+    stay honoured until task 2.6 so both paths run in the same build.
+  - `tests/test_plane_runner.py`: accepted body with `delivery`; 400 on an `http` or IP URL; PUT to a local
+    server; inputs fetched from presigned-shaped URLs; the marker and retry-only boot unchanged.
+  - _Requirements: 3.1_
+- [x] 2.3 Egress without the receiver
+  - `src/openultrasast/plane/egress.py`: `policy_for(task, workspaces, model, store_host)` emits one
+    `tlsPassthrough` rule on 443 over the sorted union of store host, Git hosts, Model hosts; no `http` rule; an
+    IP store host raises naming `S3_ENDPOINT`.
+  - `tests/test_plane_egress.py`: for every committed template under `plane/tasks/` the rendered policy is the
+    expected set; the IP refusal.
+  - _Requirements: 3.2_
+- [x] 2.4 Reconciler on `Delivery`
+  - `src/openultrasast/plane/reconciler.py`: `run()` opens the store first (no fallback), builds `Delivery`,
+    passes `delivery` to `router.starter`/`start_task` (body extended in `router.py`), `_await` on
+    `Delivery.delivered`, outcome from the collected `summary.json`; the receiver thread is kept behind
+    `OUSAST_DELIVERY=receiver` for this group only. Line bound per task 1.6.
+  - `tests/test_plane_reconciler.py`: the ax fake run completes through the delivery fake; a rerun from an empty
+    results root resumes from the store's `state.json`.
+  - _Requirements: 2.1, 3.1_
+- [x] 2.5 Kind proof and record 02
+  - `ops/ax/smoke-run.sh` with `OUSAST_DELIVERY=store`: `facts.json` and `summary.json` arrive through the store;
+    the actor's policy read back has one TLS rule naming the store host; `ousast plane status ax-e2e-smoke`
+    prints the table. Instrument: the tar's byte size from the store equals the extracted files' total.
+  - `benchmarks/measurements/<date>-k8s-02-store-delivery/record.json`: bytes, PUT-to-poll seconds, rules.
+  - _Requirements: 3.1, 3.2, 7.1_
+- [ ] 2.6 Remove the receiver (last; after 2.5 is recorded) -- paused 2026-10-04
+  - Delete `Receiver`, `_Handler`, `receiver_address`, `receiver_cluster_ip`, `RECEIVER_HOST`, `RECEIVER_PORT`
+    from `egress.py`; `OUSAST_ARTIFACT_URL/DIAL/HOST/PORT` from `runner.py` and `reconciler.py`; the
+    `OUSAST_DELIVERY` switch; `ops/ax/receiver-service.yaml.tmpl` and the `up.sh` receiver step; the EndpointSlice
+    from kind (`kubectl delete`). Docs: `docs/plane.md` sequence diagram, `docs/deployment.md`, `ops/ax/README.md`.
+  - Tests using `OUSAST_ARTIFACT_HOST` move to the delivery fake; `test_plane_literals` covers `172.19.`.
+  - Evidence: `ops/ax/smoke-run.sh` passes again on the build without the receiver; recorded in 02's record as
+    `receiver_removed: <commit>`.
+  - _Requirements: 3.1, 1.3_
+
+- [ ] 3. `--exec remote` from a CI-shaped principal (design section 2; open-question check 1) -- to re-plan (kube-ax, roadmap G4)
+- [ ] 3.1 Execution mode and RBAC manifests
+  - `cli.py`: `ousast plane run --exec local|remote` (default `profile.exec`); `reconciler.Ax` passes the
+    profile's context to `ax` (its context flag, read from `ax --help` and pinned in a test); `remote` refuses a
+    `file://` store. `ops/k8s/base/rbac-ci.yaml`: SA `ousast-ci` in `ax-system`; Roles and RoleBindings in
+    `ax-system` and `ate-system` with `pods` get/list, `services` get, `pods/portforward` create, nothing else.
+  - `tests/test_plane_rbac.py`: fake `kubectl`, `ax`, `kubectl-ate` record argv over a full fake Run; the
+    verbs implied (`port-forward` -> `pods/portforward` create + `pods` get + `services` get) equal the rendered
+    Role's rules exactly.
+  - _Requirements: 2.1, 2.2_
+- [ ] 3.2 Substrate's authorization of the SA token (open question 1)
+  - `ops/ax/probe-ci-token.sh`: `kubectl create token ousast-ci --duration=2h` into a kubeconfig; `kubectl ate
+    --token-file - get workers` and `create egress-policy` on a throwaway actor with that kubeconfig. If refused,
+    read `substrate/docs/api-guide.md` for the grant (atespace role or token audience) and add the manifest to
+    `ops/k8s/base/rbac-ci.yaml`; if no grant exists for a non-admin token, record it and the design's remote mode
+    uses the maintainer's admin kubeconfig for the proof, with the gap listed in the runbook.
+  - Evidence: `record.json` names the grant used (`grant: <kind/name>` or `admin-kubeconfig`).
+  - _Requirements: 2.1_
+- [ ] 3.3 CI-shaped container
+  - `ops/k8s/ci-shell.Dockerfile`: `python:3.12-slim` + this package + `kubectl`, `kubectl-ate`, `ax` (built from
+    the checkouts); no `.env`; runs with `-e KUBECONFIG=/run/kc -e DEEPSEEK_API_KEY -e S3_*` only.
+    `.github/workflows/plane-remote.yml` (dispatch only, no schedule): the same image running `ousast plane run
+    --exec remote --profile k3s` with repository secrets; disabled until k3s exists (labelled in the file).
+  - _Requirements: 2.2, 2.3_
+- [ ] 3.4 Kind proof and record 03 **paid: needs a budget go-ahead, ceiling 0.50 USD**
+  - From the container of 3.3 against kind (`kind.toml` with `exec = "remote"`): the smoke Run (model-free), then
+    one `verify` pass over one case of `plane/runs/validation-46.yaml` (budget `{usd: 0.5, calls: 60}`) so the
+    credential path is exercised remotely. Kube audit log (kind apiserver `--audit-policy` of the SA's verbs)
+    lists only the Role's verbs.
+  - `benchmarks/measurements/<date>-k8s-03-remote-exec/record.json`: audit verbs vs Role, grant of 3.2, run
+    status, usd spent.
+  - _Requirements: 2.1, 2.2, 2.3, 2.4, 7.1_
+
+- [ ] 4. Images on GHCR (design section 8) -- to re-plan (kube-ax, roadmap G4)
+- [ ] 4.1 `images.yml`
+  - `.github/workflows/images.yml` on tags `v*`: `docker/build-push-action` for `plane/Dockerfile.runner` ->
+    `ghcr.io/norandom/ousast-runner:<tag>` and the root `Dockerfile` -> `ghcr.io/norandom/ousast-engine:<tag>`;
+    digests as job outputs and as the release asset `images.json` (`{"runner": "...@sha256:...", "engine": ...}`);
+    `GITHUB_TOKEN` with `packages: write`.
+  - Evidence: a dry run on a `v*-rc` tag publishes both and the asset (the engine build is ~2.8 GB: record the
+    build minutes).
+  - _Requirements: 1.1, 4.2_
+- [ ] 4.2 Engine image under the runner contract
+  - Root `Dockerfile`: the `ax-task-runner` shim and `/workspace` as `plane/Dockerfile.runner:25-29`; the `s3`
+    extra is not needed in either image (the runner PUTs by URL).
+  - `tests/test_plane_images.py`: both Dockerfiles declare the shim and entrypoint (text check).
+  - _Requirements: 4.2_
+- [ ] 4.3 `repin` for two images
+  - `generate.repin(plane, images)`: `engine.yaml` (added in 5.1; until then absent is fine) takes
+    `images["engine"]`, every other template `images["runner"]`; `ousast plane repin images.json`;
+    `--runner-image FILE` keeps the single-image file working. `ops/k8s/profiles/k3s.toml` names `images.json`
+    downloaded from the release.
+  - `tests/test_plane_increment.py`: repin with a two-key file; a template with two image lines is refused.
+  - _Requirements: 1.1, 4.2_
+- [ ] 4.4 Kind proof and record 04
+  - Pools' pod template takes `imagePullSecret` from the profile (`ops/k8s/base/workerpool-*.tmpl`, task 6.1
+    brings the full template; here the field only); kind pulls `ghcr.io/norandom/ousast-runner@sha256:...` with
+    `ghcr-pull` if the package is private; the smoke Run passes on the GHCR image.
+  - `benchmarks/measurements/<date>-k8s-04-ghcr-images/record.json`: image sizes, pull seconds, visibility.
+  - _Requirements: 1.1, 4.2, 7.1_
+
+- [ ] 5. The engine as a plane task (design section 4) -- to re-plan (kube-ax, roadmap G4)
+- [ ] 5.1 `cpg/dump.py` and `tasks/engine.py`
+  - Lift `benchmarks/push/finding_dump.py`'s scan into `src/openultrasast/cpg/dump.py` (the script becomes a
+    thin caller; `ops/README.md:78` command unchanged). `src/openultrasast/plane/tasks/engine.py`: env
+    `OUSAST_WORKSPACE_DIR`, `OUSAST_FIXED_DIR`, `OUSAST_INPUT_CASE`, pins, `OUSAST_FAMILIES`, `OUSAST_DEADLINE`;
+    per pin the quick scan (`tasks.alerts.scan`) and the engine in-process; rows via `engine_alerts.engine_rows`,
+    the instrument checks via `read_record`'s rules; `summary.json` as `engine_alerts.produce` writes it with
+    `host: false`. `plane/tasks/engine.yaml`: `command: ["engine"]`, requests 3Gi/1, limits 4Gi/2, image from
+    `images["engine"]`. `generate.TEMPLATES` gains `engine`; `--loop` emits `engine` instead of `alerts` for a
+    case needing the engine when the images file has an engine key.
+  - `tests/test_plane_engine_task.py`: a fixture record -> rows and summary equal to `produce`'s shape; the
+    unread-input and zero-question failures; the generator's choice.
+  - _Requirements: 4.1_
+- [ ] 5.2 `engine-pool`
+  - `ops/k8s/base/workerpool-engine.yaml.tmpl` rendered by `plane/k8s.py` (task 6.1 generalises; here the engine
+    values): kind replicas 1, 4Gi/2; if 1.5 recorded `any`, apply the design's fallback and write it here.
+  - Evidence: `kubectl -n ax-system get workerpool engine-pool` ready on kind with `ousast-pool` at 1 replica.
+  - _Requirements: 4.1, 5.1_
+- [ ] 5.3 `alerts-engine` marked development-only
+  - `cli.py` help, `docs/plane.md` task table, `ops/README.md`: "development only (needs Docker on the host);
+    the plane task is `engine`". No code removed.
+  - _Requirements: 4.3_
+- [ ] 5.4 Kind proof and record 05
+  - The froxlor case of `validation-46` as an `engine` task on kind (model-free, deadline 1800 s per pin):
+    rows equal `benchmarks/measurements/2026-09-30-php-engine-alerts/` (6/6 per pin), placement on `engine-pool`
+    shown by `kubectl ate get workers`, peak memory from `kubectl top`.
+  - `benchmarks/measurements/<date>-k8s-05-engine-task/record.json`: rows diff, seconds per pin, peak memory,
+    placement, image size and pull seconds.
+  - _Requirements: 4.1, 4.2, 7.1_
+
+- [ ] 6. Pools, autoscaling, node checklist (design section 5) -- to re-plan (kube-ax, roadmap G4)
+- [ ] 6.1 `plane/k8s.py` and the manifests
+  - `src/openultrasast/plane/k8s.py`: `render(profile, out)` writes `workerpool-default.yaml`,
+    `workerpool-engine.yaml`, `ax-server-snapshots.yaml` (patch with `profile.snapshots_bucket`), `rbac-ci.yaml`;
+    for `k3s` also `imagepullsecret.yaml`, `hpa-default.yaml`, `hpa-engine.yaml`, `prometheus-adapter.yaml`
+    (copied from `substrate/demos/autoscaled-workerpool`, namespace `ax-system`, metric
+    `ate_workerpool_workers{state=at_capacity}`, `averageValue 0.7`). `ousast plane manifests --profile P --out
+    DIR`; `ops/k8s/render.sh`. `ops/ax/workerpool.yaml.tmpl` deleted once `up.sh` uses the renderer.
+  - `tests/test_plane_k8s.py`: both profiles render; kind has no HPA and no pull secret; k3s has both; engine
+    pool limits >= `engine.yaml` limits; every image digest-pinned; no host literal in the output.
+  - _Requirements: 5.1_
+- [ ] 6.2 Node checklist and footprint
+  - `docs/deployment.md`: the gVisor correction (no RuntimeClass; `ateom-gvisor` runs `runsc` from the
+    SandboxConfig tarball), PodSecurity for the worker pod's securityContext (recorded from kind), local-path for
+    `/var/lib/ate`, the measured footprint per task type (`verify`, `repo-facts`, `engine` from 05) and the
+    queueing consequence of one actor per worker with `--workers`.
+  - _Requirements: 5.2, 7.2_
+- [ ] 6.3 Kind proof and record 06 (HPA: render only; live is k3s-only, **manual** later)
+  - `render.sh kind` applied on kind replaces the hand template with identical pools (`kubectl diff` empty but
+    for labels); worker pod `securityContext` captured; `render.sh k3s` validates with `kubectl apply
+    --dry-run=client` against kind's API (CRDs present).
+  - `benchmarks/measurements/<date>-k8s-06-pools/record.json`: footprint table, securityContext, dry-run result.
+  - _Requirements: 5.1, 5.2, 7.1_
+
+- [ ] 7. `ousast plane scan` (design section 6) -- to re-plan (kube-ax, roadmap G4)
+- [ ] 7.1 Workspaces and candidates from URL + commits
+  - `generate.scan_cases(url, base, head, cache, *, all=False)`: shallow clone into the case cache, two pinned
+    Workspaces, head-side hunks via `fix_ranges(side="new")`, functions via `repo_facts.enclosing` over
+    `product_files`, quick alerts via `tasks.alerts.scan` on the changed files, `candidates.json` and
+    `functions.json` as store inputs (`Delivery.put_input`); `--all` takes every product function.
+  - `tests/test_plane_scan.py`: a two-commit fixture repository yields exactly the changed functions; a rename
+    and a deleted function are handled; `--all` count.
+  - _Requirements: 6.1_
+- [ ] 7.2 Run composition and the command
+  - `src/openultrasast/plane/scan.py` + `cli.py`: `ousast plane scan URL --base C --head C [--all] [--ceiling USD]
+    [--exec] [--wait S | --no-wait]`: per set `facts -> va, vb -> agree -> vc -> final -> features -> remember`,
+    `engine` when a changed file's language needs it, `roles` once per repository when memory has no `roles`
+    rows for `repo_key(url)`; budgets from `verify_budget`; Run annotated `openultrasast.io/population: scan`,
+    `split: <repo_dir>`; after the Run, `memory.ingest`, then the attribution table and the plane's agreement
+    (`final`), the decision engine's verdict only when adopted. `--no-wait` prints the Run name and exits 0.
+  - `tests/test_plane_scan.py`: the rendered Run for the fixture; budgets sum under the ceiling; `roles`
+    included only when the store lacks rows; the report's verdict line.
+  - _Requirements: 6.1, 6.2_
+- [ ] 7.3 Hook opt-in
+  - `ops/pre-push`: with `OUSAST_PLANE_SCAN=1`, after the blocking check, `ousast plane scan <remote url>
+    --base <old> --head <new> --exec remote --no-wait` and the Run name in the report; the hook's exit code never
+    depends on it. `ops/README.md` and `docs/deployment.md`: the latency split.
+  - `tests/test_push_hook.py`: the wrapper calls the scan only with the variable set and ignores its failure.
+  - _Requirements: 6.3_
+- [ ] 7.4 Kind proof and record 07 **paid: needs a budget go-ahead, ceiling 3.00 USD**
+  - A PHP pair from the development corpus (pmpro's CVE commit and its parent, never a v2 or v3 case) scanned on
+    kind: candidates, the Run through `final`, `engine` for the PHP files, rows under
+    `repos/<host>__<owner>__<name>/<head>.jsonl` on the store, the report; then `ousast plane status`.
+  - `benchmarks/measurements/<date>-k8s-07-plane-scan/record.json`: candidates, usd, calls, agreement, rows,
+    seconds from submit to verdict, whether the known CVE site is among the agreed.
+  - _Requirements: 6.1, 6.2, 6.3, 7.1_
+
+- [ ] 8. Runbook (design section 7, Req 7) -- to re-plan (kube-ax, roadmap G4)
+- [ ] 8.1 `docs/deployment.md` as the runbook
+  - Sections: prerequisites for k3s (Substrate, ax with the snapshots patch, GHCR and the pull secret, buckets,
+    the CI SA and its grant from 3.2, the node checklist of 6.2), the profile switch, `--exec remote` from a laptop
+    and from CI, and the checklist whose every item names its requirement and its record 01-07; items without a
+    recorded run (HPA live, 1.7's fetch) labelled planned. `ops/ax/README.md` and `ops/README.md` point to it;
+    the "moving to a separate cluster" tables are replaced by the checklist.
+  - _Requirements: 7.2_
+- [ ] 8.2 Spec closure
+  - `design.md` open questions updated with the outcomes of 1.5, 1.6, 3.2 (1.7 stays open until k3s); the
+    `kind-tooling` memory and `.kiro/steering/overview.md` current-state line updated; `/kiro-spec-status`.
+  - Evidence: every record 01-07 linked from the runbook's checklist.
+  - _Requirements: 7.1, 7.2_

.kiro/specs/pre-push-safety-net/implementation-group-4.md

@@ -67,7 +67,7 @@ real capability; task 8.3 still owns actual eligibility. Task 4.2 brings progres
 ## Task 4.3 verification
 
 The agent service rejected the task 4.3 dispatch with `agent thread limit reached`.
-The [kiro-impl skill](../../../.opencode/skills/kiro-impl/SKILL.md) explicitly provides:
+The [kiro-impl skill](../../../.agents/skills/kiro-impl/SKILL.md) explicitly provides:
 “If multi-agent is not available, fall back to manual mode execution for all tasks.”
 Implementation, review and completion checks therefore ran in the main context.
 This is a manual review record, not an independent-review claim.

.kiro/specs/pre-push-safety-net/implementation-group-5.md

@@ -7,7 +7,7 @@ Real capability eligibility and rollout readiness remain unproven.
 ## Execution and review mode
 
 Fresh implementer dispatch for 5.1 returned `agent thread limit reached`.
-The [Kiro implementation skill](../../../.opencode/skills/kiro-impl/SKILL.md) explicitly
+The [Kiro implementation skill](../../../.agents/skills/kiro-impl/SKILL.md) explicitly
 states: “If multi-agent is not available, fall back to manual mode execution for all tasks.”
 Its manual review rule permits review in the main context when a fresh reviewer is unavailable.
 This run therefore uses manual implementation and adversarial review, not independent review.

.kiro/specs/pre-push-safety-net/requirements.md

@@ -147,6 +147,43 @@ with Express-style routes and middleware, including vulnerable, fixed and benign
 frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or
 unsupported-property results separately from demonstrated vulnerability detection.
 
+## Requirement 9: Operate as an advisory safety net on any repository
+
+Added 2026-10-02, maintainer-approved, from the operational-readiness audit. The maintainer's
+definition: a user installs the hook on ANY repository and gets honest, advisory-only results;
+nothing ever blocks a push unless blocking was explicitly chosen.
+
+9.1 (added 2026-10-02, maintainer-approved) When the installed hook cannot run the check, because the
+`ousast` executable is missing or an argument or setting is invalid, the hook shall allow the push and
+print one line saying the check did not run and how to fix it; only an explicit `--mode blocking`
+opt-in may turn an analysis result into a rejected push.
+
+9.2 (added 2026-10-02, maintainer-approved) When a check is skipped or incomplete, the terminal output
+shall name each distinct reason in plain words, one line per reason, with repository paths decoded and
+no opaque hash tokens; a language with no engine frontend and no quick rules shall be named with its
+file count; a first push of a new branch shall say there was no base to compare against and what was
+checked instead. The JSON artifact keeps the raw reason values.
+
+9.3 (added 2026-10-02, maintainer-approved) Before the engine, within the same deadline, the Safety Net
+shall run the shipped quick pattern rules over the head side of the changed lines and show their
+findings as advisory pattern matches, labeled separately from admitted alerts. This amends 3.2 for this
+tier only: quick-rule findings are shown, never admitted, and never block.
+
+9.4 (added 2026-10-02, maintainer-approved) When a finding's operation lies in a file that is new in the
+head revision, the comparison shall classify it as new rather than unknown; an engine finding whose
+novelty is established but whose capability is not yet qualified shall be shown as an advisory engine
+finding, not an admitted alert. Genuinely unknown comparisons keep their vetoes.
+
+9.5 (added 2026-10-02, maintainer-approved) When the engine cannot finish within the deadline, one line
+shall say so; on explicit request (`--engine background`) the Safety Net shall start the engine
+detached after the push, record its result in the artifact directory, and show it on the next run as
+the engine result for that head from the previous push.
+
+9.6 (added 2026-10-02, maintainer-approved) The hook installer and remover shall ship in the wheel
+(`ousast pre-push install|uninstall`), refuse to overwrite an existing hook unless forced, and a
+hook-callable container wrapper shall pass the push input and artifact path through; the documentation
+shall give a first-run guide, a glossary of skip reasons, a CI recipe and help text for every flag.
+
 ## Accepted experimental acceptance profile
 
 These values are the initial implementation/evaluation profile accepted for task generation, not

.kiro/specs/pre-push-safety-net/tasks.md

@@ -475,7 +475,7 @@ Preserve all completed task evidence below.
   - _Depends: 13.1_
   - _Requirements: 6.2, 6.3, 8.1, 8.2_
 
-- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
+- [x] 14. Ask every declared family whether it establishes anything (precondition for M4)
   - Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
 - [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
   - Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
@@ -575,7 +575,7 @@ Preserve all completed task evidence below.
   - _Boundary: Evaluation harness, full-run integration_
   - _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
   - _Requirements: 6.2, 6.3, 8.1, 8.2_
-- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
+- [x] 13.4 Let a question inside one partition complete, whatever else the repository holds
   - **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
   - The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
   - **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
@@ -587,6 +587,7 @@ Preserve all completed task evidence below.
   - _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
   - _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
   - _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
+  - _Checked 2026-10-02 per the operational-readiness audit:_ implemented and verified on PMPro (120 of 127, above). 13.3 stays open on its own blocker; this box was left unchecked only because it waits on it.
   - _Depends: 13.3_
   - _Requirements: 2.1, 2.2, 6.3, 8.1_
 
@@ -619,6 +620,55 @@ Preserve all completed task evidence below.
   - _Requirements: 8.1, 8.2_
 
 
+- [x] 17. Operational readiness (2026-10-02)
+  - **Named by the 2026-10-02 operational-readiness audit**, maintainer-approved as the first increment. The maintainer's definition: a user installs the hook on ANY repository and gets honest, advisory-only results; nothing ever blocks a push. The audit found the plumbing sound (snapshots, resolution, deadline accounting, advisory exit 0, the JSON artifact) and five things in the way: the hook can block a push, quick rules are not part of pre-push, the engine cannot finish a 26-file repository inside 30 s, the terminal hides every skip reason, and there is no supported install path without the source checkout. Before numbers for every item are the audit's live runs on records (Python), serve-static (JavaScript) and unrolled/secure (Go), each with one planted commit.
+  - This group is plumbing and honesty. It enables no capability, edits no rule for a planted sample, and leaves graph reuse (the engine speed work decided in 11.6) as a later task.
+  - Record: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (see 17.7).
+  - _Done 2026-10-02:_ all seven subtasks done. The engine still does not fit the 30 s hook deadline (graph reuse, the 11.6 direction, stays a later task), and plane verdicts after the push were not in this increment.
+  - _Requirements: 9.1–9.6_
+- [x] 17.1 Never block a push the user did not choose to block
+  - The installed hook exits 0 with one line when `ousast` is missing (the check did not run, and how to install). In hook (stdin) mode, an argument or usage error (an invalid `OUSAST_PUSH_DEADLINE`, a cache directory inside the repository) is advisory: one line and exit 0. `--mode blocking` stays the explicit opt-in that can reject a push, and keeps its usage errors. The prior-hook input-retention failure keeps exit 1, because it is the chained hook's verdict that cannot be obtained, not ours.
+  - Files: `ops/pre-push`, `src/openultrasast/cli.py`. Tests: `tests/test_push_hook.py` (a push with `ousast` absent from PATH succeeds and prints the line; a bad deadline in hook mode exits 0; blocking mode keeps exit 2).
+  - Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0.
+  - _Done 2026-10-02:_ the hook prints one line and exits 0 (chaining a prior hook if present) when `ousast` (or `OUSAST_COMMAND`) is missing; in hook mode without `--mode blocking`, argparse/usage errors and unexpected exceptions print one line and exit 0. Tests: `test_missing_ousast_never_blocks_the_push`, `test_hook_mode_usage_errors_are_advisory_unless_blocking` (blocking keeps exit 2), and `test_cli_refuses_the_flag_outside_explicit_replay` updated (the experimental flag is still refused in hook mode, now advisory). Gates: 1987 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.1, 5.3, 5.4_
+- [x] 17.2 Name every skipped check in plain words
+  - One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: <lang> (<n> files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`).
+  - Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`.
+  - Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch.
+  - _Done 2026-10-02:_ new `push/explain.py` turns every raw coverage reason into one plain `Skipped:` line per distinct kind (hex paths decoded, sha256 tokens dropped and counted, unknown reasons still printed); the existing one-line `Notice:` keeps its next action and gains an engine-missing variant. `language_not_covered:<lang>:<changed>:<total>` is recorded for a changed language with no engine frontend and no quick rules; `ousast scan` prints `notice: language not covered: go (N files)` and its report says so. `reports.py` renders the eight previously dropped degradations and prints any reason it has no sentence for. The hook passes `OUSAST_COMPARISON_BASE`. The new-branch line names the base it lacks; what was checked instead lands with the quick tier (17.3). Tests: `tests/test_push_explain.py` (6), two in `tests/test_reports.py`; module-audit manifest regenerated for the new module. Gates: 1995 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.2, 5.2, 8.2_
+- [x] 17.3 Run the quick rules as the fast first tier
+  - Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block.
+  - Files: `src/openultrasast/push/quick.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_quick.py` (new).
+  - Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
+  - _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.3_
+- [x] 17.4 Classify a file new in head as new, not unknown
+  - `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
+  - Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
+  - Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
+  - _Done 2026-10-02:_ `ChangeContext.added_paths` records Git status-A paths (rename targets excluded). In `_compare_evidence`, after every existing context/head/deadline check, an operation in an added file is `new`/`file_added_in_head` when the claim is entailed, undischarged and has change evidence, and no base answer holds the same operation (else `operation_moved_within_change`); an unpaired add/delete still blocks as `change_context_unresolved`. Admission accepts `(new, file_added_in_head)`; with the empty registry such a candidate's only veto is `capability_unavailable`, and the terminal now shows it under "Engine: N finding(s) this push introduced. Advisory ... not an alert." Any comparison, evidence or location veto keeps a candidate out of that line. Tests: `tests/test_push_new_file.py` (9). Live engine evidence depends on host memory (17.7). Gates: 2007 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.4, 3.4_
+- [x] 17.5 Say when the engine cannot finish, and offer it in the background
+  - One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
+  - Files: `src/openultrasast/push/background.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/cli.py`, `ops/pre-push`. Tests: `tests/test_push_background.py` (new).
+  - Evidence: a hook push with `OUSAST_PUSH_ENGINE=background` followed by a second push showing the previous result.
+  - _Done 2026-10-02:_ `--engine inline|background|off` (hook: `OUSAST_PUSH_ENGINE`) and `--background-deadline` (default 900 s). `off` and `background` run the quick tier and record `engine_off`/`engine_background` with a plain skip line; the deadline line names the deadline and the background option. New `push/background.py` starts one detached explicit replay (`start_new_session`, stdin closed, output to a log) per artifact directory and repository (lock file with a live-pid check: a second start reports busy), writes its artifact under `<artifact dir>/engine/<repo key>/<head>.json`, and the next run prints "Engine result for <head> from the previous push: ..." once, then marks it shown. Not usable through the Docker wrapper (the container exits with the hook); documented in 17.6. Tests: `tests/test_push_background.py` (4, one launching a real detached run). Gates: 2011 passed, 42 skipped; ruff, format, mypy clean.
+  - _Requirements: 9.5, 4.1, 4.2_
+- [x] 17.6 Install, Docker and documentation for a first-time user
+  - `ousast pre-push install|uninstall [--force] [--docker] [REPO]` ships the hook as package data, honours `core.hooksPath`, refuses an existing hook unless `--force` (which chains it as `pre-push.before-ousast`), and works from a pip install. `ops/ousast-docker` is a hook-callable script (not a shell function) that passes stdin and the artifact path through. The `openultrasast:dev` image is rebuilt only if `df -h /` shows at least 4 GB free; otherwise the rebuild command is documented and the record says it was not rebuilt. `docs/scanning.md` gains a first-run guide, a glossary of skip reasons, a GitHub Actions recipe and full `--help` text; README tiers 1 and 2 updated, pulling `ghcr.io/norandom/openultrasast:2.0.1` where the wrapper allows.
+  - Files: `src/openultrasast/push/hook/pre-push`, `src/openultrasast/push/install.py` (new), `src/openultrasast/cli.py`, `ops/ousast-docker`, `docs/scanning.md`, `README.md`, `ops/README.md`. Tests: `tests/test_push_install.py` (new), `mkdocs build --strict`.
+  - Evidence: install from the package into a scratch clone, push through it, uninstall restores the prior hook.
+  - _Done 2026-10-02:_ `ousast pre-push install|uninstall [REPO] [--force] [--docker]` (`push/install.py`) installs the hook shipped as package data (`openultrasast/push/hook/pre-push`, byte-identical to `ops/pre-push`, verified in the built wheel), honours `core.hooksPath` without editing it, refuses an existing hook unless `--force` (ours is replaced; a foreign hook moves to `pre-push.before-ousast` and is chained), and uninstall removes only a hook it wrote and restores the chained one. `ops/ousast-docker` (also package data; `--docker` installs it beside the hook, which then uses it) is a script that mounts the repository read-only and the artifact directory at their own absolute paths, passes stdin with `-i`, runs as the user with no network and `--memory 3g`, defaults to `ghcr.io/norandom/openultrasast:2.0.1`, never pulls inside a hook, and exits 0 unless `OUSAST_PUSH_MODE=blocking`. The hook passes `--engine`/`--comparison-base` only when set, so an older ousast or image still runs. Every pre-push flag has help text; `docs/scanning.md` has the first-run guide, settings table, new-branch and background notes, the artifact keys, a skip-reason glossary (a test keeps it in sync with `push/explain.py`), a GitHub Actions recipe and the full `--help` text; README tiers 1 and 2 and `ops/README.md` updated (pull and tag the GHCR image instead of building). **Image not rebuilt:** `df -h /` showed 1.1 GB free (< 4 GB); rebuild with `docker build -t openultrasast:dev .`. Tests: `tests/test_push_install.py` (11) and the glossary test. Gates: 2022 passed, 42 skipped; ruff, format, mypy clean; `mkdocs build --strict` exit 0.
+  - _Requirements: 9.6, 7.4_
+- [x] 17.7 Prove it on the audit's three repositories
+  - Rerun records, serve-static and unrolled/secure (same planted commits) with the new build in quick mode and with the hook installed, plus one Docker engine run on serve-static if free memory >= 3 GB and free disk >= 1 GB. Prove each run read its input (files, bytes).
+  - Files: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json`. Evidence: before (the audit) and after: exit codes, wall time against the deadline, findings shown, every skip line printed. A planted sample still missed is a coverage gap, recorded, never a rule edit.
+  - _Done 2026-10-02:_ `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (before = the audit, after = this build installed from its wheel into a scratch venv). 10 hook pushes and 6 replays, all exit 0, 0.69-1.21 s against 30 s on the host; the quick tier read exactly the changed file and every head snapshot read the full tree (records 43,577 B, serve-static 61,525 B, secure 103,023 B). The planted JS `eval` in the new `debug.js` surfaces: quick tier on the host, and as two advisory engine findings classified `new`/`file_added_in_head` in one Docker engine run (190.9 s against a 300 s deadline; MemAvailable 3.28 GB, 1.16 GB disk). **The planted Python `%`-formatted SQL is still missed**: no quick rule matches it and the host has no engine. Recorded as a coverage gap, no rule edited. Go is named as not covered. The proof caught a wrong install hint (`pip install openultrasast`, not on PyPI), corrected in the hook in this commit.
+  - _Requirements: 9.1–9.6, 6.3_
+
+
 ## Implementation Notes
 
 - Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.

.kiro/specs/unseen-repo-evaluation/brief.md

@@ -0,0 +1,32 @@
+# Brief: does the safety net work on repositories it has never seen?
+
+Maintainer, 2026-10-04: "we want to spread the safety net across repos in general, unseen code. that was the purpose
+of the AB tests to see if our approach is generalizeable." And: "we should run our tests / runs in kube. now we can
+speed it up and design the workloads for ax directly. kind is only a fallback."
+
+## Why this spec exists
+
+The A/B experiments so far (exp-003, exp-004, exp-005) score the decision engine on repository-held-out folds of known
+fix pairs. That measures whether the vulnerable side of a pair scores above its fixed side. It does not measure what a
+safety net on an arbitrary repository lives on:
+
+- on an ordinary push, does it stay quiet? (false-alarm rate)
+- on a push that introduces a real vulnerability, does it say so? (catch rate)
+
+Populations v1 and v2 measured the engine scan on fresh cases once (1 of 11, 0 of 17) and are spent. Nothing since has
+measured push-level behaviour on unseen repositories.
+
+## What this spec delivers
+
+1. A frozen pool of repositories never used anywhere in development, with two kinds of change each: real
+   vulnerability-introducing commits and ordinary commits.
+2. A push-level metric: each change replayed through the hook exactly as a user would run it, and through other
+   arms (decision engine, engine evidence) where an experiment asks.
+3. Execution designed for the kube-ax cluster from the start: one push replay is one AX Task; kind and the VM are
+   fallbacks.
+4. A baseline for today's hook, then a table of arms scored on the same pool.
+
+## Not in scope
+
+Changing detection. This spec only builds the measurement. Improvements are separate, registered experiments judged
+by it.

.kiro/specs/unseen-repo-evaluation/design.md

@@ -0,0 +1,431 @@
+# Design Document
+
+## Overview
+
+This spec builds the instrument for roadmap G0: a frozen pool of repositories that no development step has seen,
+and a push-level score on it. The score is what a user of the hook sees. On a vulnerability-introducing push, did
+the hook say so (catch rate)? On an ordinary push, did it say anything (false-alarm rate)? Replays run as AX Tasks
+on kube-ax. kind and the VM run the same tasks as fallbacks.
+
+Four facts from the code shape the design:
+
+- **Populations already have a freeze-and-guard pattern.** `benchmarks/independent/freeze.py` verifies cases by
+  reading git only, `evaluate.py` scores with fixed rules and Wilson intervals, and
+  `tests/test_independent_population.py` fails when a reserved repository is named anywhere in `benchmarks/`,
+  `src/` or `tests/` outside its directory. Harvest batch 3 used that guard as a black box: candidates written,
+  guard run with output discarded, subsets bisected on red. That is how v3 stays unread.
+- **The hook can already replay a change.** `ousast pre-push PATH --base B --head H --artifact F` resolves two
+  local commits (`snapshot.py`, `resolve_replay`: no fetch, no merge-base), runs quick rules and the engine inside
+  one deadline (default 30 s, engine inline) and writes a JSON artifact. The terminal text has one `Skipped:` line
+  per skipped check. The artifact holds three finding streams: admitted defects, advisory engine findings
+  (new or worsened, vetoed only by capability) and quick-rule matches with their CWE.
+- **The AX pattern works but lives in one script.** `engine_trace_ax.py` (manifest, egress checks, presigned
+  objects, retry once under a new name, cleanup) and the lane scheduler in `engine_trace.py` (`run_plan`: fixed
+  lanes, fallback queue, `progress.json`, `STOP`, resume by input digest) are engine-specific.
+- **Placement on kube-ax is random.** One worker per node, shared by all actors. Two heavy tasks fit per worker,
+  but only two concurrent tasks in all guarantee at most two on one worker.
+
+### Goals
+
+- A pool frozen by code against every used source (Req 1), with two kinds of change per repository (Req 2).
+- A scorer whose numbers mean what the hook shows, with silent zeros visible (Req 3).
+- One replay is one AX Task via a dispatcher shared with the engine (Req 4); a baseline and fresh slices (Req 5).
+
+### Non-Goals
+
+Detection changes, population v3, model calls in the cluster, languages the hook does not analyse.
+
+## Architecture
+
+```mermaid
+flowchart LR
+    M[pool manifest slice] --> D[batch dispatcher on VM]
+    D --> P[presign spec GET and record PUT]
+    P --> T[AX Task ousast-engine-replay]
+    T --> G[shallow fetch base and head from GitHub]
+    G --> H[ousast pre-push --base --head]
+    H --> R[record tar: artifact, text, instrument]
+    R --> S[(store via presigned PUT)]
+    D -->|sandbox died twice or OOM| V[VM docker lane]
+    D -->|cluster down| K[kind lane, same tasks]
+    V --> S
+    K --> S
+    S --> C[scorer: catch, false alarm, coverage]
+    S --> A[model arms on VM, budgeted]
+    A --> C
+    C --> O[record: counts and digests]
+```
+
+## Components and Interfaces
+
+### 1. Pool sourcing and eligibility (Req 1)
+
+**Decision 1.1: sources and ecosystems.** Candidates come from GitHub reviewed advisories (`gh api /advisories`,
+the enumeration harvest-3 used) for pip, npm, maven and composer, published 2019 or later, whose CWE maps to one of
+the seven families through the project's family CWE table. Each needs a linked fix commit with one parent in one
+repository. OSV records for the same advisory give the first affected version. That version is a cross-check for
+the introducing commit (section 2). Reason: these four ecosystems are the five languages the hook analyses
+(Python, JavaScript, TypeScript, PHP, Java). GHSA links commits and CWEs; OSV adds version brackets. Rejected:
+drawing from every ecosystem (Go, Ruby, C#). That would mostly measure missing frontends, which is already known.
+Coverage still reports uncovered files inside the drawn repositories. Rejected: NVD and published fix datasets.
+NVD links commits weakly, and the datasets are mined corpora that development or model training has likely seen.
+
+**Decision 1.2: eligibility by code against every used source.** `benchmarks/unseen/eligibility.py` builds the
+used set before the draw. It never prints names; it records counts per source and a digest of the set.
+
+| Source (Req 1.1) | Where it is read |
+| --- | --- |
+| Pair corpora | every catalog, recipe and dataset file under `benchmarks/pairs/`, plus the local pointer-pair manifests |
+| Populations v1, v2 | `population-v1.toml`, `population-v2.toml`, read directly |
+| Population v3 | never opened; the existing guard test only (below) |
+| Harvest records | the batch catalogs under `benchmarks/pairs/advisory-fixes*` (the records hold counts only) |
+| Memory examples | `example` rows of the local FileStore and the S3 store (`repo` field) |
+| Experiment units | `plane/experiments/*.units.jsonl` and registered units files |
+| Measurement records | everything under `benchmarks/measurements/` and `benchmarks/experiments/` |
+| Anything else | a text sweep for `github.com/<owner>/<name>` over `benchmarks/`, `src/`, `tests/`, `plane/`, `.kiro/` |
+
+Names are normalised (lower case, no `.git`, no trailing slash). Each candidate is resolved through the GitHub API
+to its canonical `full_name`, its former names (redirects) and its fork network (`parent`, `source`). A candidate
+is out if any of these is in the used set. A fork shares code with what it was forked from.
+
+v3 is checked like harvest-3 did. The draft manifest is written into `benchmarks/unseen/`. Then
+`pytest -q tests/test_independent_population.py -k referenced` runs with its output discarded. The guard already
+sweeps `benchmarks/` outside the population directory. On red, the candidate list is bisected by rewriting the
+draft until green, and the dropped count is recorded. No v3 name is ever printed or read by a person.
+
+**Decision 1.3: post-cutoff preference.** An advisory published after 2026-06-01 (harvest-3's cutoff for the
+decision engine's model) is flagged `post_cutoff`. Within each ecosystem stratum post-cutoff advisories are drawn
+first. Results are reported for both groups. Reason: a post-cutoff advisory cannot be in the model's training
+data as a known vulnerability. Rejected: post-cutoff only. Harvest-3 kept 12 post-cutoff pairs out of 251, and
+three slices of 100 repositories cannot be filled from that window.
+
+**Decision 1.4: licences.** The licence is read at the pinned head commit (LICENSE file, then the API's SPDX id).
+Only permissive licences are published by name: MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, 0BSD, Unlicense,
+Zlib, PostgreSQL, BSL-1.0. Everything else (copyleft, source-available, none) goes to the gitignored
+`benchmarks/unseen/private/pool-p1.toml`. The committed manifest then holds an opaque id and the sha256 of
+(URL, commits) for that entry, so the freeze digest still covers it. Reason: Req 1.4 read literally. Rejected:
+"OSI-approved" as the bar, as population v2 used. It is wider than the requirement says (open question 1,
+resolved 2026-10-04: copyleft stays private).
+
+Also required: anonymous HTTPS, at most 500 MB, at least 40 eligible ordinary commits, one per fork network.
+
+### 2. Change extraction (Req 2.1, 2.2)
+
+Extraction runs on the VM over a full `--filter=blob:none` clone in a scratch cache. It uses git only, like
+`freeze.py`. No scanner, engine or model touches a pool repository before the freeze.
+
+**Decision 2.1: the vulnerability-introducing range.** Let F be the fix and P its parent. The vulnerable lines are
+the old-side lines of F's hunks in the vulnerable function. For a pure-addition fix (an absence bug such as a
+missing guard), they are the vulnerable function's own lines.
+
+1. **`introducing` (preferred).** `git blame -w -M -C --porcelain P -L <lines> -- <file>`. The candidate V is the
+   commit that last wrote the sink line (for absence bugs, the function's declaration). V is accepted if all hold:
+   at V's first parent the sink line's normalised text is absent from the file's vulnerable function; V is not a
+   merge or root commit; V changes at most 50 files and 2,000 lines; when OSV names a first affected version, V is
+   reachable from that tag and not from the previous release tag. If the text is present at V's parent, blame
+   walks back from there, at most five steps. The replay is `base = V^`, `head = V`.
+2. **`last_touch` (fallback).** When no V passes, `git log -L :<function>:<file> P` gives the newest commit L
+   before F that changed the vulnerable function. The replay is `base = L^`, `head = P`. The vulnerable code is in
+   head, and the range contains a change to the vulnerable function.
+
+Each change records its method, family, vulnerable function (resolved in head), head-side changed lines,
+`post_cutoff` and the advisory id.
+
+Bias of each, stated in the record. `introducing` inherits SZZ noise: blame finds the last writer, not the author
+of the flaw. A refactor, a move the heuristics miss or a reformat can stand in for the real introduction. The
+absence check, the walk-back and the version bracket cut this down, but do not remove it. The bias runs both
+ways: a wrong V may introduce nothing (a miss is not the hook's fault) or may move a flaw (the delta-based hook is
+right to call it existing). `last_touch` is pessimistic for a delta hook. The flaw is often already in base, so
+the hook's novelty rule correctly stays quiet and the change counts as a miss. Catch is therefore reported per
+method. The headline is `introducing`; `last_touch` is a lower bound. Rejected: fix-parent against a fixed old
+base (for example 50 commits back). The range often misses the vulnerable function entirely and grows without
+bound on busy repositories.
+
+Label check: every third `introducing` change is read by an agent before the freeze, with no model call, as
+harvest-3 reviewed diffs. It is marked `confirmed`, `refactor` or `unclear`. The confirmed share is the label
+noise estimate. The scorer also reports catch on the confirmed subset.
+
+**Decision 2.2: ordinary commits.** The candidates are non-merge commits on the first-parent line of the default
+branch that change at least one source file of an analysed language. Excluded:
+
+- explicit security signals in the message: `secur|cve|ghsa|vuln|inject|xss|csrf|ssrf|exploit|sanitiz|escap`;
+- reverts (`^Revert`, `This reverts commit`);
+- dependency bumps (bot authors such as dependabot and renovate, `bump|upgrade .* from`, or only manifest and lock
+  files changed);
+- commits whose source changes are all generated or vendored (`vendor/`, `node_modules/`, `dist/`, `*.min.js`,
+  lock files, `linguist-generated` in `.gitattributes`, "generated" headers);
+- any fix commit of any advisory of the repository, and any commit touching a known vulnerable function of it;
+- commits over 1,000 changed source lines (counted, reported).
+
+Twelve are drawn per repository with a seeded random draw. Strata are three time thirds of the repository's
+history crossed with four size buckets of changed source lines (1-10, 11-50, 51-200, 201-1,000). Allocation is
+proportional to the repository's own commits per cell, with at least one per time third. Each change records base
+(its first parent), head, changed files and line ranges. Reason: proportional allocation keeps the false-alarm
+rate a per-push rate as a user meets it. Rejected: `freeze.py`'s broad `SECURITY_WORDS`. It also drops commits
+saying "query", "token", "session" or "valid", which removes the ordinary commits where false alarms are most
+likely, and the rate would look better than it is. Rejected: docs-only commits. They are trivially quiet and
+would dilute the rate.
+
+Label noise: an ordinary commit may itself introduce an unknown vulnerability, so a flag counted as a false alarm
+may be true. The false-alarm rate stays as Req 3.2 defines it. Flagged ordinary changes are adjudicated beside it
+(all if 30 or fewer, else every third) against protocol-v2's standard, and the adjudicated share is reported.
+
+### 3. Sizing and slices (Req 2.3, 5.2)
+
+Wilson 95% interval, z = 1.96: centre (p + z²/2n)/(1 + z²/n), half width z·sqrt(p(1-p)/n + z²/4n²)/(1 + z²/n).
+Taking the larger distance from p as the width, the smallest n that meets the bar, computed exactly:
+
+| Rate | Assumed p | Bar | n needed |
+| --- | --- | --- | --- |
+| false alarm | 0.05 | +/- 3 points | 315 |
+| false alarm | 0.10 | +/- 3 points | 483 |
+| false alarm | 0.20 | +/- 3 points | 756 |
+| catch | 0.10 | +/- 10 points | 62 |
+| catch | 0.25 | +/- 10 points | 88 |
+| catch | 0.40 | +/- 10 points | 97 |
+
+Catch: 100 vulnerability-introducing changes per slice, one per repository. At p = 0.4, Wilson gives
+[0.309, 0.498], within 10 points. One per repository keeps them independent.
+
+False alarm: ordinary changes of one repository are correlated (noisy code flags often). Wilson assumes
+independence, so the count is inflated by a design effect 1 + (m - 1)·ICC. With m = 12 per repository and an
+assumed ICC of 0.05, that is 1.55. 100 repositories × 12 = 1,200 changes, effective n 774 ≥ 756. The plain Wilson
+interval at p = 0.2, n = 1,200 is about +/- 2.3 points. A repository-cluster bootstrap interval is reported beside
+it, and the ICC measured on slice 1 is recorded. If it exceeds 0.05, the next pool sizes up. A frozen slice is
+never changed.
+
+**Decision 3.1: slices.** Pool p1 has three slices of 100 repositories each (100 + 1,200 changes per slice; 300
+repositories and 3,900 changes in all). Repositories are assigned by a seeded shuffle, stratified by ecosystem and
+`post_cutoff`. Slice 1 serves the baseline and the G1 arms table. Slices 2 and 3 serve the first two adoption
+decisions. When they are spent, pool p2 is drawn by the same code, and it excludes p1. Reason: each slice alone
+meets the Req 2.3 widths, so any decision can stand on one slice. Rejected: one large pool scored whole. Every
+decision would then see every repository, which Req 5.2 forbids. Rejected: ten small slices. None would meet the
+widths. Per family, 100 changes split over seven families give intervals of about +/- 20 to 25 points. They are
+reported, not sized (open question 3).
+
+Use ledger: `benchmarks/unseen/usage-p1.jsonl` is append-only, one row per use (slice, decision or experiment id,
+date, result digest). A slice with a row whose purpose is `informed` cannot appear again as `qualifies` for that
+change. A test checks this.
+
+### 4. Scoring rules (Req 3.1, 3.2, 3.3)
+
+Each replay uses the hook's defaults: `--deadline 30`, `--engine inline`, `--max-regions 500`, `--mode advisory`.
+That is what a user's push gets. The text output is kept with every `Skipped:` line, and so is the artifact.
+
+**Decision 4.0: a registered long-deadline arm** (maintainer decision 2026-10-04, open question 5). Beside the
+default replay, slice 1 is replayed once more with `--deadline 300` and every other flag unchanged. The arm is
+registered (manifest committed, `unseen: {pool: p1, slice: 1}`) before its first replay. G1 then reports both
+what the user sees (30 s) and what the engine could catch given time (300 s), so a miss caused by the deadline is
+told apart from a miss by the engine. It is a measurement arm, not a proposed default: adopting a longer hook
+deadline would be a separate decision under 7.2.
+
+**Decision 4.1: catch, per family.** A finding counts if its family is the change's family. Engine streams carry a
+family; a quick-rule match maps its CWE through the family CWE table. Its head location must also be in the
+vulnerable function or within 10 lines of the change's head-side hunks (protocol v1's window).
+
+| Family | Location rule |
+| --- | --- |
+| injection, path, deserialization, output_encoding, untrusted_destination | sink line in the vulnerable function, or the changed lines +/- 10 |
+| access_control | the vulnerable function itself (the handler that lacks the check), or the changed lines +/- 10 |
+| config_secrets | the changed file and lines +/- 10 (file scope; function `<global>`) |
+
+All three streams count: admitted defects, advisory engine findings and quick-rule matches. Each catch also names
+the stream that made it. Reason: the hook shows all three to the user. Rejected: admitted defects only. That
+scores the blocking path, which G3 decides separately, and would hide what users read.
+
+**Decision 4.2: false alarm.** An ordinary change is a false alarm when any of the three streams holds at least one
+finding, of any family, anywhere. The alerts-only rate (admitted defects) is reported beside it.
+
+**Decision 4.3: coverage, so zeros cannot hide.** Every change gets exactly one state:
+
+- `instrument_failure`: no record, a failed fetch, the head commit not verified, or zero bytes of the changed files
+  read. It is re-run once, then reported. It never counts as quiet or missed.
+- `unanalysable`: no changed file is in a language the hook covers (the hook's `language_not_covered` notice).
+- `quick_only`: the quick tier read the changed files, but no engine question completed on them.
+- `engine_covered`: at least one engine question completed on a changed file.
+
+The task entry proves its input as AGENTS.md requires. It records files and bytes in the checkout, bytes of each
+changed file, the JVM's wall time and the hook's exit code. A replay faster than a JVM can start is flagged
+`suspect_fast` and goes to the failure path. Catch and false alarm are reported over all changes and over each
+coverage state.
+
+### 5. The AX workload (Req 4)
+
+**Decision 5.1: the image.** `plane/Dockerfile.engine-task` gains a second stage, `FROM engine AS replay`, built
+with `--target replay` and pinned by digest. It adds `git`, the package with its runtime dependencies and
+`benchmarks/unseen/replay_entry.py`. Joern, php-cli, Java 21 and the frontend-retention patch come from the engine
+stage, so both lanes run the same Joern. The import smoke runs `ousast pre-push --help`. The root image stays
+`ghcr.io/norandom/ax-task-runner:v0.3.1`. Rejected: a separate Dockerfile. It would duplicate the Joern install,
+and two Joern copies are two instruments that drift. Rejected: one stage for both. The engine image's isolated
+venv without yaml and boto3 is proven, and replay dependencies would change its digest on every bump.
+
+**Decision 5.2: the task entry.** `replay_entry.py` is standard library only, like `engine_task_entry.py`, and
+reuses its `transfer` (egress readiness wait, three attempts, no URL text in errors).
+
+1. GET `SPEC_URL` (presigned, `.json`): repository URL, base, head, change id, hook flags.
+2. `git init`, then `git fetch --depth 1 origin <head>` and `git fetch --depth 1 origin <base>` over HTTPS
+   (GitHub serves commits by SHA). Verify both with `git cat-file -e`. The fetch is bounded at 180 s; the 500 MB
+   repository cap bounds its size.
+3. Run `ousast pre-push <checkout> --base <base> --head <head> --artifact /tmp/out/push.json` with defaults, the
+   backend heap at 1700 MB and the cluster JVM flags of `engine_trace_worker.py`. Capture stdout.
+4. Pack `push.json`, `push.txt`, `engine.json` (the head and base scan records and witness rows from the artifact,
+   keyed by repository and revision, for engine-evidence arms) and `instrument.json` (section 4.3, image digest,
+   step timings). PUT to `RESULT_URL` as `.dat`. The PUT is completion.
+
+On any error it PUTs a failure record naming the step and HTTP status, as the engine entry does. Task names are
+`ousast-engine-replay-<slug>-<digest>`, the prefix the operator's egress watcher matches. No key or store
+credential enters the manifest. Only the two presigned URLs do, and the spec object keeps repository names out of
+AX status and cluster telemetry.
+
+**Decision 5.3: the generic dispatcher.** `benchmarks/ax/batch.py` takes over from `engine_trace_ax.AX` and
+`engine_trace.run_plan`:
+
+- `Item`: id, input objects (name to bytes), the result object name, command, extra env, input digest.
+- `Workload`: image digest, allowed URL env names, deadline, how to validate a received result.
+- Lanes: `ax` (manifest, apply, poll the result object and AX phase, delete, clean objects), `kind` (the same
+  manifest against the kind AX), `docker` (the VM image, one container at a time).
+- Scheduler: two AX lanes plus one VM lane by default. A sandbox death is retried once under a new name; a second
+  death or a JVM OOM goes to the VM fallback queue, which beats overflow on the VM lane. `progress.json` is written
+  only by the scheduler. A `STOP` file halts new launches. Resume skips items whose stored record has the same
+  input digest. Cleanup failure stops dispatching.
+
+The engine pass becomes a workload over it, with the same tests. Reason: the egress, secrecy and cleanup logic took
+several fixes and must exist once. Rejected: a replay-specific copy of `engine_trace_ax.py`. Rejected: putting it in
+`src/` now. G4's `ousast plane scan` may need it there, and it moves when G4 is planned.
+
+Concurrency is two AX tasks. Placement is random, and only two guarantees two per worker at most. Four would need
+the operator to add placement (open question 2, resolved 2026-10-04: stays at two, no placement change
+requested). Estimated wall time per slice: about 75 s per replay (fetch, 30 s hook, setup), 1,300 replays over
+three lanes, about 9 hours. The long-deadline arm (4.0) is bounded at about 345 s per replay; at that bound 1,300
+replays over three lanes take about 42 hours, less where the hook finishes early. It runs with `STOP` and resume
+across sessions.
+
+**Decision 5.4: model arms on the VM.** Arms that need DeepSeek (the decision engine, the combiner) read the stored
+records on the VM. They build candidates from the head side's changed functions, score them, and take the change
+score as the maximum. Each runs under its experiment manifest's `budget_usd` through the metered client, dry run
+first, with responses cached. Nothing in the cluster calls a model.
+
+**kind (Req 4.5)** runs the identical manifest and image through the `kind` lane. Every new task is proven there
+before kube-ax, as `plane-on-kubernetes` requires.
+
+### 6. Records and the guard (Req 1.2, 1.3, 5.1)
+
+`benchmarks/unseen/` holds `pool-p1.toml` (repositories, licences, slices, changes with base, head, method,
+family, function, lines), `freeze-p1.json` (used-set digests and counts per source, guard result, label-check
+counts, freeze digest = sha256 of the canonical manifest), `protocol-p1.md` (the scoring rules of section 4,
+committed before the first replay) and `usage-p1.jsonl`. The `private/` directory is gitignored.
+
+The guard grows by one parametrised reservation. The existing sweep becomes a list of (reservation directory,
+names): `benchmarks/independent/` for populations and `benchmarks/unseen/` for pools, including the private
+manifest when present. It also sweeps `plane/`. A second test asserts that pool and population names are disjoint,
+and prints only a count on failure. A third asserts that a frozen pool's digest matches `freeze-p1.json`, so the
+pool is never edited. Raw per-change records stay outside the repository (`~/ousast-results/unseen/` and the
+store). Committed result records under `benchmarks/measurements/` carry counts, rates, intervals and digests,
+never names.
+
+### 7. Metrics and reporting (Req 3.3, 3.4, 5.3)
+
+`benchmarks/unseen/score.py` reports per family and overall: catch and false-alarm rate with Wilson 95%
+intervals, the cluster bootstrap interval for false alarms, coverage counts per state, catch per method,
+post-cutoff and per stream, and the alerts-only rate.
+
+**Decision 7.1: arms at fixed budgets.** An arm with a score is thresholded per budget (1%, 5%, 10%). The
+threshold is the lowest score whose false-alarm rate on the slice's ordinary changes stays within the budget.
+Catch at that threshold is reported with its Wilson interval and the realised false-alarm rate. That in-slice
+threshold is labelled exploratory. For a qualification, the threshold is fixed beforehand on an earlier slice or
+on held-out folds, and it is applied unchanged. A binary arm (today's hook) has one operating point, reported with
+the budgets it meets. Rejected: AUC over changes (it averages operating points no user runs).
+
+**Decision 7.2: next to held-out folds.** Experiment manifests registered after this spec lands carry
+`unseen: {pool, slice}`. `analyse` writes an `unseen` block into `result.json` next to the fold estimates: catch at
+each budget for each arm, and the paired difference B - A with an exact McNemar test on the 100 changes. The
+fold rule decides as today. The slice is recorded in the usage ledger.
+
+**Decision 7.3: adoption gate on the unseen slice** (amended 2026-10-04, maintainer decision; Req 5.4; open
+question 4 resolved). Reporting is not enough. An arm B is adopted over the production arm A only if, on the
+untouched slice that qualifies it, both hold:
+
+- catch at the 5% budget, with B's threshold fixed beforehand (7.1), is not significantly lower than A's: exact
+  two-sided McNemar on the paired vulnerability-introducing changes, alpha 0.05;
+- for an arm with one operating point (a binary arm such as a hook setting), its false-alarm rate is not
+  significantly higher than A's: exact two-sided McNemar on the paired ordinary changes, alpha 0.05, with the
+  repository-cluster bootstrap interval of the difference reported beside it, since those changes are clustered.
+
+`analyse` writes `unseen.gate: pass | regression | not_run` into `result.json`. `not_run` (no slice, slice
+already `informed` for this change, or more than 5% `instrument_failure`) blocks adoption like `regression`. The
+fold rule and this gate must both pass.
+
+### 8. Increments
+
+1. **Used set and guard.** `eligibility.py`, the extended guard and the usage-ledger test. Exit: guard green on an
+   empty pool; a planted used repository, a renamed one and a fork are each rejected in tests.
+2. **Sourcing and extraction.** Candidates, both change kinds and the label check, on the VM. Exit: a draft for 300
+   repositories with method counts, the confirmed share and exclusion counts.
+3. **Freeze p1.** Guard bisection for v3, then manifest, freeze record and protocol committed. Exit: freeze digest
+   committed and guard green.
+4. **Dispatcher.** `benchmarks/ax/batch.py`, with the engine pass moved onto it. Exit: existing tests green and a
+   two-pin engine smoke on kube-ax matching its stored records.
+5. **Replay task.** Image stage and entry, proven on kind, then kube-ax on five changes from this repository's own
+   history (never pool changes). Exit: instrument fields show bytes read and realistic JVM time, and quick-tier
+   output matches the VM lane.
+6. **Scorer.** `score.py` on synthetic records, including every coverage state and an instrument failure. Exit:
+   unit tests green.
+7. **Baseline.** Today's hook, from a frozen export of HEAD, on slice 1 (1,300 replays). Exit: the record committed
+   (counts and digests only) and a `baseline` row in the usage ledger. That is G0's exit.
+8. **Arms on slice 1 (G1)** (added 2026-10-04 with decisions 4.0 and 7.3). The registered long-deadline arm,
+   the engine-evidence arm from the stored records, the model arms on the VM (5.4) under their budgets, and the
+   `unseen` block with the adoption gate in `analyse`. Exit: one committed table of arms on slice 1 at the 1/5/10%
+   budgets (counts, rates, intervals, digests), and `exploratory` rows in the usage ledger.
+
+## Error Handling
+
+A failed fetch, a missing commit or a hook crash gives a failure record, never an empty success. A dead sandbox is
+retried once, then the VM runs it. A store or AX cleanup error stops the dispatcher. A slice with more than 5%
+`instrument_failure` after re-runs is not scored; the cause is fixed first.
+
+## Testing Strategy
+
+Unit tests: used-set normalisation (case, `.git`, renames, forks), blame acceptance on a synthetic repository,
+ordinary-commit exclusions, Wilson values against section 3, budget thresholds, coverage states, the task manifest
+and dispatcher resume and fallback with a fake lane. Guard tests run in CI; cluster runs stay manual.
+
+## Requirement Traceability
+
+| Requirement | Design |
+| --- | --- |
+| 1.1 eligibility against every source, v3 via the guard | 1.2 |
+| 1.2 frozen manifest, never edited | 6, increment 3 |
+| 1.3 pool directory, guard extended | 6 |
+| 1.4 licences, private manifest | 1.4 |
+| 2.1 introducing changes, family, function, post-cutoff | 1.1, 1.3, 2.1 |
+| 2.2 ordinary changes across time and size | 2.2 |
+| 2.3 sizes for interval widths | 3 |
+| 3.1 default replay, skipped checks kept | 4, 5.2 |
+| 3.2 catch and false-alarm rules | 4.1, 4.2 |
+| 3.3 per family, Wilson, coverage | 4.3, 7 |
+| 3.4 arms at 1/5/10% budgets | 4.0, 7.1, increment 8 |
+| 4.1 one change one Task, prefix, image by digest | 5.1, 5.2 |
+| 4.2 HTTPS fetch, presigned record, no keys | 5.2 |
+| 4.3 generic dispatcher, concurrency, retry, fallback, resume | 5.3 |
+| 4.4 model arms on the VM within budgets | 5.4 |
+| 4.5 kind runs the same tasks | 5.3, 5 (kind paragraph) |
+| 5.1 baseline on slice 1, counts and digests | 6, increment 7 |
+| 5.2 fresh slice per decision | 3.1, usage ledger |
+| 5.3 experiments report unseen next to folds | 7.2 |
+| 5.4 adoption needs no significant unseen regression | 7.3 |
+
+## Open Questions
+
+Maintainer decisions of 2026-10-04 resolve questions 1, 2, 4 and 5; question 3 stays open for G3.
+
+1. **Resolved 2026-10-04.** Licences: only permissively licensed repositories are published by name. Copyleft and
+   all other licences go to the gitignored private manifest, covered in the committed manifest by an opaque id and
+   a digest (Req 1.4 as written; decision 1.4). Population v2's OSI-approved bar is not used.
+2. **Resolved 2026-10-04.** Concurrency stays at two AX lanes plus the VM lane. No operator placement change is
+   requested (5.3).
+3. Per-family BLOCK (G3) needs about 100 changes per family. Pool slices together, or draw a family-enriched pool
+   for G3?
+4. **Resolved 2026-10-04.** An adoption requires no significant regression on the unseen slice, not only a report
+   (Req 5.4, amended; decision 7.3).
+5. **Resolved 2026-10-04.** G1 adds a registered long-deadline arm (`--deadline 300`) next to the default 30 s, so
+   the table shows both what the user sees and what the engine could catch (decision 4.0, increment 8).

.kiro/specs/unseen-repo-evaluation/requirements.md

@@ -0,0 +1,107 @@
+# Requirements Document
+
+## Introduction
+
+The safety net is meant for repositories it has never seen. This spec builds the instrument that measures that: a
+frozen pool of unseen repositories, a push-level metric (catch rate and false-alarm rate as the hook reports them),
+and an execution path that runs the replays as AX Tasks on the kube-ax cluster. Context: `brief.md`, the roadmap's
+G0-G1 (`.kiro/steering/roadmap.md`).
+
+## Boundary Context
+
+- **In scope**: pool selection and freezing; change extraction (vulnerability-introducing and ordinary commits);
+  replay of each change through `ousast pre-push` and through registered experiment arms; the metric and its
+  confidence intervals; AX Task execution with kind/VM fallback; the baseline for today's hook; rotation of pool
+  slices.
+- **Out of scope**: any detection change; population v3 (stays reserved and unread); model calls inside the cluster.
+- **Adjacent expectations**: `learned-decision-engine` experiments use this pool as their primary unseen-repo score;
+  `pre-push-safety-net` owns the hook being replayed; `plane-on-kubernetes` and the AX executor
+  (`benchmarks/learn/engine_trace_ax.py`) supply the execution pattern; the reserved-repository guard covers every
+  population and this pool.
+
+## Requirements
+
+### Requirement 1: A pool no development step has touched
+
+**User Story:** As the maintainer, I want the evaluation repositories to be ones nothing in development has seen, so a
+good number means the approach generalises.
+
+#### Acceptance Criteria
+
+1. A repository is eligible only if it appears in no pair corpus, no population (v1, v2, v3), no harvest record, no
+   memory example, no experiment unit and no measurement record. Eligibility is checked by code against every source
+   before the pool is frozen; v3 is checked through the existing guard without being opened.
+2. The pool is frozen (a committed manifest with repository URLs, commit SHAs, licences and the freeze digest) before
+   any change in it is replayed. A frozen pool is never edited; a correction is a new pool.
+3. Pool repositories are referenced only inside the pool's own directory (like `benchmarks/independent/`), and the
+   reserved-repository guard is extended to them.
+4. Only permissively licensed repositories are published by name; others may be used with their names kept out of
+   the repository (pointers in a local, gitignored manifest), recorded as such.
+
+### Requirement 2: Two kinds of change per repository
+
+**User Story:** As the maintainer, I want both vulnerable and ordinary pushes from the same repositories, so the
+false-alarm rate is measured on the code the catch rate is measured on.
+
+#### Acceptance Criteria
+
+1. Vulnerability-introducing changes: from public advisories with a linked fix, the commit range that introduced the
+   vulnerable code where it can be determined, else the fix's parent compared with an earlier base; each labelled
+   with family and the vulnerable function. Post-model-cutoff advisories are preferred and flagged.
+2. Ordinary changes: commits from the same repositories' histories with no security fix, revert or advisory link,
+   sampled across time and size; each recorded with its changed files and lines.
+3. Per repository, at least one vulnerability-introducing change and several ordinary changes; the pool's totals are
+   sized so the false-alarm rate's 95% interval is no wider than +/- 3 points and the catch rate's no wider than
+   +/- 10 points (sizes computed in the design).
+
+### Requirement 3: The push-level metric
+
+**User Story:** As the maintainer, I want the score to be what a user would see, so the number means something.
+
+#### Acceptance Criteria
+
+1. Each change is replayed as `ousast pre-push --base <base> --head <head>` with the hook's default settings, in a
+   clean checkout, with the record of every skipped check kept.
+2. Catch: a vulnerability-introducing change counts as caught when the hook (or the arm) reports a finding of the
+   right family on the vulnerable function or its changed lines. False alarm: an ordinary change counts as a false
+   alarm when the hook reports any advisory finding.
+3. Reported per family and overall: catch rate, false-alarm rate, and both with 95% Wilson intervals; plus coverage
+   (changes where the hook could analyse the changed language) so a silent zero is visible.
+4. Arms (decision engine, combiner, engine evidence, fix mechanisms) are scored on the same replays: catch rate at
+   fixed false-alarm budgets (1%, 5%, 10%), so arms are compared at equal noise.
+
+### Requirement 4: Execution on kube-ax, designed for AX
+
+**User Story:** As the maintainer, I want the replays to run on the cluster, so measurements are fast and the VM is
+free.
+
+#### Acceptance Criteria
+
+1. One replayed change is one AX Task, named with the `ousast-engine-` prefix the operator's egress watcher matches,
+   using an image built `FROM ghcr.io/norandom/ax-task-runner:v0.3.1` with this package, Joern and php-cli, pinned by
+   digest.
+2. The task fetches the repository at the needed commits itself over HTTPS (public hosts are reachable from actors)
+   and writes its record through a presigned link; no API keys or store credentials enter the cluster.
+3. A generic AX batch dispatcher serves both this replay and the existing engine passes: items to Tasks, at most the
+   cluster's safe concurrency (two heavy tasks per worker), retry once on sandbox death, then a VM or kind fallback
+   lane; progress and resume as in the engine runner.
+4. Arms that need model calls take the cluster's records and make their calls on the VM, within registered budgets.
+5. kind runs the same tasks unchanged as the local fallback.
+
+### Requirement 5: Baseline, rotation and use
+
+**User Story:** As the maintainer, I want a baseline now and a fresh slice for each decision, so the pool is not
+worn out by tuning.
+
+#### Acceptance Criteria
+
+1. The first use is a baseline of today's hook on the first slice; its record is committed (counts and digests only,
+   no repository names outside the pool directory).
+2. A slice that informed a change can no longer qualify that change; each adoption decision draws a new, untouched
+   slice, recorded with its freeze digest.
+3. Every learned-decision-engine experiment from now on reports its unseen-repo score from this pool next to its
+   held-out-fold score.
+4. An arm is adopted only if, on the untouched slice that qualifies it, it shows no significant regression against
+   the current production arm: catch at the 5% false-alarm budget not significantly lower, and for an arm with one
+   operating point the false-alarm rate not significantly higher (exact paired tests, two-sided, alpha 0.05).
+   Reporting the unseen score alone is not enough to adopt. *(amended 2026-10-04, maintainer decision)*

.kiro/specs/unseen-repo-evaluation/spec.json

@@ -0,0 +1,22 @@
+{
+  "feature_name": "unseen-repo-evaluation",
+  "created_at": "2026-10-04T18:00:00Z",
+  "updated_at": "2026-10-04T20:00:00Z",
+  "language": "en",
+  "phase": "implementation",
+  "approvals": {
+    "requirements": {
+      "generated": true,
+      "approved": true
+    },
+    "design": {
+      "generated": true,
+      "approved": true
+    },
+    "tasks": {
+      "generated": true,
+      "approved": true
+    }
+  },
+  "ready_for_implementation": true
+}

.kiro/specs/unseen-repo-evaluation/tasks.md

@@ -0,0 +1,384 @@
+# Implementation Plan
+
+One task group per increment of `design.md` section 8, in that order. A group starts only after the previous
+group's exit is recorded. Maintainer decisions of 2026-10-04 are folded in: a registered long-deadline arm
+(`--deadline 300`) beside the hook's default 30 s (design 4.0, group 8); adoption needs no significant regression
+on the unseen slice (Req 5.4 as amended, design 7.3, tasks 6.4 and 8.2); only permissively licensed repositories
+are published by name, everything else goes to the gitignored private manifest under a digest (Req 1.4, design
+1.4); concurrency stays at two AX lanes plus the VM lane (design 5.3).
+
+Execution rules:
+
+- **Who does what.** Codex implements every coding task offline: no network, no model, no store, no cluster. The
+  coordinator runs every networked step (GitHub and OSV API calls, git fetches and clones, the S3 store, image
+  pushes, kind and kube-ax). Those tasks are marked **(coordinator, network)**. Steps that need the coordinator but
+  no network (agent label reads, adjudication, the v3 guard bisection, commits) are marked **(coordinator)**.
+  Steps that spend model money are marked **paid: needs a budget go-ahead**. Each starts only after the maintainer
+  confirms a ceiling against the provider balance read just before it.
+- **Done means.** A coding task is done when its tests pass in the host suite, `ruff check`, `ruff format --check`
+  and `mypy` are clean, and the reserved-repository guard is green. No test makes a network, model or cluster call.
+- **Names stay in the pool directory.** Pool repository names and URLs appear only under `benchmarks/unseen/`
+  (copyleft and other non-permissive ones only under the gitignored `benchmarks/unseen/private/`). Raw per-change
+  records live in `~/ousast-results/unseen/` and the store, never in the repository.
+- **Population v3 is never opened.** It is checked only through the guard test, run as a black box with its
+  output discarded.
+- **Every group ends with a measured exit** and a record at
+  `benchmarks/measurements/<date>-unseen-NN-<slug>/record.json`: commit, image digests where used, the instrument
+  checks (bytes read, JVM time, exit codes), counts and digests. No repository names. The guard sweeps these
+  records.
+- **Instrument first** (AGENTS.md). A zero, an empty draft or a fast run is treated as an instrument failure until
+  bytes read and wall time prove otherwise.
+
+- [x] 1. Used set and guard (design 1.2, 6; increment 1)
+- [x] 1.1 Used-set builder
+  - `benchmarks/unseen/eligibility.py`: normalises names (lower case, no `.git`, no trailing slash) and builds the
+    used set from every source of design table 1.2: pair catalogs, recipes and datasets under `benchmarks/pairs/`
+    plus the local pointer-pair manifests as `benchmarks/pairs/catalog_gen.py` reads them; `population-v1.toml`
+    and `population-v2.toml`; the advisory-fix batch catalogs; memory `example` rows (`repo` field) through an
+    injected `MemoryStore` (`FileStore` locally, the S3 store when the coordinator passes one); experiment units
+    under `plane/experiments/`; `benchmarks/measurements/` and `benchmarks/experiments/`; a text sweep for
+    `github.com/<owner>/<name>` over `benchmarks/`, `src/`, `tests/`, `plane/` and `.kiro/`. It never reads
+    population v3 and never prints a name. Output: counts per source and the sha256 of the sorted set.
+  - `tests/test_unseen_eligibility.py`: a synthetic tree with one repository per source, each found; case, `.git`
+    and trailing-slash variants collapse; stdout and the output JSON contain no name; a v3-shaped file planted in
+    the synthetic tree is not opened (open is patched to fail on it).
+  - _Requirements: 1.1_
+- [x] 1.2 Canonical resolution, renames and forks
+  - `eligibility.py`: `resolve(candidate, api)` with an injected GitHub client returns the canonical `full_name`,
+    former names (redirects) and the fork network (`parent`, `source`). A candidate is ineligible when any of them
+    is in the used set; one candidate per fork network is kept.
+  - Tests with a fake client: a planted used repository, a renamed one and a fork of a used one are each
+    rejected; two forks of one network keep one; the rejection reason is a source label, not a name.
+  - _Requirements: 1.1_
+- [x] 1.3 Guard extended to pools
+  - `tests/test_independent_population.py`: the sweep becomes a list of (reservation directory, names):
+    `benchmarks/independent/` for populations, `benchmarks/unseen/` for pools (public manifests, plus
+    `benchmarks/unseen/private/*.toml` when present). The sweep also covers `plane/`. New tests: pool and
+    population names are disjoint (failure prints a count only); a frozen pool's manifest digest equals its
+    `freeze-pN.json` (skipped while no pool is frozen). The existing v3 handling is unchanged.
+  - Each reservation is exempt only inside its own directory: population names (v3 included) are still swept in
+    `benchmarks/unseen/`, the gitignored `private/` files included. Task 3.1 depends on this.
+  - `.gitignore`: `benchmarks/unseen/private/`.
+  - Evidence: the guard is green with an empty `benchmarks/unseen/`, and red when a test fixture names a pool
+    repository in `benchmarks/measurements/` (shown once in the record).
+  - _Requirements: 1.2, 1.3, 1.4_
+- [x] 1.4 Usage ledger
+  - `benchmarks/unseen/ledger.py`: append-only `usage-pN.jsonl`, one row per use (slice, decision or experiment
+    id, purpose `baseline | exploratory | informed | qualifies`, date, result digest, freeze digest). It refuses a
+    `qualifies` row for a change whose slice already has an `informed` row for it, and refuses to rewrite rows.
+  - `tests/test_unseen_ledger.py`: append, the refusal, a rewritten row detected by a digest chain.
+  - _Requirements: 5.2_
+- [x] 1.5 Used-set measurement and record 01 **(coordinator, network)**
+  - Run `eligibility.py` against the real sources, the S3 memory store included (read only).
+  - Instrument: each source reports files read and bytes; a source with zero rows fails the run unless it is
+    known empty.
+  - `benchmarks/measurements/<date>-unseen-01-used-set/record.json`: counts per source, used-set digest, guard
+    result (green on the empty pool), test counts of 1.1-1.4.
+  - _Requirements: 1.1, 1.3_
+
+- [ ] 2. Sourcing and extraction (design 1.1, 1.3, 1.4, 2, 3; increment 2)
+- [x] 2.1 Advisory candidates and repository filters
+  - `benchmarks/unseen/source.py`: from GHSA advisory JSON (pip, npm, maven, composer; published 2019 or later)
+    keep those whose CWE maps to a family through `src/openultrasast/model/taxonomy.py` and that link one fix
+    commit with one parent in one repository; attach the OSV first affected version; flag `post_cutoff` when
+    published after 2026-06-01. Repository filters: anonymous HTTPS, at most 500 MB, licence at the pinned head
+    (LICENSE file, then the API's SPDX id) classed `permissive` (the design 1.4 list) or `private`.
+  - `tests/test_unseen_source.py` on recorded fixture JSON (no network): CWE mapping, multi-parent and
+    multi-repository fixes dropped, cutoff flag, licence classing (GPL, AGPL, LGPL, none and source-available go
+    private).
+  - _Requirements: 1.4, 2.1_
+- [x] 2.2 Vulnerability-introducing range
+  - `benchmarks/unseen/extract.py`: the `introducing` method of design 2.1 (blame of the sink line, or the
+    declaration for an absence bug; absence at V's first parent; no merge or root; at most 50 files and 2,000
+    lines; OSV version bracket when present; walk-back of at most five steps) and the `last_touch` fallback. Each
+    change records method, family, vulnerable function resolved in head, head-side lines, `post_cutoff`, advisory
+    id. Git only; no scanner or engine.
+  - `tests/test_unseen_extract.py` on a synthetic repository built in a temp dir: accepted V; a reformat walked
+    back; a merge rejected; size cap; bracket rejection; fallback chosen when no V passes.
+  - _Requirements: 2.1_
+- [x] 2.3 Ordinary-commit draw
+  - `extract.py`: first-parent, non-merge commits touching an analysed language; the exclusions of design 2.2
+    (security words of the narrow pattern, reverts, bot and bump commits, manifest- or lock-only, generated or
+    vendored, advisory fix commits, commits touching a known vulnerable function, more than 1,000 source lines,
+    counted). Seeded draw of 12 per repository over three time thirds by four size buckets, proportional, at least
+    one per third; a repository with fewer than 40 eligible commits is dropped.
+  - Tests: each exclusion; the narrow pattern keeps "query", "token", "session"; allocation and the seed
+    reproduce the same draw; the 40-commit floor.
+  - _Requirements: 2.2, 2.3_
+- [x] 2.4 Draft manifest and slices
+  - `benchmarks/unseen/draft.py`: writes `benchmarks/unseen/draft-p1.toml` (permissive entries by name) and
+    `benchmarks/unseen/private/draft-p1.toml` (the rest); the public file holds an opaque id and
+    sha256(URL, commits) per private entry. Seeded shuffle into three slices of 100 repositories, stratified by
+    ecosystem and `post_cutoff`; each repository has one introducing change and 12 ordinary changes.
+  - Tests: the digest covers private entries; no private URL in the public file; the same seed gives the same
+    slices; per-slice strata within one repository of proportional.
+  - _Requirements: 1.2, 1.4, 2.3, 5.2_
+- [ ] 2.5 Sourcing run **(coordinator, network)**
+  - `gh api /advisories` enumeration, OSV lookups, canonical resolution against the used set (1.2), blobless
+    clones into a scratch cache, extraction (2.2, 2.3), draft (2.4).
+  - Instrument: clones counted with bytes on disk; a repository whose clone reads zero source files is an
+    instrument failure, not a drop. Fewer than 300 eligible repositories stops the run with counts; the
+    eligibility bars are not relaxed to fill it.
+  - Observable: a draft of 300 repositories (3,900 changes) on disk.
+  - _Requirements: 1.1, 1.4, 2.1, 2.2, 2.3_
+- [ ] 2.6 Label check and record 02 **(coordinator)**
+  - Every third `introducing` change read by an agent, no model call, marked `confirmed | refactor | unclear`
+    in the draft.
+  - `benchmarks/measurements/<date>-unseen-02-draft/record.json`: candidates per ecosystem, rejections per used
+    source, licence split (public, private), method counts, post-cutoff counts, confirmed share, ordinary-commit
+    exclusion counts. No names.
+  - _Requirements: 2.1, 2.2_
+
+- [ ] 3. Freeze p1 (design 1.2, 6; increment 3)
+- [ ] 3.1 Guard bisection tool
+  - `benchmarks/unseen/bisect_guard.py`: rewrites the draft with a candidate subset, runs
+    `.venv/bin/pytest -q -p no:cacheprovider tests/test_independent_population.py -k referenced` with stdout and
+    stderr discarded, reads only the exit code, and bisects until green. Reports the dropped count only.
+  - `tests/test_unseen_bisect.py` with an injected fake guard (a hidden set): finds and drops exactly the hidden
+    members; never logs a candidate; an exit code other than 0 or 1 aborts (instrument failure).
+  - _Requirements: 1.1_
+- [ ] 3.2 Freeze writer
+  - `benchmarks/unseen/freeze.py`: canonical `pool-p1.toml` (and the private counterpart) from the bisected draft;
+    `freeze-p1.json` with used-set digest and per-source counts, guard result, dropped count, label-check counts
+    and the freeze digest (sha256 of the canonical manifest). Refuses to write over a frozen pool; a correction
+    is `p2`.
+  - Tests: digest stable under key order; overwrite refused; the guard's digest test (1.3) passes on a fixture
+    pool.
+  - _Requirements: 1.2_
+- [ ] 3.3 Protocol p1
+  - `benchmarks/unseen/protocol-p1.md`: the scoring rules of design section 4 (default flags, decision 4.0's
+    long-deadline arm, catch and location rules, false alarm, coverage states), the statistics of section 3 and
+    7, and the adoption gate of 7.3, stated before any replay.
+  - Observable: the file exists and its sha256 is listed in `freeze-p1.json`.
+  - _Requirements: 3.1, 3.2, 3.3, 3.4, 5.4_
+- [ ] 3.4 Freeze run and record 03 **(coordinator)**
+  - Run the bisection (3.1), then the freeze (3.2). Guard green, digest test active.
+  - `benchmarks/measurements/<date>-unseen-03-freeze/record.json`: freeze digest, protocol digest, dropped
+    count, final counts per slice and kind.
+  - _Requirements: 1.1, 1.2, 1.3_
+
+- [ ] 4. Dispatcher (design 5.3; increment 4)
+- [x] 4.1 Items, workloads and the AX lane
+  - `benchmarks/ax/batch.py`: `Item` (id, input objects, result object name, command, extra env, input digest),
+    `Workload` (image digest, allowed URL env names, deadline, result validator) and the `ax` lane, moved from
+    `benchmarks/learn/engine_trace_ax.py`: manifest render, egress readiness, presigned GET and PUT as `.dat`,
+    apply, poll result object and AX phase, delete, object cleanup. The `kind` lane is the same lane with the
+    kind context.
+  - `tests/test_ax_batch.py` with a fake `ax` CLI and a fake store: manifest holds only the allowed URL env
+    names and no credential; names carry the `ousast-engine-` prefix; cleanup on success and failure.
+  - _Requirements: 4.1, 4.2, 4.5_
+- [x] 4.2 Scheduler, fallback and resume
+  - `batch.py`: two AX lanes and one VM `docker` lane by default (one container at a time); sandbox death retried
+    once under a new name, a second death or a JVM OOM goes to the VM fallback queue ahead of overflow;
+    `progress.json` written by the scheduler only; a `STOP` file halts launches; resume skips items whose stored
+    record has the same input digest; a cleanup failure stops dispatching.
+  - Tests with fake lanes: retry then fallback; OOM straight to fallback; resume skips; `STOP`; cleanup failure
+    halts; never more than two AX tasks in flight.
+  - _Requirements: 4.3_
+- [x] 4.3 Engine pass on the dispatcher
+  - `benchmarks/learn/engine_trace_ax.py` and `engine_trace.run_plan` become an engine `Workload` over `batch.py`;
+    no behaviour change.
+  - Observable: `tests/test_ax_executor.py` and `tests/test_engine_trace.py` pass with their assertions unchanged
+    (moved imports only).
+  - _Requirements: 4.3_
+- [ ] 4.4 Engine smoke on kube-ax and record 04 **(coordinator, network)**
+  - Two pins through the new dispatcher on kube-ax, compared with their stored records.
+  - `benchmarks/measurements/<date>-unseen-04-dispatcher/record.json`: image digest, per-pin wall time and JVM
+    time, record digests equal to the stored ones (or the diff counted), cleanup result.
+  - _Requirements: 4.3_
+
+- [ ] 5. Replay task (design 5.1, 5.2; increment 5)
+- [x] 5.1 Image stage
+  - `plane/Dockerfile.engine-task`: second stage `FROM engine AS replay` adding `git`, the package with its
+    runtime dependencies and `benchmarks/unseen/replay_entry.py`; smoke `ousast pre-push --help`. The root image
+    stays `ghcr.io/norandom/ax-task-runner:v0.3.1`; the engine stage is unchanged.
+  - `tests/test_unseen_image.py`: parses the Dockerfile; the `replay` stage builds from `engine`; the root image
+    pin; the engine stage's text is byte-identical to before.
+  - _Requirements: 4.1_
+- [x] 5.2 Task entry
+  - `benchmarks/unseen/replay_entry.py`, standard library only, reusing `engine_task_entry.transfer`: GET the spec;
+    `git init` and depth-1 fetches of head and base over HTTPS (180 s bound), verified with `git cat-file -e`; run
+    `ousast pre-push` with the spec's hook flags (defaults, or `--deadline 300` for the long-deadline arm), heap
+    1700 MB and the cluster JVM flags; pack `push.json`, `push.txt`, `engine.json`, `instrument.json` (files and
+    bytes in the checkout, bytes of each changed file, JVM wall time, hook exit code, step timings, image
+    digest); PUT as `.dat`. Any error PUTs a failure record naming the step and status, never URL text.
+  - `tests/test_unseen_replay_entry.py`: a local bare repository and a localhost presign stub; success record
+    fields; a missing commit, a failed fetch and a hook crash each give a failure record; zero bytes read and a
+    replay faster than JVM start are flagged `suspect_fast`.
+  - _Requirements: 3.1, 4.1, 4.2_
+- [x] 5.3 Replay workload
+  - `benchmarks/unseen/replay.py`: builds `Item`s from a frozen slice and an arm (`default` or `long_deadline`);
+    the spec object carries the repository URL so names stay out of AX status and telemetry; task names
+    `ousast-engine-replay-<slug>-<digest>` with an opaque slug; the validator rejects a record without
+    `instrument.json`; records go to `~/ousast-results/unseen/<pool>/<slice>/<arm>/` and the store.
+  - Tests: no repository name in any task name or manifest; only the two presigned URLs in the env; input
+    digest changes with the arm's flags; `instrument_failure` items queued for one re-run.
+  - _Requirements: 3.1, 4.1, 4.2, 4.3_
+- [ ] 5.4 Proof on kind, then kube-ax, and record 05 **(coordinator, network)**
+  - Build and push the `replay` target, pinned by digest. Five changes from this repository's own history (never
+    pool changes) on kind, then the same five on kube-ax, then on the VM lane.
+  - Exit: instrument fields show bytes read and realistic JVM time on every lane; quick-tier findings equal
+    across lanes.
+  - `benchmarks/measurements/<date>-unseen-05-replay-task/record.json`: image digest, per-lane timings, bytes,
+    exit codes, the quick-tier comparison.
+  - _Requirements: 4.1, 4.2, 4.5_
+
+- [x] 6. Scorer (design 3, 4, 7.1, 7.3; increment 6)
+- [x] 6.1 Coverage states
+  - `benchmarks/unseen/score.py`: each record gets one state (`instrument_failure`, `unanalysable`, `quick_only`,
+    `engine_covered`) from `instrument.json` and the artifact; `suspect_fast` goes to failure; a slice above 5%
+    `instrument_failure` after re-runs is refused.
+  - `tests/test_unseen_score.py` on synthetic records: every state; the refusal; a failure never counts as quiet
+    or missed.
+  - _Requirements: 3.3_
+- [x] 6.2 Catch and false-alarm rules
+  - `score.py`: family match (engine family, quick-rule CWE through `taxonomy.py`), the location table of design
+    4.1, all three streams with the stream named per catch; false alarm on any finding of any stream; the
+    alerts-only rate.
+  - Tests: right family wrong place misses; `access_control` on the handler; `config_secrets` file scope; a
+    quick-rule-only catch; one advisory finding on an ordinary change is a false alarm.
+  - _Requirements: 3.2_
+- [x] 6.3 Rates, intervals and breakdowns
+  - `score.py`: Wilson 95% intervals; repository-cluster bootstrap for false alarms; ICC; breakdowns per family,
+    method, `post_cutoff`, stream, coverage state and the confirmed subset.
+  - Tests: the n values of design section 3 (315, 483, 756, 62, 88, 97) reproduced; bootstrap seeded; ICC on a
+    synthetic clustered set.
+  - _Requirements: 2.3, 3.3_
+- [x] 6.4 Budgets, paired tests, the gate and record 06
+  - `score.py`: per-budget thresholds (1%, 5%, 10%), labelled `exploratory` in-slice and `fixed` when supplied;
+    a binary arm's single operating point with the budgets it meets; exact two-sided McNemar for catch and for
+    false alarm; `gate(A, B)` returning `pass | regression | not_run` per design 7.3.
+  - Tests: thresholds on a synthetic score set; McNemar exact values; a binary arm with a significant
+    false-alarm increase gives `regression`; missing slice or excess failures give `not_run`.
+  - `benchmarks/measurements/<date>-unseen-06-scorer/record.json`: test counts and the digest of the scorer's
+    output on the committed synthetic fixture.
+  - _Requirements: 3.4, 5.4_
+
+- [ ] 7. Baseline (design 4, 6; increment 7; G0 exit)
+- [ ] 7.1 Frozen source and dry run **(coordinator, network)**
+  - Export HEAD to a frozen tree; build and push the `replay` image from it, pinned by digest.
+  - `benchmarks/push/python_overhead.py` run with the stub engine to size per-replay Python cost; dispatcher dry
+    run on slice 1 lists 1,300 items without launching.
+  - _Requirements: 5.1, 4.1_
+- [ ] 7.2 Baseline replay of slice 1 **(coordinator, network)**
+  - Default arm, 1,300 replays over two AX lanes and the VM lane (about 9 hours); `instrument_failure` items
+    re-run once; the run stops for a cause above 5% failures.
+  - Observable: 1,300 records in `~/ousast-results/unseen/p1/1/default/` and the store, each with
+    `instrument.json`.
+  - _Requirements: 3.1, 4.3, 5.1_
+- [ ] 7.3 Adjudication **(coordinator)**
+  - Flagged ordinary changes adjudicated against protocol-v2's standard (all if 30 or fewer, else every third);
+    the adjudicated share kept beside the false-alarm rate. ICC measured; above 0.05 noted as "next pool sizes
+    up".
+  - _Requirements: 3.2, 2.3_
+- [ ] 7.4 Record 07 and the ledger **(coordinator)**
+  - `benchmarks/measurements/<date>-unseen-07-baseline/record.json`: `score.py` output (counts, rates, Wilson and
+    bootstrap intervals, coverage per state, per family, method, stream), freeze digest, image digest, commit.
+    `usage-p1.jsonl` row `baseline`.
+  - Observable: guard green over the record; G0's exit met.
+  - _Requirements: 5.1, 5.2, 3.3_
+
+- [ ] 8. Arms on slice 1 (design 4.0, 5.4, 7.1-7.3; increment 8; G1)
+- [ ] 8.1 Registered long-deadline arm
+  - `plane/experiments/` manifest for the long-deadline arm: hook flags with `--deadline 300`, budget 0,
+    `unseen: {pool: p1, slice: 1}`, committed before its first replay. `src/openultrasast/learn/experiments.py`
+    accepts `unseen` and a hook-flag arm.
+  - `tests/test_learn_experiments.py`: the manifest loads; an `unseen` block without pool or slice is refused.
+  - _Requirements: 3.4, 5.3_
+- [ ] 8.2 `unseen` block and adoption gate in `analyse`
+  - `experiments.py` `analyse`: writes `unseen` into `result.json` (catch at each budget per arm, paired B - A,
+    McNemar, `gate`) from `score.py`; checks the usage ledger; adoption requires the fold rule and
+    `gate == pass`.
+  - Tests: `regression` and `not_run` each block adoption; an `informed` slice cannot qualify; result JSON shape.
+  - _Requirements: 5.2, 5.3, 5.4_
+- [ ] 8.3 Engine-evidence arm
+  - `benchmarks/unseen/arms.py`: a score per change from `engine.json` (witness rows on head-side changed
+    functions), no model; plugs into the budget thresholds of 6.4.
+  - Tests on synthetic records: score monotone in witness strength; a change without engine coverage gets no
+    score and is reported, not counted quiet.
+  - _Requirements: 3.4_
+- [ ] 8.4 Model-arm adapter
+  - `arms.py`: candidates from head-side changed functions of stored records, scored by the current decision
+    engine program (and the combiner when its program exists; else its row reads `not_built`); change score is the
+    maximum; metered client, dry run first, response cache, the manifest's `budget_usd` as the hard ceiling.
+  - Tests with a scripted client: dry-run estimate, ceiling stop, cache replay; no model call.
+  - _Requirements: 4.4, 3.4_
+- [ ] 8.5 Long-deadline replay of slice 1 **(coordinator, network)**
+  - 1,300 replays of the `long_deadline` arm, bounded at about 345 s each (up to about 42 hours over three
+    lanes), with `STOP` and resume across sessions; failures re-run once.
+  - Observable: 1,300 records under `~/ousast-results/unseen/p1/1/long_deadline/`, JVM time in each.
+  - _Requirements: 3.1, 3.4, 4.3_
+- [ ] 8.6 Model arms on slice 1 **(coordinator, network; paid: needs a budget go-ahead)**
+  - Dry run gives the candidate count; estimate at the injection slice's measured cost (about $0.008 per
+    candidate at k = 5); the maintainer sets the ceiling against the balance read just before. Then the decision
+    engine arm (and the combiner if built) on the VM.
+  - _Requirements: 4.4, 3.4_
+- [ ] 8.7 Record 08: the arms table **(coordinator)**
+  - `benchmarks/measurements/<date>-unseen-08-arms/record.json`: per arm (hook 30 s, hook 300 s, engine evidence,
+    decision engine, combiner or `not_built`) catch at 1/5/10% budgets with Wilson intervals and realised
+    false-alarm rates, coverage per state, spend; `exploratory` rows in `usage-p1.jsonl`.
+  - Observable: one committed table of arms on slice 1; G1's table exists.
+  - _Requirements: 3.4, 5.2, 5.3_
+
+## Implementation Notes
+
+- 2026-10-04: Tasks 1.1–1.4 implemented offline and independently reviewed. Task 1.5 remains for the
+  coordinator; no networked measurement or commit was performed. Command and API contracts:
+  `benchmarks/unseen/README.md`. Offline evidence:
+  `benchmarks/measurements/2026-10-04-unseen-01-offline/record.json`.
+- Population guard scope is preserved under the maintainer's hard rule: benchmarks/src/tests for
+  populations, plus plane for pools. Each directory exemption is reservation-specific, including private
+  pool manifests. Extending population scope to plane exposed 30 existing references; no historical
+  records were changed or exempted inside the existing population scope.
+- S3 inventory opens through `open_store(read_only=True)` because ordinary store construction writes a
+  Select capability probe. Default store behavior is unchanged; inventory uses LIST/GET only.
+- The freeze writer must match the canonical TOML digest encoding documented in the pool README and
+  exercised by `test_independent_population.py`; final pool discovery does not trust mutable status.
+
+### Group 2 implementation evidence (2026-10-04)
+
+Tasks 2.1–2.4 are implemented offline and independently reviewed. Task 2.5 is
+`python -m benchmarks.unseen.draft --root "$PWD" --s3-store s3:// --seed 20260601`
+with the documented PATH/PYTHONPATH and GH_TOKEN environment. No networked draw,
+label check, group-2 measurement record, guard bisection, or commit was performed.
+The group header remains pending its coordinator exits 2.5/2.6.
+
+The command rebuilds the group-1 used set, resolves renamed/fork repositories,
+uses disposable external blobless clones, re-reads licences at extracted heads,
+and emits counts/digests only. The README documents injected clients, method bias,
+the hard 300-repository floor, and an explicitly unmeasured resource estimate.
+Independent review caught and fixed ambiguous parent-function absence and stale
+candidate-404 handling; regression tests cover both.
+
+Validation: requested pytest selection 103 passed, 1 skipped, exit 0 (population-test
+output discarded); `ruff check .` exit 0; `ruff format --check .` exit 0 (411 files);
+`mypy src/openultrasast` exit 0 (160 source files). The CLI `--help` smoke exited 0.
+Source-read instrumentation printed 20,101 bytes for the extraction implementation.
+
+### Groups 4–6 offline implementation (2026-10-04)
+
+Tasks 4.1–4.3, 5.1–5.3 and 6.1–6.4 are implemented and independently reviewed.
+Tasks 4.4 and 5.4 remain coordinator-owned network proofs; group headers 4 and 5 stay
+pending those exits. No commit, network request, Docker, AX or kubectl command was run.
+The sourcing job's draft.py, source.py, extract.py and journal.py are unchanged.
+
+The shared dispatcher retains the engine adapter's historical questions.json object
+name to preserve its unchanged assertions; replay uses spec.dat and result.dat.
+The engine Dockerfile body is unchanged except its required AS engine alias; CI builds
+engine and replay separately. S3Store's read_only class default preserves existing
+constructor-bypassing transport fixtures; initialized read-only stores still refuse writes.
+
+Exact coordinator commands and input contracts are in benchmarks/unseen/README.md,
+including frozen TOML/private pointer verification, scoring joins, own-history proof
+preparation and anonymous measurement-record generation. JVM wall time is the hook's
+aggregate scan build/query timing and is labelled as such. Unsupported-only partitions
+with explicit language_not_covered evidence do not pretend to have launched a JVM.
+
+Record 06 contains the committed synthetic fixture and scorer-output digests. The
+reserved-repository guard was run as a black box with both output streams discarded.
+Independent review: APPROVED; all reported integration and coverage defects resolved.
+
+Final verification: requested pytest command 279 passed, 8 skipped, exit 0; separate
+`tests/test_ax_batch.py` 5 passed, exit 0; `ruff check .`, `ruff format --check .`,
+`mypy src/openultrasast`, and the output-discarded reserved-repository guard all exit 0.
+The own-history proof preparer and replay dry run read five items and exit 0.

.kiro/steering/overview.md

@@ -1,8 +1,23 @@
 # Overview — what this is, and what is actually true of it
 
-## Current state, 2026-10-02
+## Current state, 2026-10-04
+
+Everything below this section is dated history; where it disagrees with this section, this section wins. The
+direction lives in `roadmap.md`, section of 2026-10-04 (phases G0-G5).
+
+- **Goal.** A pre-push safety net that works on repositories it has never seen. Two numbers per push decide it:
+  catch rate on real vulnerability-introducing changes, and false-alarm rate on ordinary changes. Neither has been
+  measured yet; the unseen-repository pool (G0, spec `unseen-repo-evaluation`) comes first.
+- **Correction, 2026-10-04.** Infrastructure and engine work had drifted from that goal. It is now done only when
+  it removes a bottleneck for those measurements.
+- **Engine.** Joern runs on the operator's kube-ax cluster as AX Tasks, with a VM fallback lane; kind stays the
+  development profile. It is almost blind to fixes: paths differ between sides in 2 of 105 both-asked pairs.
+- **Decision engine.** Ranks but cannot block (exp-003 to exp-005 inconclusive; 710 fix pairs in memory). Fix
+  mechanisms is one G2 arm, paused after mining until the pool exists.
+- **Hook.** `ousast pre-push` is operational and advisory on any repository; rollout stays NO-GO.
+
+## State, 2026-10-02 (superseded by the section above)
 
-Everything below this section is dated history; where it disagrees with this section, this section wins.
 User-facing documentation: `README.md`, `docs/architecture.md`, `docs/evaluation.md`, `docs/decision-engine.md`,
 `ops/ax/README.md`.
 
@@ -200,10 +215,10 @@ Counted 2026-10-02 from each spec's `tasks.md` checkboxes and `spec.json`.
 |---|---|---|
 | `learned-decision-engine` | implementation | 11 of 17 tasks done |
 | `pre-push-safety-net` | implementation | 67 of 79 tasks done |
-| `contributor-scan` | implementation | 27 of 51 tasks done |
+| `contributor-scan` | paused | 27 of 51 tasks done; paused since 2026-09-23 (`v1.2.0-alpha.1` NO-GO), waits on pre-push M2/M3 |
-| `ai-service-plane` | implementation | 11 of 11 tasks done; the ax plane is the agentic path |
+| `ai-service-plane` | complete | 11 of 11; the ax plane is the agentic path |
 | `harnessx-removal` | complete | 8 of 8; HarnessX removed 2026-09-30 |
-| `finding-feedback-loop` | design | requirements approved, design awaiting approval |
+| `finding-feedback-loop` | dormant | requirements approved, design unapproved since 2026-09-09; depends on `contributor-scan` |
 | `model-grounded-detection` | complete | 16 of 16; built the arbiter this all rests on |
 | `authorization-obligations` | dormant | tasks approved, 0 of 24 started; superseded in practice by the dominance arbiter |
 | `reachability-flow-model` | dormant | 23 tasks generated, nothing approved; largely delivered by adopting Joern |
@@ -214,6 +229,5 @@ Counted 2026-10-02 from each spec's `tasks.md` checkboxes and `spec.json`.
 > **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax (`ai-service-plane`),
 > and evolutionary proposals come from the plane memory. The HarnessX specs in the table above are history.
 
-As of 2026-09-09: 97 source modules, 90 load-bearing and 7 standalone capabilities; 814 tests.
+151 source modules, 146 load-bearing and 5 standalone capabilities, 0 orphaned
-150 source modules, 145 load-bearing and 5 standalone capabilities, 0 orphaned
+(`benchmarks/measurements/2026-09-08-module-audit.json`, regenerated 2026-10-02); 2,023 tests collected.
-(`benchmarks/measurements/2026-09-08-module-audit.json`, regenerated 2026-10-02); 1,964 tests collected.

.kiro/steering/roadmap.md

@@ -1,5 +1,100 @@
 # Roadmap
 
+## Roadmap to detection with the decision engine on kube-ax (2026-10-04)
+
+This section supersedes the ordering below it. The older sections stay as history.
+
+### Destination
+
+On every push, from a hook or CI, OpenUltraSAST answers for the changed code. The answer is advisory for every
+family, and BLOCK only for families whose precision has been proven on repositories the system never saw. The work
+is split by what each place is good at:
+
+- **kube-ax cluster** (operator k3s, Agent Substrate v0.3.0, AX v0.3.1, 2 nodes): the Joern engine, as AX Tasks.
+  No API keys and no store credentials in the cluster; tasks read and write through short-lived presigned links.
+- **The VM or CI runner**: candidate selection, the decision engine's model calls (DeepSeek), the memory store,
+  and the fallback engine lane (2.5 GB heap) for pins that do not fit the cluster's 1.7 GB heap.
+- **kind** stays the local development profile, with the same code and manifests.
+
+### The question everything answers
+
+Does the safety net work on repositories it has never seen? That is the purpose of the A/B experiments, and every
+phase below is judged by it. On an unseen repository almost every push is an ordinary change, so two numbers decide
+whether the safety net is useful, measured on the hook's advisory output exactly as a user would see it:
+
+1. **Catch rate:** on real vulnerability-introducing changes in unseen repositories, how often it flags the change.
+2. **False-alarm rate:** on ordinary changes in the same unseen repositories, how often it flags anything.
+
+Within-pair AUC on known fix pairs answers neither. It stays a development signal, not the goal.
+
+### Where we are (2026-10-04)
+
+| Area | State | Evidence |
+| --- | --- | --- |
+| Generalisation measured so far | The classifier ranks above chance on repository-held-out folds (within-pair AUC about 0.76) but cannot reach the block bar; the engine-based scan failed fresh populations (v1 1/11, v2 0/17); push-level catch and false-alarm rates on unseen repositories have never been measured | exp-003/004/005, `results-v1.json`, `results-v2.json` |
+| Pre-push hook | Operational and advisory on any repository | `2026-10-02-pre-push-operational-1/record.json` |
+| Labelled data | 710 complete real fix pairs | `2026-10-04-decision-engine-harvest-3/record.json` |
+| Engine | Almost blind to fixes (paths differ between sides in 2 of 105 both-asked pairs) | `2026-10-04-joern-slice-leak-audit/record.json` |
+| Infrastructure | Engine runs on kube-ax as AX Tasks with a VM fallback; kind for development | memory `kube-ax-cluster` |
+
+Correction recorded 2026-10-04: the previous two days went mostly into infrastructure and engine internals that did
+not move a generalisation number. From here, infrastructure and engine work is done only when it removes a
+bottleneck for the measurements below.
+
+### Phases
+
+**G0. The unseen-repository pool and the push-level metric** (new spec `unseen-repo-evaluation`)
+
+1. A frozen pool of repositories never used for training, tuning, mining or harvesting, and not in any population.
+   Two kinds of change per repository: real vulnerability-introducing commits (the parent of a fix, post-cutoff where
+   possible) and ordinary commits from the same history.
+2. The metric: replay each change through `ousast pre-push` (and the decision engine where an arm uses it) and
+   record catch rate and false-alarm rate, with confidence intervals, per family and overall.
+3. Rotation: the pool is spent per decision, like the populations; a new slice is drawn for the next one.
+
+Exit: the pool is frozen and the current production hook has a measured baseline on it.
+
+**G1. Re-score the arms we have on the pool**
+
+Current hook (quick rules + engine), decision-engine advisory (current programs), combiner, engine evidence. Each is
+an arm with a registered decision rule, scored by catch rate at a fixed false-alarm budget.
+
+Exit: one measured table of arms on unseen code; keep only what generalises.
+
+**G2. Improve what moves the unseen-repo numbers**
+
+Candidates, each admitted only if it improves the pool metric on a fresh slice: fix mechanisms in the engine
+(approved plan, now one arm, not the spine), engine coverage and TypeScript, decision-engine prompt and retrieval
+changes, more labelled pairs from more distinct repositories.
+
+**G3. Decide BLOCK per family** on the pool's false-alarm rate and the pooled precision bound (Req 6.4).
+
+**G4. Detection on push for any repository** (`plane-on-kubernetes` P4 work: `ousast plane scan --base --head`,
+engine pins on kube-ax, model calls on the VM or CI), sized by what G1-G3 kept.
+
+**G5. Qualification and rollout:** adopt, commit `prediction-v3.json`, spend population v3 once, advisory rollout,
+BLOCK only for families that passed.
+
+### Cross-cutting rules
+
+- No hand-tuning per miss. Every detection change is a registered experiment or an admission by held-out
+  separation.
+- Records carry counts, digests and pattern names, never repository names: the reserved-repository guard covers
+  every population.
+- Instrument first: a zero from an unread input is a failed run (AGENTS.md).
+- Cluster stability: sandbox deaths are retried once, then sent to the VM lane; the operator owns workers, egress
+  and Substrate.
+
+### Spec map
+
+| Spec | Role in this roadmap | State |
+| --- | --- | --- |
+| `unseen-repo-evaluation` (new) | G0, G1: the pool and the push-level metric | To specify |
+| `learned-decision-engine` | G1-G3, G5; fix-mechanisms plan is one G2 arm | Implementation; experiments 3-5 recorded |
+| `plane-on-kubernetes` | G4; the engine executors for kube-ax | Groups 1-2 done (kind); production path re-scoped to the operator's kube-ax (AX Tasks), groups 3-8 to re-plan |
+| `pre-push-safety-net` | the hook replayed in G0-G1; M4-M5 in G5 | Operational increment 1 done; M4-M5 open |
+| `ai-service-plane` | The plane the above runs on | Complete |
+
 ## Status, 2026-10-02
 
 The milestone table below dates from 2026-09-14. Since then (`.kiro/specs/pre-push-safety-net/release-milestones.md`):

.opencode/skills/kiro-spec-batch/SKILL.md

@@ -1,165 +0,0 @@
----
-name: kiro-spec-batch
-description: Create complete specs (requirements, design, tasks) for all features in roadmap.md using parallel sub-agent dispatch by dependency wave.
----
-
-
-# Spec Batch
-
-<background_information>
-- **Success Criteria**:
-  - All features have complete spec files (spec.json, requirements.md, design.md, tasks.md)
-  - Dependency ordering respected (upstream specs complete before downstream)
-  - Independent features processed in parallel via sub-agent dispatch
-  - Cross-spec consistency verified (data models, interfaces, naming)
-  - Mixed roadmap context understood without breaking `## Specs (dependency order)` parsing
-  - Controller context stays lightweight (sub-agents do the heavy work)
-</background_information>
-
-<instructions>
-
-## Step 1: Read Roadmap and Validate
-
-1. Read `.kiro/steering/roadmap.md`
-2. Parse the `## Specs (dependency order)` section to extract:
-   - Feature names
-   - One-line descriptions
-   - Dependencies for each feature
-   - Completion status (`[x]` = done, `[ ]` = pending)
-3. If present, also read for context:
-   - `## Existing Spec Updates`
-   - `## Direct Implementation Candidates`
-   Do not include these in dependency-wave execution; they are awareness-only inputs for sequencing and consistency review.
-4. For each pending feature in `## Specs (dependency order)`, verify `.kiro/specs/<feature>/brief.md` exists
-5. If any brief.md is missing, stop and report: "Missing brief.md for: [list]. Run `/kiro-discovery` to generate briefs first."
-
-## Step 2: Build Dependency Waves
-
-Group pending features into waves based on dependencies:
-
-- **Wave 1**: Features with no dependencies (or all dependencies already completed `[x]`)
-- **Wave 2**: Features whose dependencies are all in Wave 1 or already completed
-- **Wave N**: Features whose dependencies are all in earlier waves or already completed
-
-Display the execution plan:
-```
-Spec Batch Plan:
-  Wave 1 (parallel): app-foundation
-  Wave 2 (parallel): block-editor, page-management
-  Wave 3 (parallel): sidebar-navigation, database-views
-  Wave 4 (parallel): cli-integration
-  Total: 6 specs across 4 waves
-```
-
-If roadmap contains `## Existing Spec Updates` or `## Direct Implementation Candidates`, mention them separately as non-batch items so the user can see the whole decomposition.
-
-## Step 3: Execute Waves
-
-For each wave, dispatch all features in the wave as **parallel sub-agents**.
-
-**For each feature in the wave**, spawn a sub-agent with this task:
-
-```
-Create a complete specification for feature "{feature-name}".
-
-1. Read the brief at .kiro/specs/{feature-name}/brief.md for feature context
-2. Read the roadmap at .kiro/steering/roadmap.md for project context
-3. Execute the full spec pipeline. For each phase, read the corresponding skill's SKILL.md for complete instructions (templates, rules, review gates):
-   a. Initialize: Read .opencode/skills/kiro-spec-init/SKILL.md, then create spec.json and requirements.md
-   b. Generate requirements: Read .opencode/skills/kiro-spec-requirements/SKILL.md, then follow its steps
-   c. Generate design: Read .opencode/skills/kiro-spec-design/SKILL.md, then follow its steps
-   d. Generate tasks: Read .opencode/skills/kiro-spec-tasks/SKILL.md, then follow its steps
-4. Set all approvals to true in spec.json (auto-approve mode, equivalent of -y flag)
-5. Report completion with file list and task count
-```
-
-If multi-agent is not available, execute features in the wave sequentially.
-
-**After all sub-agents in the wave complete**:
-1. Verify each feature has: spec.json, requirements.md, design.md, tasks.md
-2. If any feature failed, report the error and continue with features that succeeded
-3. Display wave completion: "Wave N complete: [features]. Files verified."
-4. Proceed to next wave
-
-## Step 4: Cross-Spec Review
-
-After all waves complete, spawn a **single sub-agent** for cross-spec consistency review. This is the highest-value quality gate -- it catches issues that per-spec review gates cannot.
-
-**Sub-agent task**:
-
-Read ALL generated specs and check for consistency across the entire project:
-- `.kiro/specs/*/design.md` (primary: contains interfaces, data models, architecture)
-- `.kiro/specs/*/requirements.md` (for scope and acceptance criteria)
-- `.kiro/specs/*/tasks.md` (for boundary annotations only -- read _Boundary:_ lines, skip task descriptions)
-- `.kiro/steering/roadmap.md`
-
-Reading priority: Focus on design.md files (they contain interfaces, data models, architecture). For requirements.md, focus on section headings and acceptance criteria. For tasks.md, focus on _Boundary:_ annotations.
-
-Check:
-1. **Data model consistency**: Same entities defined consistently across specs (field names, types, relationships)
-2. **Interface alignment**: Where spec A outputs what spec B consumes, do contracts match exactly?
-3. **No duplicate functionality**: Any capability specified in more than one spec?
-4. **Dependency completeness**: Every design.md references correct upstream specs? Implicit dependencies not in roadmap?
-5. **Naming conventions**: Component names, file paths, API routes, table names consistent across specs?
-6. **Shared infrastructure**: Shared concerns (auth, error handling, logging) handled in one spec and correctly referenced?
-7. **Task boundary alignment**: Task _Boundary:_ annotations partition codebase cleanly? No files claimed by multiple specs?
-8. **Roadmap boundary continuity**: If roadmap includes `Existing Spec Updates` or `Direct Implementation Candidates`, do the generated new specs avoid absorbing that work by accident?
-9. **Architecture boundary integrity**: Do the specs preserve clean responsibility seams, avoid shared ownership, keep dependency direction coherent, and include enough revalidation triggers to catch downstream impact?
-10. **Change-friendly decomposition**: Has any spec absorbed multiple independent seams that should probably be split instead of kept together?
-
-Output: CONSISTENT areas + ISSUES with (which specs, what's inconsistent, suggested fix).
-
-**After the review sub-agent returns**:
-- **Critical/important issues found**: Dispatch fix sub-agents for each affected spec to apply the suggested fixes. If the issue is really a decomposition problem (for example boundary overlap or one spec carrying multiple independent seams), stop and return to roadmap/discovery instead of papering over it locally. Re-run cross-spec review after fixes (max 3 remediation rounds).
-- **Minor issues only**: Report them for user awareness, proceed to Step 5.
-- **No issues**: Proceed to Step 5.
-
-## Step 5: Finalize
-
-1. Scan `.kiro/specs/*/tasks.md` to verify all specs exist
-2. For each completed spec, read spec.json to confirm phase and approvals
-3. Update roadmap.md: mark completed specs as `[x]`
-4. If roadmap.md includes `Existing Spec Updates` or `Direct Implementation Candidates`, leave them untouched and mention them as remaining follow-up items unless already explicitly completed elsewhere
-
-Display final summary:
-```
-Spec Batch Complete:
-  ✓ app-foundation: X requirements, Y design components, Z tasks
-  ✓ block-editor: ...
-  ✓ page-management: ...
-  ...
-  Total: N specs created, M tasks generated
-  Cross-spec review: PASSED / N issues found (M fixed)
-  Existing spec updates pending: <count or none>
-  Direct implementation candidates pending: <count or none>
-
-Next: Review generated specs, then start implementation with /kiro-impl <feature>
-```
-
-</instructions>
-
-## Critical Constraints
-- **Controller stays lightweight**: Only read roadmap.md and brief.md existence checks in main context. All spec generation happens in sub-agents.
-- **Wave ordering is strict**: Never start a wave until all features in previous waves are complete.
-- **Parallel within waves**: All features in the same wave should be dispatched in parallel if multi-agent is available.
-- **No partial waves**: If a feature in a wave fails, still complete the other features in that wave before reporting.
-- **Skip completed specs**: Features with `[x]` in roadmap.md or existing tasks.md are skipped.
-- **`## Specs (dependency order)` remains authoritative for batch execution**: Other roadmap sections are context, not wave inputs.
-
-## Safety & Fallback
-
-**Sub-agent failure**:
-- Log the error, skip the failed feature
-- Continue with remaining features in the wave
-- Report failed features in the summary
-- Suggest: "Run `/kiro-spec-quick <feature> --auto` manually for failed features."
-
-**Circular dependencies**:
-- If dependency graph has cycles, report the cycle and stop
-- Suggest: "Fix dependency ordering in roadmap.md"
-
-**Roadmap not found**:
-- Stop and report: "No roadmap.md found. Run `/kiro-discovery` first."
-
-**All specs already complete**:
-- Report: "All specs in roadmap.md are already complete. Nothing to do."

.opencode/skills/kiro-spec-quick/SKILL.md

@@ -1,256 +0,0 @@
----
-name: kiro-spec-quick
-description: Quick spec generation with interactive or automatic mode
----
-
-# Quick Spec Generator
-
-<instructions>
-## CRITICAL: Automatic Mode Execution Rules
-
-**If `--auto` flag is present in `$ARGUMENTS`, you are in AUTOMATIC MODE.**
-
-In Automatic Mode:
-- Execute ALL 4 phases in a continuous loop without stopping
-- Display progress after each phase (e.g., "Phase 1/4 complete: spec initialized")
-- IGNORE any "Next Step" messages from Phase 2-4 (they are for standalone usage)
-- After Phase 4, run the final sanity review before exiting
-- Stop ONLY after the sanity review completes or if error occurs
-
----
-
-## Core Task
-Execute 4 spec phases sequentially. In automatic mode, execute all phases without stopping. In interactive mode, prompt user for approval between phases.
-
-Before claiming quick generation is complete, run one lightweight sanity review over the generated requirements, design, and tasks. If the host supports fresh sub-agents, use one. Otherwise run the sanity review inline.
-
-## Execution Steps
-
-### Step 1: Parse Arguments and Initialize
-
-Parse `$ARGUMENTS`:
-- If contains `--auto`: **Automatic Mode** (execute all 4 phases)
-- Otherwise: **Interactive Mode** (prompt at each phase)
-- Extract description (remove `--auto` flag if present)
-
-Example:
-```
-"User profile with avatar upload --auto" → mode=automatic, description="User profile with avatar upload"
-"User profile feature" → mode=interactive, description="User profile feature"
-```
-
-Display mode banner and proceed to Step 2.
-
-### Step 2: Execute Phase Loop
-
-Execute these 4 phases in order:
-
----
-
-#### Phase 1: Initialize Spec (Direct Implementation)
-
-**Core Logic**:
-
-1. **Check for Brief**:
-   - If `.kiro/specs/{feature-name}/brief.md` exists (created by `/kiro-discovery`), read it for discovery context (problem, approach, scope, constraints)
-   - Use brief content as the project description instead of `$ARGUMENTS`
-
-2. **Generate Feature Name**:
-   - Convert description to kebab-case
-   - Example: "User profile with avatar upload" → "user-profile-avatar-upload"
-   - Keep name concise (2-4 words ideally)
-
-3. **Check Uniqueness**:
-   - Use Glob to check `.kiro/specs/*/`
-   - If directory exists with only `brief.md` (no `spec.json`), use that directory (discovery created it)
-   - Otherwise if feature name exists, append `-2`, `-3`, etc.
-
-4. **Create Directory**:
-   - Use Bash: `mkdir -p .kiro/specs/{feature-name}` (skip if already exists from discovery)
-
-5. **Initialize Files from Templates**:
-
-   a. Read templates:
-   ```
-   - .kiro/settings/templates/specs/init.json
-   - .kiro/settings/templates/specs/requirements-init.md
-   ```
-
-   b. Replace placeholders:
-   ```
-   {{FEATURE_NAME}} → feature-name
-   {{TIMESTAMP}} → current ISO 8601 timestamp (use `date -u +"%Y-%m-%dT%H:%M:%SZ"`)
-   {{PROJECT_DESCRIPTION}} → description
-   en → language code (detect from user's input language, default to `en`)
-   ```
-
-   c. Write files using Write tool:
-   ```
-   - .kiro/specs/{feature-name}/spec.json
-   - .kiro/specs/{feature-name}/requirements.md
-   ```
-
-6. **Output Progress**: "Phase 1/4 complete: Spec initialized at .kiro/specs/{feature-name}/"
-
-**Automatic Mode**: IMMEDIATELY continue to Phase 2.
-
-**Interactive Mode**: Prompt "Continue to requirements generation? (yes/no)"
-- If "no": Stop, show current state
-- If "yes": Continue to Phase 2
-
----
-
-#### Phase 2: Generate Requirements
-
-Invoke `/kiro-spec-requirements {feature-name}`.
-
-Wait for completion. IGNORE any "Next Step" message (it is for standalone usage).
-
-**Output Progress**: "Phase 2/4 complete: Requirements generated"
-
-**Automatic Mode**: IMMEDIATELY continue to Phase 3.
-
-**Interactive Mode**: Prompt "Continue to design generation? (yes/no)"
-- If "no": Stop, show current state
-- If "yes": Continue to Phase 3
-
----
-
-#### Phase 3: Generate Design
-
-Invoke `/kiro-spec-design {feature-name} -y`. The `-y` flag auto-approves requirements.
-
-Wait for completion. IGNORE any "Next Step" message.
-
-**Output Progress**: "Phase 3/4 complete: Design generated"
-
-**Automatic Mode**: IMMEDIATELY continue to Phase 4.
-
-**Interactive Mode**: Prompt "Continue to tasks generation? (yes/no)"
-- If "no": Stop, show current state
-- If "yes": Continue to Phase 4
-
----
-
-#### Phase 4: Generate Tasks
-
-Invoke `/kiro-spec-tasks {feature-name} -y`. The `-y` flag auto-approves requirements, design, and tasks.
-
-Wait for completion.
-
-**Output Progress**: "Phase 4/4 complete: Tasks generated"
-
-#### Final Sanity Review
-
-After Phase 4, run a lightweight sanity review before claiming completion.
-
-- Review `requirements.md`, `design.md`, and `tasks.md` directly from disk. If `brief.md` exists, use it only as supporting context.
-- Prefer a fresh review sub-agent when the host supports it. Pass only file paths and the review objective; the reviewer should read the generated files itself.
-- Review focus:
-  - Do requirements, design, and tasks tell a coherent story?
-  - Are there obvious contradictions, missing prerequisites, or missing task coverage for required design work?
-  - Are `_Depends:_`, `_Boundary:_`, and `(P)` markers plausible for implementation?
-- If the review finds only task-plan-local issues, repair or update the generated `tasks.md` once, then re-run the sanity review.
-- If the review finds a real requirements/design gap or contradiction, stop and report follow-up instead of claiming the quick spec is implementation-ready.
-
-**All 4 phases plus sanity review complete.**
-
-Output final completion summary (see Output Description section) and exit.
-
----
-
-## Important Constraints
-
-### Error Handling
-- Any phase failure stops the workflow
-- Display error and current state
-- Suggest manual recovery command
-
-</instructions>
-
-## Output Description
-
-### Mode Banners
-
-**Interactive Mode**:
-```
-Quick Spec Generation (Interactive Mode)
-
-You will be prompted at each phase.
-Note: Skips gap analysis and design validation.
-```
-
-**Automatic Mode**:
-```
-Quick Spec Generation (Automatic Mode)
-
-All phases execute automatically without prompts.
-Note: Skips optional validations (gap analysis, design review) and user approval prompts. Internal review gates still run.
-Final sanity review still runs.
-```
-
-### Intermediate Output
-
-After each phase, show brief progress:
-```
-Spec initialized at .kiro/specs/{feature}/
-Requirements generated → Continuing to design...
-Design generated → Continuing to tasks...
-```
-
-### Final Completion Summary
-
-Provide output in the language specified in `spec.json`:
-
-```
-Quick Spec Generation Complete!
-
-## Generated Files:
-- .kiro/specs/{feature}/spec.json
-- .kiro/specs/{feature}/requirements.md ({X} requirements)
-- .kiro/specs/{feature}/design.md ({Y} components, {Z} endpoints)
-- .kiro/specs/{feature}/tasks.md ({N} tasks)
-
-Quick generation skipped:
-- `/kiro-validate-gap` - Gap analysis (integration check)
-- `/kiro-validate-design` - Design review (architecture validation)
-
-Sanity review: PASSED | FOLLOW-UP REQUIRED
-
-## Next Steps:
-1. Review generated specs (especially design.md)
-2. Optional validation:
-   - `/kiro-validate-gap {feature}` - Check integration with existing codebase
-   - `/kiro-validate-design {feature}` - Verify architecture quality
-3. Start implementation: `/kiro-impl {feature}`
-
-```
-
-## Safety & Fallback
-
-### Error Scenarios
-
-**Template Missing**:
-- Check `.kiro/settings/templates/specs/` exists
-- Report specific missing file
-- Exit with error
-
-**Directory Creation Failed**:
-- Check permissions
-- Report error with path
-- Exit with error
-
-**Phase Execution Failed** (Phase 2-4):
-- Stop workflow
-- Show current state and completed phases
-- Suggest: "Continue manually from `/kiro-spec-{next-phase} {feature}`"
-
-**Sanity Review Failed**:
-- Stop workflow
-- Report the exact contradiction, missing prerequisite, or task-plan issue
-- Suggest targeted follow-up with `/kiro-spec-design {feature}`, `/kiro-spec-tasks {feature}`, or manual edits depending on the finding
-
-**User Cancellation** (Interactive Mode):
-- Stop gracefully
-- Show completed phases
-- Suggest manual continuation

.opencode/skills/kiro-steering-custom/SKILL.md

@@ -1,145 +0,0 @@
----
-name: kiro-steering-custom
-description: Create custom steering documents for specialized project contexts
-metadata:
-  shared-rules: "steering-principles.md"
----
-
-
-# Kiro Custom Steering Creation
-
-<background_information>
-**Role**: Create specialized steering documents beyond core files (product, tech, structure).
-
-**Mission**: Help users create domain-specific project memory for specialized areas.
-
-**Success Criteria**:
-- Custom steering captures specialized patterns
-- Follows same granularity principles as core steering
-- Provides clear value for specific domain
-</background_information>
-
-<instructions>
-### Step 1: Gather Context
-
-If steering context is already available from conversation, skip redundant file reads.
-Otherwise:
-- Check `.kiro/settings/templates/steering-custom/` for available templates
-- Read `rules/steering-principles.md` from this skill's directory for steering principles
-
-## Workflow
-
-1. **Ask user** for custom steering needs:
-   - Domain/topic (e.g., "API standards", "testing approach")
-   - Specific requirements or patterns to document
-
-2. **Check if template exists**:
-   - Load from `.kiro/settings/templates/steering-custom/{name}.md` if available
-   - Use as starting point, customize based on project
-
-3. **Analyze codebase** (JIT) for relevant patterns:
-
-#### Parallel Research
-
-The following research areas are independent and can be executed in parallel:
-1. **Template & principles**: Load matching template and steering-principles.md
-2. **Domain patterns**: Analyze codebase for domain-specific patterns using Glob/Grep/Read
-
-If multi-agent is enabled, spawn sub-agents for each area above. Otherwise execute sequentially.
-
-After all parallel research completes, synthesize findings for steering document.
-
-4. **Generate custom steering**:
-   - Follow template structure if available
-   - Apply principles from `rules/steering-principles.md` from this skill's directory
-   - Focus on patterns, not exhaustive lists
-   - Keep to 100-200 lines (2-3 minute read)
-
-5. **Create file** in `.kiro/steering/{name}.md`
-
-## Available Templates
-
-Templates available in `.kiro/settings/templates/steering-custom/`:
-
-1. **api-standards.md** - REST/GraphQL conventions, error handling
-2. **testing.md** - Test organization, mocking, coverage
-3. **security.md** - Auth patterns, input validation, secrets
-4. **database.md** - Schema design, migrations, query patterns
-5. **error-handling.md** - Error types, logging, retry strategies
-6. **authentication.md** - Auth flows, permissions, session management
-7. **deployment.md** - CI/CD, environments, rollback procedures
-
-Load template when needed, customize for project.
-
-## Steering Principles
-
-From `rules/steering-principles.md` (in this skill's directory):
-
-- **Patterns over lists**: Document patterns, not every file/component
-- **Single domain**: One topic per file
-- **Concrete examples**: Show patterns with code
-- **Maintainable size**: 100-200 lines typical
-- **Security first**: Never include secrets or sensitive data
-
-</instructions>
-
-## Tool guidance
-
-- **Read**: Load template, analyze existing code
-- **Glob**: Find related files for pattern analysis
-- **Grep**: Search for specific patterns
-- **Bash** with `ls`: Understand relevant structure
-
-**JIT Strategy**: Load template only when creating that type of steering.
-
-## Output description
-
-Chat summary with file location (file created directly).
-
-```
-✅ Custom Steering Created
-
-## Created:
-- .kiro/steering/api-standards.md
-
-## Based On:
-- Template: api-standards.md
-- Analyzed: src/api/ directory patterns
-- Extracted: REST conventions, error format
-
-## Content:
-- Endpoint naming patterns
-- Request/response format
-- Error handling conventions
-- Authentication approach
-
-Review and customize as needed.
-```
-
-## Examples
-
-### Success: API Standards
-**Input**: "Create API standards steering"  
-**Action**: Load template, analyze src/api/, extract patterns  
-**Output**: api-standards.md with project-specific REST conventions
-
-### Success: Testing Strategy
-**Input**: "Document our testing approach"  
-**Action**: Load template, analyze test files, extract patterns  
-**Output**: testing.md with test organization and mocking strategies
-
-## Safety & Fallback
-
-- **No template**: Generate from scratch based on domain knowledge
-- **Security**: Never include secrets (load principles)
-- **Validation**: Ensure doesn't duplicate core steering content
-
-## Notes
-
-- Templates are starting points, customize for project
-- Follow same granularity principles as core steering
-- All steering files loaded as project memory
-- Custom files equally important as core files
-- Avoid documenting agent-specific tooling directories (e.g. `.cursor/`, `.gemini/`, `.claude/`)
-- Light references to `.kiro/specs/` and `.kiro/steering/` are acceptable; avoid other `.kiro/` directories
-

.opencode/skills/kiro-steering-custom/rules/steering-principles.md

@@ -1,90 +0,0 @@
-# Steering Principles
-
-Steering files are **project memory**, not exhaustive specifications.
-
----
-
-## Content Granularity
-
-### Golden Rule
-> "If new code follows existing patterns, steering shouldn't need updating."
-
-### ✅ Document
-- Organizational patterns (feature-first, layered)
-- Naming conventions (PascalCase rules)
-- Import strategies (absolute vs relative)
-- Architectural decisions (state management)
-- Technology standards (key frameworks)
-
-### ❌ Avoid
-- Complete file listings
-- Every component description
-- All dependencies
-- Implementation details
-- Agent-specific tooling directories (e.g. `.cursor/`, `.gemini/`, `.claude/`)
-- Detailed documentation of `.kiro/` metadata directories (settings, automation)
-
-### Example Comparison
-
-**Bad** (Specification-like):
-```markdown
-- /components/Button.tsx - Primary button with variants
-- /components/Input.tsx - Text input with validation
-- /components/Modal.tsx - Modal dialog
-... (50+ files)
-```
-
-**Good** (Project Memory):
-```markdown
-## UI Components (`/components/ui/`)
-Reusable, design-system aligned primitives
-- Named by function (Button, Input, Modal)
-- Export component + TypeScript interface
-- No business logic
-```
-
----
-
-## Security
-
-Never include:
-- API keys, passwords, credentials
-- Database URLs, internal IPs
-- Secrets or sensitive data
-
----
-
-## Quality Standards
-
-- **Single domain**: One topic per file
-- **Concrete examples**: Show patterns with code
-- **Explain rationale**: Why decisions were made
-- **Maintainable size**: 100-200 lines typical
-
----
-
-## Preservation (when updating)
-
-- Preserve user sections and custom examples
-- Additive by default (add, don't replace)
-- Add `updated_at` timestamp
-- Note why changes were made
-
----
-
-## Notes
-
-- Templates are starting points, customize as needed
-- Follow same granularity principles as core steering
-- All steering files loaded as project memory
-- Light references to `.kiro/specs/` and `.kiro/steering/` are acceptable; avoid other `.kiro/` directories
-- Custom files equally important as core files
-
----
-
-## File-Specific Focus
-
-- **product.md**: Purpose, value, business context (not exhaustive features)
-- **tech.md**: Key frameworks, standards, conventions (not all dependencies)
-- **structure.md**: Organization patterns, naming rules (not directory trees)
-- **Custom files**: Specialized patterns (API, testing, security, etc.)

.opencode/skills/kiro-validate-design/SKILL.md

@@ -1,103 +0,0 @@
----
-name: kiro-validate-design
-description: Interactive technical design quality review and validation
-metadata:
-  shared-rules: "design-review.md"
----
-
-
-# Technical Design Validation
-
-<background_information>
-- **Mission**: Conduct interactive quality review of technical design to ensure readiness for implementation
-- **Success Criteria**:
-  - Critical issues identified (maximum 3 most important concerns)
-  - Balanced assessment with strengths recognized
-  - Clear GO/NO-GO decision with rationale
-  - Actionable feedback for improvements if needed
-</background_information>
-
-<instructions>
-## Core Task
-Interactive design quality review for feature **$1** based on approved requirements and design document.
-
-## Execution Steps
-
-1. **Gather Context**:
-   - Read `.kiro/specs/$1/spec.json` for language and metadata
-   - Read `.kiro/specs/$1/requirements.md` for requirements
-   - Read `.kiro/specs/$1/design.md` for design document
-   - Core steering context: `product.md`, `tech.md`, `structure.md`
-   - Additional steering files only when directly relevant to architecture boundaries, integrations, runtime prerequisites, domain rules, security/performance constraints, or team conventions that affect implementation readiness
-   - Relevant local agent skills or playbooks only when they clearly match the feature's host environment or use case and provide review-relevant context
-
-#### Parallel Research
-
-The following research areas are independent and can be executed in parallel:
-1. **Context & rules loading**: Spec documents, core steering, task-relevant extra steering, relevant local agent skills/playbooks, and `rules/design-review.md` from this skill's directory for review criteria
-2. **Codebase pattern survey**: Gather existing architecture patterns, naming conventions, and component structure from the codebase to use as reference during review
-
-If multi-agent is enabled, spawn sub-agents for each area above. Otherwise execute sequentially.
-
-After all parallel research completes, synthesize findings for review.
-
-2. **Execute Design Review**:
-   - Reference conversation history when available: leverage prior requirements discussion and user's stated design intent
-   - Follow design-review.md process: Analysis → Critical Issues → Strengths → GO/NO-GO
-   - Limit to 3 most important concerns
-   - Engage interactively with user — ask clarifying questions, propose alternatives
-   - Use language specified in spec.json for output
-
-3. **Decision and Next Steps**:
-   - Clear GO/NO-GO decision with rationale
-   - Provide specific actionable next steps (see Next Phase below)
-
-## Important Constraints
-- **Quality assurance, not perfection seeking**: Accept acceptable risk
-- **Critical focus only**: Maximum 3 issues, only those significantly impacting success
-- **Conversation-aware**: Leverage discussion history for requirements context and user intent when available
-- **Interactive approach**: Engage in dialogue, ask clarifying questions, propose alternatives
-- **Balanced assessment**: Recognize both strengths and weaknesses
-- **Actionable feedback**: All suggestions must be implementable
-- **Context Discipline**: Start with core steering and expand only with review-relevant steering or use-case-aligned local agent skills/playbooks
-</instructions>
-
-## Tool Guidance
-- **Read first**: Load spec, core steering, relevant local playbooks/agent skills, and rules before review
-- **Grep if needed**: Search codebase for pattern validation or integration checks
-- **Interactive**: Engage with user throughout the review process
-
-## Output Description
-Provide output in the language specified in spec.json with:
-
-1. **Review Summary**: Brief overview (2-3 sentences) of design quality and readiness
-2. **Critical Issues**: Maximum 3, following design-review.md format
-3. **Design Strengths**: 1-2 positive aspects
-4. **Final Assessment**: GO/NO-GO decision with rationale and next steps
-
-**Format Requirements**:
-- Use Markdown headings for clarity
-- Follow design-review.md output format
-- Keep summary concise
-
-## Safety & Fallback
-
-### Error Scenarios
-- **Missing Design**: If design.md doesn't exist, stop with message: "Run `/kiro-spec-design $1` first to generate design document"
-- **Design Not Generated**: If design phase not marked as generated in spec.json, warn but proceed with review
-- **Empty Steering Directory**: Warn user that project context is missing and may affect review quality
-- **Language Undefined**: Default to English (`en`) if spec.json doesn't specify language
-
-### Next Phase: Task Generation
-
-**If Design Passes Validation (GO Decision)**:
-- Review feedback and apply changes if needed
-- Run `/kiro-spec-tasks $1` to generate implementation tasks
-- Or `/kiro-spec-tasks $1 -y` to auto-approve and proceed directly
-
-**If Design Needs Revision (NO-GO Decision)**:
-- Address critical issues identified
-- Re-run `/kiro-spec-design $1` with improvements
-- Re-validate with `/kiro-validate-design $1`
-
-**Note**: Design validation is recommended but optional. Quality review helps catch issues early.

.opencode/skills/kiro-validate-gap/SKILL.md

@@ -1,109 +0,0 @@
----
-name: kiro-validate-gap
-description: Analyze implementation gap between requirements and existing codebase
-metadata:
-  shared-rules: "gap-analysis.md"
----
-
-
-# Implementation Gap Validation
-
-<background_information>
-- **Mission**: Analyze the gap between requirements and existing codebase to inform implementation strategy
-- **Success Criteria**:
-  - Comprehensive understanding of existing codebase patterns and components
-  - Clear identification of missing capabilities and integration challenges
-  - Multiple viable implementation approaches evaluated
-  - Technical research needs identified for design phase
-</background_information>
-
-<instructions>
-## Core Task
-Analyze implementation gap for feature **$1** based on approved requirements and existing codebase.
-
-## Execution Steps
-
-1. **Load Context**:
-   - Read `.kiro/specs/$1/spec.json` for language and metadata
-   - Read `.kiro/specs/$1/requirements.md` for requirements
-   - Core steering context: `product.md`, `tech.md`, `structure.md`
-   - Additional steering files only when directly relevant to the feature's domain rules, integrations, runtime prerequisites, compliance/security constraints, or existing product boundaries
-   - Relevant local agent skills or playbooks only when they clearly match the feature's host environment or use case and provide analysis-relevant context
-
-2. **Read Analysis Guidelines**:
-   - Read `rules/gap-analysis.md` from this skill's directory for comprehensive analysis framework
-
-#### Parallel Research
-
-The following research areas are independent and can be executed in parallel:
-1. **Codebase analysis**: Existing implementations, architecture patterns, integration points, extension possibilities
-2. **External dependency research**: Dependency compatibility, version constraints, known integration challenges (when needed)
-3. **Context loading**: Requirements, core steering, task-relevant extra steering, relevant local agent skills/playbooks, and gap-analysis rules
-
-If multi-agent is enabled, spawn sub-agents for each area above. Otherwise execute sequentially.
-
-After all parallel research completes, synthesize findings for gap analysis.
-
-3. **Execute Gap Analysis**:
-   - Follow gap-analysis.md framework for thorough investigation
-   - Analyze existing codebase using Grep and Read tools
-   - Use WebSearch/WebFetch for external dependency research if needed
-   - Evaluate multiple implementation approaches (extend/new/hybrid)
-   - Use language specified in spec.json for output
-
-4. **Generate Analysis Document**:
-   - Create comprehensive gap analysis following the output guidelines in gap-analysis.md
-   - Present multiple viable options with trade-offs
-   - Flag areas requiring further research
-
-5. **Write Gap Analysis to Disk**:
-
-   **Write the gap analysis to disk so it survives session boundaries and can be referenced during design phase.**
-
-   - Save the gap analysis to `.kiro/specs/$1/research.md`
-   - If the file already exists, append the new analysis (separated by a horizontal rule `---`) rather than overwriting previous research
-   - Verify the file was written by reading it back
-
-## Important Constraints
-- **Information over Decisions**: Provide analysis and options, not final implementation choices
-- **Multiple Options**: Present viable alternatives when applicable
-- **Thorough Investigation**: Use tools to deeply understand existing codebase
-- **Explicit Gaps**: Clearly flag areas needing research or investigation
-- **Context Discipline**: Start with core steering and expand only with analysis-relevant steering or use-case-aligned local agent skills/playbooks
-</instructions>
-
-## Tool Guidance
-- **Read first**: Load spec, core steering, relevant local playbooks/agent skills, and rules before analysis
-- **Grep extensively**: Search codebase for patterns, conventions, and integration points
-- **WebSearch/WebFetch**: Research external dependencies and best practices when needed
-- **Write last**: Generate analysis only after complete investigation
-
-## Output Description
-Provide output in the language specified in spec.json with:
-
-1. **Analysis Summary**: Brief overview (3-5 bullets) of scope, challenges, and recommendations
-2. **Document Status**: Confirm analysis approach used
-3. **Next Steps**: Guide user on proceeding to design phase
-
-**Format Requirements**:
-- Use Markdown headings for clarity
-- Keep summary concise (under 300 words)
-- Detailed analysis follows gap-analysis.md output guidelines
-
-## Safety & Fallback
-
-### Error Scenarios
-- **Missing Requirements**: If requirements.md doesn't exist, stop with message: "Run `/kiro-spec-requirements $1` first to generate requirements"
-- **Requirements Not Approved**: If requirements not approved, warn user but proceed (gap analysis can inform requirement revisions)
-- **Empty Steering Directory**: Warn user that project context is missing and may affect analysis quality
-- **Complex Integration Unclear**: Flag for comprehensive research in design phase rather than blocking
-- **Language Undefined**: Default to English (`en`) if spec.json doesn't specify language
-
-### Next Phase: Design Generation
-
-**If Gap Analysis Complete**:
-- Review gap analysis insights
-- Run `/kiro-spec-design $1` to create technical design document
-- Or `/kiro-spec-design $1 -y` to auto-approve requirements and proceed directly
-
-**Note**: Gap analysis is optional but recommended for brownfield projects to inform design decisions.

.opencode/skills/openultrasast-fix-audit/SKILL.md

@@ -1,26 +0,0 @@
----
-name: openultrasast-fix-audit
-description: OpenUltraSAST patch, fix audit, differential review, adversarial review, and patch validation workflow. Use when proposing or auditing fixes for OpenUltraSAST findings from opencode.
----
-
-# OpenUltraSAST Fix Audit
-
-Use this skill to steer fixes for OpenUltraSAST findings.
-
-## Fix Lifecycle
-
-1. Intake: restate the finding ID, evidence level, affected path, and acceptance criteria.
-2. Plan: choose the smallest defensive change that addresses the root cause.
-3. Implement: modify only files required by the finding.
-4. Adversarial review: look for bypasses, regressions, false assumptions, and unrelated edits.
-5. Reconcile: either fix the review finding or document why it is not applicable.
-6. Verify: rerun the reproducer or affected checks.
-7. Ready: only then consider advancing to `patch_validated`.
-
-## Guardrails
-
-- Do not rewrite modules when a local fix is enough.
-- Do not change public APIs unless the vulnerability requires it.
-- Do not mark a patch validated from model reasoning alone.
-- Use differential review for risky changes in auth, crypto, config, parsers, and sandbox behavior.
-- Run Semgrep or CodeQL delta checks when those analyzers supplied the original evidence.

.opencode/skills/openultrasast-scan/SKILL.md

@@ -1,35 +0,0 @@
----
-name: openultrasast-scan
-description: OpenUltraSAST scan, index, quick, standard, deep, SARIF, and report workflows. Use when running or planning `ousast scan`, repository indexing, or evidence-gated SAST analysis from opencode.
----
-
-# OpenUltraSAST Scan
-
-Use this skill to steer OpenUltraSAST scan workflows from opencode.
-
-## Workflow
-
-1. Confirm the target is a local repository path.
-2. Run commands through uv, never by activating a virtualenv manually.
-3. Start with quick mode unless the task explicitly requires standard or deep analysis.
-4. Treat quick findings as `static_corroboration`, not verified vulnerabilities.
-5. Preserve run artifacts under `.openultrasast/runs/<scan-id>/`.
-6. Report `scan_id`, `run_dir`, finding count, and the strongest available evidence level.
-
-## Commands
-
-```bash
-uv run ousast scan <repo> --mode quick
-uv run ousast index <repo>
-uv run pytest
-```
-
-## Evidence Rules
-
-- `suspicion` means a model or weak signal raised a hypothesis.
-- `static_corroboration` means deterministic code evidence exists.
-- `crash_reproduced` requires an artifact-backed reproducer.
-- `root_cause_explained` requires independent verifier reasoning.
-- `patch_validated` requires rerunning relevant checks after the patch.
-
-Do not describe a finding as verified unless its evidence level meets the configured report threshold.

.opencode/skills/openultrasast-triage/SKILL.md

@@ -1,34 +0,0 @@
----
-name: openultrasast-triage
-description: OpenUltraSAST false-positive elimination, ranking calibration, evidence ladder, verifier triage, and finding adjudication. Use when reviewing findings, reducing false positives, or deciding whether evidence is enough.
----
-
-# OpenUltraSAST Triage
-
-Use this skill when triaging OpenUltraSAST findings from opencode.
-
-## Triage Loop
-
-1. Identify the finding, path, rule, severity, confidence, and evidence level.
-2. Separate deterministic evidence from model assumptions.
-3. Check attacker control, reachability, sanitizer behavior, duplicates, impact, and contradictory evidence.
-4. If rejected or unresolved, record the false-positive reason and scope.
-5. Prefer scoped demotion over deleting a vulnerability class globally.
-6. Feed the outcome into ranking calibration and prompt constraints.
-
-## False-Positive Reasons
-
-- `unreachable_path`
-- `missing_attacker_control`
-- `sanitizer_disproved`
-- `static_rule_mismatch`
-- `incorrect_model_assumption`
-- `duplicate`
-- `insufficient_impact`
-- `unsupported`
-- `contradicted`
-- `unverified`
-
-## Decision Standard
-
-If evidence is uncertain, mark the finding as requiring next evidence instead of escalating severity. Difficult or high-impact disagreements should trigger fusion/deeper reasoning when available.

AGENTS.md

@@ -28,17 +28,14 @@ Project memory keeps persistent guidance (steering, specs notes, component docs)
 - Think in English, generate responses in English. All Markdown content written to project files (e.g., requirements.md, design.md, tasks.md, research.md, validation reports) MUST be written in the target language configured for this specification (see spec.json.language).
 
 ## Minimal Workflow
-- Phase 0 (optional): `/kiro-steering`, `/kiro-steering-custom`
+- Phase 0 (optional): `/kiro-steering` (bootstrap, sync, or a custom steering document)
 - Discovery: `/kiro-discovery "idea"` — determines action path, writes brief.md + roadmap.md for multi-spec projects
-- Phase 1 (Specification):
+- Phase 1 (Specification), one spec at a time, each phase human-reviewed:
-  - Single spec: `/kiro-spec-quick {feature} [--auto]` or step by step:
   - `/kiro-spec-init "description"`
   - `/kiro-spec-requirements {feature}`
-    - `/kiro-validate-gap {feature}` (optional: for existing codebase)
+  - `/kiro-spec-design {feature} [-y | --gap | --review]` (`--gap`: optional gap analysis against the existing code first; `--review`: optional interactive review of an existing design; the design-review gate always runs)
-    - `/kiro-spec-design {feature} [-y]`
-    - `/kiro-validate-design {feature}` (optional: design review)
   - `/kiro-spec-tasks {feature} [-y]`
-  - Multi-spec: `/kiro-spec-batch` — creates all specs from roadmap.md in parallel by dependency wave
+  - Multi-spec projects: the specs of roadmap.md in dependency order, with the same steps each
 - Phase 2 (Implementation): `/kiro-impl {feature} [tasks]`
   - Without task numbers: autonomous mode (subagent per task + independent review + final validation)
   - With task numbers: manual mode (selected tasks in main context, still reviewer-gated before completion)
@@ -46,14 +43,17 @@ Project memory keeps persistent guidance (steering, specs notes, component docs)
 - Progress check: `/kiro-spec-status {feature}` (use anytime)
 
 ## Skills Structure
-Skills are located in `.opencode/skills/kiro-*/SKILL.md`
+Skills are located in `.agents/skills/*/SKILL.md` (the plural directory OpenCode searches by default; moved from `.opencode/skills` on 2026-10-02)
 - Each skill is a directory with a `SKILL.md` file
 - Use `/skills` to inspect currently available skills
 - Invoke a skill directly with `/kiro-<skill-name>`
 - **If there is even a 1% chance a skill applies to the current task, invoke it.** Do not skip skills because the task seems simple.
+- Workflow skills: `kiro-steering`, `kiro-discovery`, `kiro-spec-init`, `kiro-spec-requirements`, `kiro-spec-design`, `kiro-spec-tasks`, `kiro-impl`, `kiro-validate-impl`, `kiro-spec-status`
 - `kiro-review` — task-local adversarial review protocol used by reviewer subagents
-- `kiro-debug` — root-cause-first debug protocol used by debugger subagents
+- `kiro-debug` — root-cause-first debug protocol used by debugger subagents; carries the instrument rule below
-- `kiro-verify-completion` — fresh-evidence gate before success or completion claims
+- `kiro-verify-completion` — fresh-evidence gate before success or completion claims; carries the instrument rule below
+- Product skills for users of the tool (the only ones user documentation names): `openultrasast-scan`, `openultrasast-triage`
+- Reviewed 2026-10-02: `kiro-spec-quick`, `kiro-spec-batch` removed (no fast-track or parallel spec creation); `kiro-validate-gap` and `kiro-validate-design` merged into `kiro-spec-design` (`--gap`, `--review`); `kiro-steering-custom` merged into `kiro-steering`; `openultrasast-fix-audit` merged into `openultrasast-triage`
 
 ## Development Rules
 - 3-phase approval workflow: Requirements → Design → Tasks → Implementation
@@ -81,5 +81,5 @@ So, before reporting any measurement:
 
 ## Steering Configuration
 - Load entire `.kiro/steering/` as project memory
-- Default files: `product.md`, `tech.md`, `structure.md`
+- Core files in this repository: `overview.md` (current state wins over dated history), `roadmap.md`, `safety-net.md`
-- Custom files are supported (managed via `/kiro-steering-custom`)
+- Custom files may exist beside them and carry equal weight (created via `/kiro-steering`, templates under `.kiro/settings/templates/steering-custom/`)

Dockerfile

@@ -10,6 +10,8 @@
 # contributor a working tool from `docker compose run` with nothing installed on their machine.
 
 FROM eclipse-temurin:21-jre-noble
+LABEL org.opencontainers.image.licenses="Apache-2.0" \
+      org.opencontainers.image.source="https://github.com/norandom/OpenUltraSAST"
 
 ARG JOERN_VERSION=v4.0.625
 ARG ARCHIVE=joern-cli-linux-x86_64.zip
@@ -50,7 +52,7 @@ COPY benchmarks ./benchmarks
 # The core install is zero-dependency by design; `semantic` adds the tree-sitter grammars the candidate
 # enumerator uses. The LLM endpoint stays optional and is configured by environment, never baked in.
 RUN python3 -m venv /venv \
- && /venv/bin/pip install --no-cache-dir -e ".[semantic]"
+ && /venv/bin/pip install --no-cache-dir -e ".[semantic,s3]"
 ENV PATH="/venv/bin:${PATH}"
 
 # Analysis never needs root.

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,239 +1,326 @@
 # OpenUltraSAST
 
-**v2.0.0 (Python package `2.0.0`), state as of 2026-10-02.** An experimental static
+OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
-security analyser. The pre-push safety net is **NO-GO** for rollout: no hook capability is
+repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
-enabled and no independent population has passed the qualification gates (see
+keeps what it can prove apart from what it only suspects.
-[docs/evaluation.md](docs/evaluation.md)). See the [release notes](RELEASE_NOTES.md).
 
-OpenUltraSAST separates what it can prove from what it merely suspects, and says which is which.
+A scan combines up to four parts:
-A scan combines language-scoped pattern rules, a semantic overlay, a Joern-based taint engine
-and optional model calls; every finding carries the evidence that earned its place, and a scan
-that could not look at something records that as a degradation instead of reporting a clean
-result.
 
-`ousast` is the interface. Model-driven work over whole repositories runs on a separate
+- language-scoped pattern rules;
-agentic plane (google/ax), and a learned decision engine that weighs all the signals is in
+- a tree-sitter overlay (a syntax tree of each file, built by the tree-sitter parser);
-development; neither is needed for a scan.
+- a Joern code property graph (a graph of the program's syntax, control flow and data flow);
+- optionally, a model. It is asked only the question the graph cannot settle.
 
-## Install and quickstart
+Every finding carries the evidence that earned its place. When a scan could not look at
+something, it records a degradation. It does not report a clean result.
+
+`ousast` is the interface. Model work over whole repositories runs on a separate agentic plane
+(google/ax). A learned decision engine is in development. A scan needs neither of them.
+
+This file is the front door. Each section links to its page of the documentation site.
+The site is published at <https://norandom.github.io/OpenUltraSAST/>. Its sources are in `docs/` and
+`mkdocs.yml`; to preview it locally, run `uv sync --extra docs && uv run mkdocs serve`.
+
+## Status
+
+**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate.
+
+- No hook capability is enabled.
+- Two independent populations (case sets frozen before any scan) have been spent so far. Engine
+  recall was 1 of 11 and then 0 of 17.
+- The learned decision engine ranks but may not block. BLOCK is offered for no family.
+
+The project has stopped trying to reach the gates by hand. Every detection change now has to win
+a controlled measurement on held-out repositories. One page states all of this, with the record
+behind each figure: [docs/where-we-stand.md](docs/where-we-stand.md). The release history is in
+[RELEASE_NOTES.md](RELEASE_NOTES.md).
+
+## Install and first scan
+
+Pick the tier you need:
+
+1. Tier 1: a deterministic scan of a checkout, or a CI gate.
+2. Tier 2: the Joern engine (`standard`, `deep`).
+3. Tier 3, for maintainers: model work over whole repositories on the plane.
+4. Tier 4 does not exist yet.
+
+### Tier 1: quick scan, no Docker
+
+The core install depends only on PyYAML. From a checkout, `uv sync` (or `pip install .`) installs
+the `ousast` entry point. `uv sync --group dev` adds the test and lint tools.
 
 ```bash
-uv sync --group dev
+uv sync
-
-# Deterministic scan, no model calls
 uv run ousast scan /path/to/repo --mode quick
-
+uv run ousast scan . --mode quick --fail-on verified   # CI: exit 1 on any evidence-verified finding
-# CI: exit 1 on any evidence-verified finding
-uv run ousast scan . --mode quick --fail-on verified
 ```
 
-The core install depends only on PyYAML. Optional extras: `semantic` (tree-sitter grammars
+This runs the language-scoped pattern rules, entry-point reachability hints, ranking,
-for the overlay) and `s3` (the plane's memory store on an S3-compatible bucket, boto3).
+verification and scoring. It is deterministic and reproducible. It needs no network and no keys.
 
-For `standard` and `deep` scans, use the Docker image, which ships Joern (with `php-cli` for
+The same install gives you the advisory pre-push hook for any repository. It runs the quick rules
-the PHP frontend) next to the tool. Source the shell wrapper once and `ousast` runs in the
+on the lines a push changes, names every check it could not run, and never blocks a push:
-container with your directory mounted read-only:
+
+```bash
+cd /path/to/your/repository && ousast pre-push install    # remove: ousast pre-push uninstall
+```
+
+What it cannot do:
+
+- There is no engine. Nothing follows a value across statements or files.
+- The PHP quick rules are a low-precision, in-sample development set. Per-rule precision lower
+  bounds run from 0.03 to 1.0 on the real pins
+  (`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
+
+The optional `semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for
+the overlay.
+
+### Tier 2: standard and deep scans (Docker)
+
+`standard` and `deep` need Joern. The Docker image ships three things in one image: the tool,
+Joern v4.0.625 and `php-cli` (the PHP frontend needs it). Each release publishes it to GHCR. It is
+big: a JRE plus about 2 GB of Joern. Pull it instead of building, and tag it with the name the
+shell wrapper uses:
+
+```bash
+docker pull ghcr.io/norandom/openultrasast:2.0.1
+docker tag ghcr.io/norandom/openultrasast:2.0.1 openultrasast:dev   # or build: docker compose build
+```
+
+The shell wrapper runs `ousast` in the container. It builds the image only if `openultrasast:dev`
+is missing. Your directory is mounted read-only and the container network is off. For the pre-push
+hook, use `ousast pre-push install --docker`: Git cannot call a shell function, so it installs the
+script `ousast-docker` beside the hook.
 
 ```bash
 source /path/to/OpenUltraSAST/ops/shell/ousast.sh   # PowerShell: ops/shell/ousast.ps1
-ousast scan . --mode standard
+ousast scan . --mode standard                       # the graph decides; no model is asked
+ousast-with-judge scan . --mode standard            # network on, the model is asked (DEEPSEEK_API_KEY)
 ```
 
-Details, including a pinned host install of Joern, are in [ops/README.md](ops/README.md).
+`docker compose build` and `TARGET=/path/to/repo docker compose run --rm ousast scan /target` do
+the same without the wrapper.
 
-Every run writes auditable artifacts under `<target>/.openultrasast/runs/<scan-id>/`:
+`deep` adds REGRESS. REGRESS runs each reproduction snippet with `docker run` from wherever the
-`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`,
+scan runs. The engine container has no Docker socket. So run `deep` in one of two ways:
-`report.sarif` and `trace/events.jsonl`, plus the stage artifacts of the mode.
+
+- from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
+  verified) and a working `docker`;
+- or accept the recorded `sandbox_unavailable` degradation.
+
+Details, including the host install, are in [ops/README.md](ops/README.md).
+
+Keys and store settings live in `.env` in the working directory. The file is gitignored;
+`.env.example` lists the names.
+
+- `DEEPSEEK_API_KEY` serves every LLM call.
+- `OPENROUTER_API_KEY` is used for embeddings only (`ousast learn memory embed`).
+- `.env` never overrides a variable already exported in your shell. If a key change seems to have
+  no effect, `unset` the stale export.
+- The plane's memory store defaults to a local file store under `~/ousast-results/plane/memory`.
+- An S3-compatible bucket is optional: `uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>`
+  with `S3_ENDPOINT` and the AWS credentials in `.env`. Nothing in a scan needs it.
+
+### Tier 3: the plane on ax (maintainers)
+
+Tier 1 and 2 need none of this. Today's profile is a single-node kind cluster on the maintainer's
+host.
+
+- `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the PATH. It refuses to
+  start when one is missing.
+- It builds Agent Substrate and ax from their checkouts under `~/.cache/ousast/ax-src/`.
+- Measured idle footprint with two workers: the kind node takes 1.7 GiB of memory (27% of the
+  7.7 GiB host) and 4.9 GB of disk for images. The build uses several GB more for Go caches.
+- The worker pool is sized for that 7 GB host.
+- The verify and roles tasks need `DEEPSEEK_API_KEY`.
+- Memory lives in the local file store or the S3 store. RustFS is the tested server.
+
+Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
+
+### Tier 4: Kubernetes (k3s), planned
+
+Nothing has been deployed to any cluster other than that kind cluster. The target is two profiles
+of one code base: `kind` locally and `k3s` in production. The same `ousast plane run` would
+submit a Run in a remote execution mode. It needs:
+
+- a kubeconfig context or the cluster's ingress (and an `imagePullSecret` on the worker
+  ServiceAccount for a private package);
+- runner and engine images published to GHCR and pinned by digest;
+- the S3 store as the artifact path, with no receiver on a laptop.
+
+The plan, and the code changes it needs with file paths, are in
+[docs/deployment.md](docs/deployment.md#production-topology-ax-on-a-kubernetes-cluster).
 
 ## Scan modes
 
-| Mode | What runs | Needs |
+There are three modes. Each adds to the one before.
-| --- | --- | --- |
-| `quick` | Language-scoped pattern rules, entry-point reachability hints, ranking, verification, scoring. Deterministic and reproducible. | nothing |
-| `standard` | `quick` plus the MAP stage: complexity map, semantic overlay, authorization obligations, and the model layer, which builds one Joern code property graph per repository and decides taint, guard-dominance and configuration questions on it. An LLM is asked only the residual question the graph cannot settle (a `suspicion`); `[models] hunter` adds the tool hunter. | Joern for the model layer (else a `cpg_unavailable` degradation); a provider key for the LLM parts (else they are skipped and recorded) |
-| `deep` | The MAP stage of `standard` plus REGRESS: promoted candidates are loaded by a small snippet inside a Docker sandbox (no network, read-only source, non-root, memory/pid/time limits) and get a `triggerable` / `not_triggerable` / ... verdict. | a working `docker` (else a `sandbox_unavailable` degradation) |
 
-`--fail-on never|findings|verified|worth-fixing` sets the exit code: `findings` fails on any
+- `quick` runs the pattern rules, hints, ranking, verification and score. It needs nothing.
-reported finding, `verified` on any evidence-verified one, `worth-fixing` on a deep-mode
+- `standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and
-verdict that is worth fixing. `--config openultrasast.toml` loads settings.
+  the Joern model layer. The model layer builds one code property graph per repository. It decides
+  taint, guard-dominance and configuration questions on that graph. A model is asked only the
+  residual question. Without a key, that question is skipped and only the graph's entailed
+  findings are reported.
+- `deep` adds REGRESS. Promoted candidates are loaded by a small snippet in a Docker sandbox. Each
+  gets a `triggerable` or `not_triggerable` verdict.
 
-## Detection coverage (quick rules)
+A missing engine is a recorded degradation (`cpg_unavailable`, `sandbox_unavailable`), never a
+clean result.
 
-Rules run only against files of their language. The bundled ruleset lives in
+`--fail-on never|findings|verified|worth-fixing` sets the exit code. `--config openultrasast.toml`
-`src/openultrasast/ruleset/<language>/rules.toml`:
+loads settings. Every run writes these files under `<target>/.openultrasast/runs/<scan-id>/`:
+`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`, `report.sarif`
+and `trace/events.jsonl`.
 
-| Language | Classes (CWE) |
+`ousast pre-push` (experimental) analyses the commits a push would publish against their base.
-| --- | --- |
+First, the quick rules run on the changed lines. Then the engine compares head with base, and only
-| C / C++ | stack buffer overflow (121), format string (134), command injection (78) |
+new or worsened defects become candidates. It is advisory by default, and it prints one line per
-| Python | command injection (78), code injection (95), SSTI (94), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), insecure default (489) |
+check it could not run. Its capability registry is empty, so it emits no normal alert today. An
-| JavaScript / TS | command injection (78), code injection (95), XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) |
+engine finding the push introduced is shown as advisory. Install, settings, skip reasons and a CI
-| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
+recipe: [docs/scanning.md](docs/scanning.md#first-run-install-the-hook-on-any-repository).
-| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules |
 
-The detection gate (`uv run python -m openultrasast.gate`, also `tests/test_detection_benchmarks.py`)
+Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md).
-enforces the project goal of **at least 90% recall and under 10% false positives** per language
-on the bundled corpora in `benchmarks/manifests/`. Its output on this tree:
 
-| Language | Recall | False-positive rate |
+## How detection works
-| --- | --- | --- |
-| C / C++ | 93.3% (14/15) | 0.0% |
-| Python | 93.3% (14/15) | 0.0% |
-| JavaScript | 92.3% (12/13) | 0.0% |
-| Java | 100% (4/4) | 0.0% |
-| **Overall** | **93.6% (44/47)** | **0.0%** |
 
-These corpora are cheat-sheet fixtures modelled on deliberately vulnerable applications. They
+The tool uses three representations of code, from cheap to expensive:
-show that a rule change did not break a known detection; they are not a recall estimate on
-real code. Real-code results are in [docs/evaluation.md](docs/evaluation.md).
 
-**PHP is not in the gate, and its numbers are in-sample only.** The PHP rules were written and
+1. Text: regex quick rules, one statement at a time.
-tightened while looking at the development corpus they were measured on
+2. Tree: the tree-sitter CST (concrete syntax tree) and an IR (intermediate representation) per
-(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): 13 of 14 expected
+   function. The overlay and the obligations use it.
-findings on the `php-vulnerable` fixture with no false positive, 2 false positives on
+3. Graph: the Joern CPG. It is the only one that follows a value across statements, calls and
-`php-benign`, and per-rule precision lower bounds from 0.03 to 1.0 on the real pins. Treat
+   files.
-them as development numbers, not as a holdout result.
 
-**Framework knowledge is optional.** Rule and taint-fact entries that know a framework or
+Taint runs as fact tables, queries and verdicts on the graph. Absence bugs are operations that
-library (WordPress, Flask, Django, Express, Spring, ...) carry a `framework` or `library` tag
+should have been guarded. They are found as obligations, dominance and configuration census. A
-naming a row of `src/openultrasast/ruleset/frameworks.toml`. The loaders take `priors=`
+model answers only the residual question.
-(`"all"`, the default and today's behaviour; `"off"`, language-level knowledge only; or a set
-of ids), so their contribution can be measured and switched off.
 
-## Pre-push checks (experimental)
+- [docs/detection-techniques.md](docs/detection-techniques.md) explains each technique, with the
+  code it comes from and what it misses.
+- [docs/architecture.md](docs/architecture.md) gives the stage order and the evidence ladder (a
+  state machine, not a label a model may assign). It also covers the central CWE policy, the
+  project score and the self-improving loops (`ousast improve`, with `--memory` for proposals from
+  plane rows).
 
-`ousast pre-push` analyses the commits a `git push` would publish, or an explicit
+Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP. Each
-`--base`/`--head` pair, and compares the head against its base so that only new or worsened
+rule runs only against files of its language.
-defects are candidates. `--mode advisory` (the default) never blocks; `--mode blocking` and
-`--incomplete-coverage block` opt into enforcement. The artifact must be written outside the
-analysed repository.
 
-```bash
+The detection gate (`uv run python -m openultrasast.gate`) enforces at least 90% recall and under
-uv run ousast pre-push /path/to/repo --base BASE --head HEAD \
+10% false positives per language on the bundled cheat-sheet corpora. On this tree it reports
-  --artifact /outside/the/repo/result.json
+93.62% recall (44/47) and 0.0% false positives. Those corpora show that a rule change did not
-```
+break a known detection. They are not a recall estimate on real code, and PHP is not in the gate.
 
-The default capability registry is empty, so no normal alert is emitted today; every
+Framework knowledge (WordPress, Flask, Django, Express, Spring) is tagged as an optional prior in
-candidate stays diagnostic in the artifact. Installation into a repository's hooks
+`src/openultrasast/ruleset/frameworks.toml`. So its contribution can be measured and switched off.
-(`ops/install-pre-push`), caching and the optional witness model are documented in
-[ops/README.md](ops/README.md).
 
-## Model providers and keys
+## The plane on ax
 
-Keys live in `.env` in the working directory (gitignored, never committed):
+Model-driven work over whole repositories runs as a Run manifest. A Run is a DAG of ax Tasks. Each
+Task runs in its own gVisor actor (an isolated sandbox) on Agent Substrate. ax is the only agentic
+executor; there is no local subprocess path.
 
-```bash
+- Each Task binds at most one Model and its own `usd`/`calls` budget.
-DEEPSEEK_API_KEY=...      # every LLM call: model layer, tool hunter, plane tasks
+- Egress is deny-by-default per Task, by hostname only.
-OPENROUTER_API_KEY=...    # only OpenAI embeddings (openai/text-embedding-3-small)
+- The provider key travels only in the Task's start request, never in a manifest or an image.
-```
 
-The project uses DeepSeek (`deepseek-flash`) for LLM calls and OpenRouter only for embeddings.
+Measured on the first increment: the plane agreed on 16 of 20 declared sites at $0.0205 per
-Set `DEEPSEEK_API_KEY`: without it, chat calls fall back to OpenRouter when that key is set.
+candidate.
-
-**`.env` never overrides a variable already exported in your shell.** A stale
-`export DEEPSEEK_API_KEY=...` in your profile wins over the `.env` file silently; `unset` it
-when a key change in `.env` seems to have no effect.
-
-## Benchmarks and pair evaluation
-
-```bash
-# Score the scan against a ground-truth manifest
-uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
-
-# Vulnerable-vs-fixed pairs: fire on the vulnerable side, stay silent on the fix
-uv run ousast pairs
-uv run ousast pairs --slice sast       # OWASP Benchmark + Juliet
-uv run ousast pairs --slice vibe-py --pointers   # also fetch non-vendored pointer pairs (network)
-uv run python -m openultrasast.pair_gate         # CI: local pairs must all pass
-```
-
-Each benchmark run writes `.openultrasast/benchmarks/<id>/` (`benchmark_result.json`,
-`calibration_records.json` with every miss, `external_baseline_deltas.json`). The pair
-catalogs live in `benchmarks/pairs/`.
-
-## Self-improvement
-
-`ousast improve` runs a bounded loop over the governed ruleset: benchmark, propose edits to
-rule status and score constants only, validate, re-benchmark, and accept a round only if
-recall stays at least 90%, false positives under 10%, the project score does not regress and
-no matched finding is lost. Accepted rounds land in
-`<target>/.openultrasast/calibration/rule_policy.json`, which the next scan of that target loads.
-
-```bash
-uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run
-uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
-```
-
-With `--memory`, proposals also come from the rows plane runs stored (see below), through the
-same validator and gate; rows from holdout pairs, from the gated manifest's own cases and from
-any `--qualify-population` are dropped first. Details:
-[docs/architecture.md](docs/architecture.md#self-improvement).
-
-## The agentic plane (maintainers)
-
-Model work over whole repositories runs as a Run manifest on [google/ax](ops/ax/README.md)
-over Agent Substrate (a kind cluster on the maintainer's host), one isolated actor per task.
-Each task binds its own Model and its own `usd`/`calls` budget, egress is deny-by-default per
-task, and the provider key travels only in the task's start request.
 
 ```bash
 uv run ousast plane doctor                               # kind, Agent Substrate, ax controller, runner image
-uv run ousast plane run plane/runs/validation-46.yaml    # a rerun skips tasks already done
+uv run ousast plane run plane/runs/validation-46.yaml    # a rerun skips tasks already done; --workers bounds tasks in flight
-uv run ousast plane status validation-46                 # per-task status and token attribution
+uv run ousast plane status validation-46                 # per-task status and the token attribution table
 uv run ousast plane remember validation-46               # ingest the run's rows into the memory store
 ```
 
-`ousast plane workspaces`, `alerts-engine` and `harvest` generate Runs and inputs. Bring-up,
+`workspaces`, `harvest`, `alerts-engine` and `memory-normalise` generate Runs, their inputs and
-the task catalogue, the memory store (`OUSAST_MEMORY`) and the checklist for moving the plane
+the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md). What runs
-to another Kubernetes cluster are in [ops/ax/README.md](ops/ax/README.md).
+where today, and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
 
-> HarnessX, the earlier optional agentic extra, was retired 2026-09-30, and a leftover
+HarnessX, the earlier agentic extra, was retired 2026-09-30. A `[harnessx]` section is ignored
-> `[harnessx]` section (retired 2026-09-30) is ignored with one warning. `[models] verifier`,
+with one warning. `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail and
-> `[fusion] panel_model` and `[fusion] decider_model` fail with a message naming the plane
+name the plane replacement. Config keys that nothing ever read fail the same way. Fusion is
-> replacement. The LLM judge and the LLM fusion panels were retired with it; fusion is
+deterministic.
-> deterministic. See [RELEASE_NOTES.md](RELEASE_NOTES.md).
 
-## Decision engine (in development, not adopted)
+## Memory
 
-A learned decision engine is being built to replace hand-tuned detection edits: each
+One store holds what plane runs and the decision engine learned. It is keyed by repository and pin.
-instrument (quick rules, the Joern engine, model sink classification, verify passes, repository
+It holds two kinds of data:
-facts) produces signals, never verdicts, and a compiled AI classifier with local memory turns
-them into a calibrated probability per candidate with BLOCK and ADVISORY operating points.
-No scan, pre-push or report uses it yet. Status and the first measured slice:
-[docs/decision-engine.md](docs/decision-engine.md).
 
-## Agent integrations
+- rows: facts, verdicts, unit costs, alerts, coverage and feature records;
+- content-addressed blobs: excerpts, embeddings, cached responses and compiled programs.
 
-An agent that can run shell commands needs nothing but the CLI. For
+`OUSAST_MEMORY` selects the backend. It is either a local file store or `s3://<bucket>` on any
-[OpenCode](https://opencode.ai) there are optional project skills in `.opencode/skills/`:
+S3-compatible server. The S3 server must support versioning, lifecycle rules, object tags and S3
-`openultrasast-scan`, `openultrasast-triage` and `openultrasast-fix-audit`.
+Select. RustFS is the tested server.
 
-`ousast mcp` runs a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools:
+An admin sets the bucket up once. The store verifies it at every start. If anything is missing, it
-`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`,
+refuses to run and names each missing piece. It does not fall back.
-`benchmark`, `explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a
-shell, Docker or a free-form command.
 
-```jsonc
+- Layout and the read and write paths: [docs/memory.md](docs/memory.md).
-{ "command": "uv", "args": ["run", "ousast", "mcp"] }
+- The bucket policy and admin steps: [docs/rustfs.md](docs/rustfs.md).
+- How stored rows are meant to turn into better decisions:
+  [docs/memory-and-detection.md](docs/memory-and-detection.md).
+
+## Decision engine and evaluation
+
+A learned decision engine is being built. It is meant to replace hand-tuned detection edits. It
+follows three rules:
+
+- Every instrument produces signals, never verdicts.
+- Labels come from ground truth only.
+- A compiled AI classifier with local memory turns retrieved similar cases and the candidate's
+  signals into a calibrated probability. It has BLOCK and ADVISORY operating points.
+
+It is **in development, not adopted**. No scan, pre-push check or report uses it.
+
+It was measured on six families, on repositories the compiled program was not shown:
+
+- The pooled AUC (area under the ROC curve) of its score runs from 0.77 (injection) to 0.96
+  (access control).
+- The stricter within-pair AUC is 0.72 to 0.93 across the same families.
+- BLOCK is offered for no family
+  (`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
+
+Maintainer surface: `ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`.
+Design, slices and the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
+
+[docs/evaluation.md](docs/evaluation.md) explains how anything is measured on code the tool was not
+tuned on. It covers the pair corpora, the independent populations and the M4 gates. It also covers
+the fold discipline that keeps learned decisions out of their own training data. Population v3
+(PHP, 15 cases) is frozen as the one-time final check. Nothing in the tree may read it.
+
+```bash
+uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
+uv run ousast pairs --slice sast                     # fire on the vulnerable side, stay silent on the fix
+uv run python -m openultrasast.pair_gate             # CI: local pairs must all pass
+uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
 ```
 
-> The `kiro-*` skills, `AGENTS.md` and `.kiro/` are the maintainer's development tooling and
+## Token economics
-> can be ignored by users.
 
-## Further reading
+Model tokens are the one running cost. At runtime the plane routes them:
 
-Documentation site with flow diagrams (`docs/`, `mkdocs.yml`): `uv sync --extra docs && uv run mkdocs serve`.
+- one Model and one budget per Task;
+- a metered client that refuses the next call at the ceiling;
+- repository facts computed without a model and reused;
+- blobs replayed at $0, with no new model call;
+- triage before the hunt;
+- a tie-break only on disputed candidates.
 
-- [docs/architecture.md](docs/architecture.md): the scan pipeline, evidence ladder, CWE policy
+In development, 85.5% of the maintainers' coding-session tokens were tool input and output rather
-  and project score, and the self-improving loops.
+than reasoning. That is down from 90%. The remedies in `benchmarks/dev/` target that share.
-- [docs/evaluation.md](docs/evaluation.md): independent populations, the qualification gates,
+Mechanisms, the measured costs and the before-and-after table:
-  pair corpora and the plane increment results.
+[docs/token-ergonomics.md](docs/token-ergonomics.md).
-- [docs/decision-engine.md](docs/decision-engine.md): the learned decision engine.
-- [docs/threat-model.md](docs/threat-model.md): trust boundaries, sandbox and hardening.
-- [docs/token-ergonomics.md](docs/token-ergonomics.md): model spend at runtime (budgets, reuse,
-  measured costs) and in coding sessions.
-- [docs/examples.md](docs/examples.md): end-to-end walkthroughs.
 
-## Development
+## Examples, threat model
+
+- End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
+- What the tool trusts and what it does not: [docs/threat-model.md](docs/threat-model.md). This
+  covers scanned code as untrusted input, secret redaction, per-task egress, budgets and the
+  `deep` sandbox.
+
+## Contributing and maintainer tooling
 
 ```bash
 uv run pytest                 # tests, incl. the 90/10 detection gate
@@ -245,6 +332,112 @@ uv run python -m openultrasast.pair_gate  # local vuln-vs-fix pairs
 uv run python dagger/ci.py    # containerized CI pipeline
 ```
 
-Maintainer commands not covered above: `ousast repos` (pinned known-vulnerable checkouts),
+An agent that can run shell commands needs nothing but the CLI. For [OpenCode](https://opencode.ai)
-`ousast model candidates` (what the candidate enumerator can reach) and `ousast learn`
+there are two project skills in `.agents/skills/`:
-(the decision engine's data; see [docs/decision-engine.md](docs/decision-engine.md)).
+
+- `openultrasast-scan`: modes and their prerequisites, `--fail-on`, the evidence ladder, recorded
+  degradations and run artifacts.
+- `openultrasast-triage`: false-positive reasons, what an adjudication may feed, and fixing and
+  re-scanning.
+
+`ousast mcp` runs a narrow MCP (Model Context Protocol) server over stdio. It has ten tools:
+`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`,
+`explain`, `propose_patch` and `export_report`. No tool runs a shell, Docker or a free-form
+command.
+
+```jsonc
+{ "command": "uv", "args": ["run", "ousast", "mcp"] }
+```
+
+The `kiro-*` skills, `AGENTS.md` and the Kiro-style specifications under `.kiro/` are the
+maintainer's development tooling. Users can ignore them. Other maintainer commands:
+
+- `ousast repos`: pinned known-vulnerable checkouts.
+- `ousast model candidates`: what the candidate enumerator can reach.
+
+## Licence
+
+OpenUltraSAST is licensed under the [Apache License 2.0](LICENSE). Third-party material keeps its
+own terms:
+
+- Joern (Apache-2.0) is downloaded at image build time.
+- The tree-sitter wheels of the `semantic` extra are MIT.
+- The published benchmark cases were licence-checked one by one. v1's OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md).
+
+## Engine traces on kube-ax
+
+`benchmarks/learn/engine_trace.py --executor ax` submits AX v0.3.1 Tasks with
+`ax apply -f -` and removes them with `ax delete task NAME`. The operator supplies
+kubeconfig and AX access; the dispatcher creates no Kubernetes resources or secrets.
+The `ousast-engine-task` GHCR package **must be made Public once** in its package
+settings so the actors can pull it without registry credentials.
+
+The task image contains the current engine and worker code. Rebuild it after code
+changes and pass its digest with `--image`; no analyzer download is needed. The host
+materializes each pin as usual and uploads source and prepared questions under
+`engine-queue/<run>/<pin>/`. Only object-scoped presigned URLs and deadline parameters
+enter Task env. Credentials stay on the coordinator. Configure `OUSAST_MEMORY=s3://…`
+and the store credentials in the coordinator's `.env` or environment, with
+`S3_ENDPOINT=https://files.because-security.com`. Presigned links expire after the
+pin deadline plus ten minutes. IP and HTTP endpoints are rejected: the operator's
+actor EgressPolicy must allow **files.because-security.com:443**.
+
+Completion is the result object, even while AX reports Running. Failed phases and
+host deadlines produce explicit failure/timeout rows. Task and staging-object cleanup
+runs on success and failure; cleanup failure stops further dispatch. Per-pin files,
+`--rerun-status`, resume and `progress.json` retain the existing trace-runner behavior.
+AX results include `executor: "ax"`, `worker_ip`, and the image digest. Placement is
+random; `--parallel` defaults to two AX tasks and cannot exceed two.
+
+AX workers use `--heap-profile cluster` with
+`-Xms1700m -Xmx1700m -XX:MaxMetaspaceSize=256m -XX:ReservedCodeCacheSize=128m -XX:MaxDirectMemorySize=256m -Xss512k -XX:ActiveProcessorCount=1 -XX:+UseG1GC`
+on every frontend/query JVM; instruments record the flags. A `JVM out of memory`
+result is automatically re-queued to one local Docker lane with `--heap-profile vm`
+(2560m), recording `fallback: "vm-oom"` and the AX attempt. `mixed-ax` also uses this
+one Docker lane for overflow, prioritizing pending OOM fallbacks. Thus a mixed run has
+up to two AX tasks plus one Docker container. Use `--docker-image` for the locally
+available VM image; Docker never pulls it implicitly. A STOP file prevents new
+launches; interrupted fallback queues are recovered on resume.
+
+Coordinator commands, from this worktree (the AX binary and kubeconfig remain
+configurable). Build/push the actual uncommitted tree and read its digest:
+
+```bash
+cd /home/mc/Source/OpenUltraSAST/.claude/worktrees/engine-ax
+export PATH=/home/mc/Source/OpenUltraSAST/.venv/bin:$PATH
+export PYTHONPATH="$PWD/src"
+TASK_TAG="ghcr.io/norandom/ousast-engine-task:sha-$(git rev-parse --short=7 HEAD)"
+docker buildx build --platform linux/amd64 -f plane/Dockerfile.engine-task \
+  -t "$TASK_TAG" -t ghcr.io/norandom/ousast-engine-task:main \
+  --push --metadata-file /tmp/ousast-engine-task-build.json .
+export TASK_IMAGE="ghcr.io/norandom/ousast-engine-task@$(python -c 'import json; print(json.load(open("/tmp/ousast-engine-task-build.json"))["containerimage.digest"])')"
+printf '%s\n' "$TASK_IMAGE"
+docker build -f Dockerfile -t openultrasast:engine-ax .
+```
+
+The Engine image workflow also builds/pushes both images and prints each digest;
+CI needs the changes published before dispatch. The local build above includes
+uncommitted changes without publishing a git commit.
+
+With `KUBECONFIG` pointing to the operator's kube-ax config and store settings in
+`.env`, run a one-pin smoke using the default experiment units and local example
+memory (or add `--examples /path/to/frozen-examples.jsonl`):
+
+```bash
+: "${KUBECONFIG:?Set KUBECONFIG to the operator kube-ax configuration}"
+python benchmarks/learn/engine_trace.py --executor ax \
+  --ax-bin "$HOME/.local/ax-v0.3.1/ax" --kubeconfig "$KUBECONFIG" \
+  --atespace default --image "$TASK_IMAGE" --docker-image openultrasast:engine-ax \
+  --parallel 1 --limit 1 --deadline 900 \
+  --out "$HOME/ousast-results/plane/engine-ax-smoke"
+```
+
+Then run the mixed lanes (repeat to resume; defaults re-run failed/timeout pins):
+
+```bash
+python benchmarks/learn/engine_trace.py --executor mixed-ax \
+  --ax-bin "$HOME/.local/ax-v0.3.1/ax" --kubeconfig "$KUBECONFIG" \
+  --atespace default --image "$TASK_IMAGE" --docker-image openultrasast:engine-ax \
+  --parallel 2 --deadline 900 --rerun-status failed,timeout \
+  --out "$HOME/ousast-results/plane/engine-ax-mixed"
+```

RELEASE_NOTES.md

@@ -1,3 +1,41 @@
+# v2.0.1 (2026-10-02)
+
+v2.0.0 was tagged but never published: its release job stopped at the format check (see below). v2.0.1 is
+the first published 2.x release and carries everything listed under v2.0.0 plus the following.
+
+- **Licence: Apache-2.0.** `LICENSE` added; `pyproject.toml` declares `license = "Apache-2.0"`; both images
+  carry the `org.opencontainers.image.licenses` label.
+- **Container images on GHCR.** Each version tag pushes `ghcr.io/norandom/openultrasast` (the scan image:
+  Joern and php-cli) and `ghcr.io/norandom/openultrasast-runner` (the plane's task runner), tagged with the
+  version and, for full releases, `latest`. The digests are in the release job's summary; pin by digest
+  (Agent Substrate rejects tags).
+- **CI green again.** CI and the release job check formatting and lint over the whole repository; eight
+  benchmark scripts had drifted. They were reformatted (two `noqa` comments, one renamed loop variable; no
+  script computes anything differently), and the local gate now runs the same whole-tree commands.
+- **Front-door README.** Install in four tiers (quick scan without Docker; standard and deep with the scan
+  image; the plane on kind; Kubernetes on k3s, planned), every claim checked against the CLI, and links to
+  every documentation page.
+
+## Also in this release (previously listed as Unreleased)
+
+- **Documentation audited against v2.0.0** (no behaviour change). New `docs/deployment.md`: where the plane runs
+  today, this host's setup, and step-by-step guidance for a separate Kubernetes cluster with what is implemented
+  and what is planned; no cluster deployment has been made, and the kube context (`kind-$KIND_CLUSTER_NAME`) and
+  `doctor`'s registry check (`localhost:5001`) are still hard-coded. The decision-engine pages state slice 2
+  (six families, `benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`: pooled AUC 0.77-0.96,
+  within-pair 0.72-0.93, BLOCK offered nowhere, canary below 0.9 for three families) and that the engine is not
+  adopted. The model layer's residual question is documented at `model/pipeline.py` (`model/judge.py` was
+  deleted); a missing provider key is described as it behaves (no question asked, entailed findings reported),
+  not as a recorded degradation. The `.env` reference lists the memory-store variables; `ousast plane memory-normalise` is listed.
+- **Removed: the unused vector index, `ousast index` and `[embeddings]`** (legacy cleanup, 2026-10-02). `index.py`
+  keeps only the text chunker the skill router uses; the `VectorIndex` half had no caller, and nothing read the
+  `chunks.json` the subcommand wrote. `[embeddings] model`/`store` fail to load with the reason, like the other
+  retired keys; `OPENROUTER_EMBEDDING_MODEL` is no longer read (`ousast learn memory embed --model` names the model).
+- **Retired three config keys nothing read** (2026-10-02): `[sandbox] network`, `[sandbox] workspace_readonly` and
+  `[complexity] top_k` now fail to load naming why. The sandbox always ran with `--network none` and a read-only
+  workspace regardless of the first two; no stage read `top_k`. `memory_mb`, `timeout_seconds`, `pids_limit` and
+  `max_hunter_hotspots` are read and stay.
+
 # v2.0.0 (2026-10-02)
 
 A new major release: the agentic plane runs on google/ax, HarnessX is gone, detection decisions move

benchmarks/archive/README.md

@@ -16,7 +16,7 @@ maps each old path to the new one. The records themselves are not rewritten.
 | `model_sinks.py` | Stage-1 source-only model classification of sinks. Moved into the package as `plane/tasks/roles.py`. | `independent/selection-v2-model-sinks.json`, `measurements/2026-10-01-decision-engine-harvest/record.json`, both `2026-10-01-decision-engine-leak-audit` records | `05d5699` (named in `selection-v2-model-sinks.json`) |
 | `sink_candidates.py` | Diagnostic: vocabulary-declared sink candidates per case. Its declaration patterns live on in `plane/tasks/repo_facts.py`. | `independent/results-v2-model-pipeline.json` ("vocabulary sink-first: 2/17") | no commit recorded |
 | `hunt.py` | The v2 tool-hunter run over the population (protocol `hunter-protocol-v2.md`, including its `FAMILY_WORDS` table). Replaced by the plane's `verify` / `agree` tasks. | `independent/hunter-protocol-v2.md`, `independent/results-v2-hunter.json` (a label source in `learn/sources.toml`) | `6bfb05f`, the commit that introduced the protocol, which pins the runner |
-| `harnessx_removal_equality.py` | One-off equality proof for `harnessx-removal` (record and compare deterministic outputs). The spec is closed. `tests/test_harnessx_removal_equality.py` still checks that the comparison detects differences. | `measurements/2026-09-30-harnessx-removal-baseline/`, `measurements/2026-09-30-harnessx-removal-equality/` | baseline `ca5caf7`, candidate `b9afb66` |
+| `harnessx_removal_equality.py` | One-off equality proof for `harnessx-removal` (record and compare deterministic outputs). The spec is closed; its unit test was dropped on 2026-10-02 so the archived copy is reference only like the rest. | `measurements/2026-09-30-harnessx-removal-baseline/`, `measurements/2026-09-30-harnessx-removal-equality/` | baseline `ca5caf7`, candidate `b9afb66` |
 
 The six `independent` scripts still read the population, `evaluate.py` and the results from
 `benchmarks/independent/`. Their `HERE` points there, so outputs land where they always did.

benchmarks/archive/hunt.py

@@ -40,8 +40,34 @@ from openultrasast.preprocess import build_file_target, enumerate_source_files
 # The model names a family in its own words; the protocol scores three. Fixed before any v2 pin is hunted.
 FAMILY_WORDS = {
     "untrusted_destination": ("ssrf", "server-side request", "open redirect", "redirect", "url fetch", "request forgery"),
-    "config_secrets": ("cors", "debug", "secret", "hardcoded", "hard-coded", "credential", "misconfig", "configuration", "tls", "certificate"),
+    "config_secrets": (
-    "injection": ("inject", "sql", "sqli", "nosql", "command", "rce", "code execution", "eval", "template", "ssti", "xss", "deserializ", "ldap", "xpath"),
+        "cors",
+        "debug",
+        "secret",
+        "hardcoded",
+        "hard-coded",
+        "credential",
+        "misconfig",
+        "configuration",
+        "tls",
+        "certificate",
+    ),
+    "injection": (
+        "inject",
+        "sql",
+        "sqli",
+        "nosql",
+        "command",
+        "rce",
+        "code execution",
+        "eval",
+        "template",
+        "ssti",
+        "xss",
+        "deserializ",
+        "ldap",
+        "xpath",
+    ),
 }
 
 
@@ -182,7 +208,16 @@ def main() -> int:
             result["root"] = "/case"
             out.write_text(json.dumps(result, indent=2) + "\n")
             print(
-                json.dumps({"case": case["id"], "pin": label, "findings": len(result["findings"]), "hunts": f"{result['completed']}/{result['questions']}", "usd": result["cost_usd"], "total_usd": round(sum(spent), 4)}),
+                json.dumps(
+                    {
+                        "case": case["id"],
+                        "pin": label,
+                        "findings": len(result["findings"]),
+                        "hunts": f"{result['completed']}/{result['questions']}",
+                        "usd": result["cost_usd"],
+                        "total_usd": round(sum(spent), 4),
+                    }
+                ),
                 flush=True,
             )
     return 0

benchmarks/archive/model_sinks.py

@@ -37,7 +37,9 @@ from openultrasast.config import load_config  # noqa: E402
 from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL  # noqa: E402
 from openultrasast.preprocess import detect_language, enumerate_source_files  # noqa: E402
 
-NOT_PRODUCT = re.compile(r"(^|/)(tests?|__tests__|spec|specs|docs?|examples?|fixtures?|e2e|cypress|storybook)/|\.(test|spec|stories)\.[jt]sx?$|_test\.py$|(^|/)test_[^/]*\.py$|\.min\.js$|(^|/)dist/|(^|/)build/")
+NOT_PRODUCT = re.compile(
+    r"(^|/)(tests?|__tests__|spec|specs|docs?|examples?|fixtures?|e2e|cypress|storybook)/|\.(test|spec|stories)\.[jt]sx?$|_test\.py$|(^|/)test_[^/]*\.py$|\.min\.js$|(^|/)dist/|(^|/)build/"
+)
 OPERATIONS = {
     "injection": (
         "executes a database query (SQL, NoSQL, JCR/graph query), an operating-system command or shell, code or "
@@ -69,7 +71,9 @@ def product_files(root: Path, languages: set[str]) -> list[Path]:
 
 def chunks(text: str, size: int) -> list[str]:
     lines = text.splitlines()
-    return ["\n".join(f"{n + 1:5d}| {line}" for n, line in enumerate(lines[i : i + size], start=i)) for i in range(0, max(len(lines), 1), size)]
+    return [
+        "\n".join(f"{n + 1:5d}| {line}" for n, line in enumerate(lines[i : i + size], start=i)) for i in range(0, max(len(lines), 1), size)
+    ]
 
 
 class Meter:
@@ -85,7 +89,9 @@ class Meter:
             return sum(float(c.cost_usd()) for c in self.clients)
 
 
-def classify(root: Path, path: Path, family: str, args: argparse.Namespace, meter: Meter, local: threading.local) -> tuple[list[tuple[str, str, int]], int, int]:
+def classify(
+    root: Path, path: Path, family: str, args: argparse.Namespace, meter: Meter, local: threading.local
+) -> tuple[list[tuple[str, str, int]], int, int]:
     from openultrasast import tool_hunter
 
     if not hasattr(local, "client"):
@@ -99,7 +105,9 @@ def classify(root: Path, path: Path, family: str, args: argparse.Namespace, mete
         asked += 1
         prompt = PROMPT.format(operation=OPERATIONS[family], path=relative, code=code)
         try:
-            response = local.client.complete(model=args.model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True)
+            response = local.client.complete(
+                model=args.model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True
+            )
             items = json.loads(response.content or "{}").get("functions", [])
         except Exception:  # noqa: BLE001 -- counted as unclassified, never as "no sinks"
             failed += 1
@@ -138,7 +146,7 @@ def main() -> int:
             files = product_files(checkout, {case["language"], *case.get("also", [])})
             found, failed, asked = [], 0, 0
             with concurrent.futures.ThreadPoolExecutor(max_workers=args.workers) as pool:
-                for f, fail, ask in pool.map(lambda p: classify(checkout, p, case["family"], args, meter, local), files):
+                for f, fail, ask in pool.map(lambda p, c=checkout, fam=case["family"]: classify(c, p, fam, args, meter, local), files):
                     found += f
                     failed += fail
                     asked += ask

benchmarks/archive/score_batched.py

@@ -66,7 +66,8 @@ def main() -> int:
                 "agreed_findings": len(vulnerable["findings"]), "disputed_findings": len(vulnerable.get("disputed", [])),
                 "triaged_out": vulnerable.get("triaged_out"), "candidates": vulnerable.get("questions"),
                 "fixed_side_alerts": fixed_alerts, "fixed_agreed_total": len(fixed["findings"]) if fixed else None,
-                "removed_by_fix": fixed.get("removed_by_fix") if fixed else None, "usd": vulnerable.get("usd"), "usage": vulnerable.get("usage"),
+                "removed_by_fix": fixed.get("removed_by_fix") if fixed else None,
+                "usd": vulnerable.get("usd"), "usage": vulnerable.get("usage"),
             }
         )  # fmt: skip
         pool += [{"case": case["id"], **f} for f in vulnerable["findings"]]

benchmarks/archive/sink_candidates.py

@@ -36,7 +36,9 @@ from openultrasast.preprocess import detect_language, enumerate_source_files  #
 DECLARATION = {
     "python": re.compile(r"^\s*(?:async\s+)?def\s+(\w+)"),
     "php": re.compile(r"^\s*(?:(?:public|private|protected|static|final|abstract)\s+)*function\s+&?(\w+)"),
-    "javascript": re.compile(r"^\s*(?:export\s+)?(?:async\s+)?function\s*\*?\s*(\w+)|^\s*(?:(?:public|private|protected|static|async)\s+)*(\w+)\s*\([^;]*\)\s*(?::[^{=]+)?\{\s*$|^\s*(?:export\s+)?(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?(?:function|\()"),
+    "javascript": re.compile(
+        r"^\s*(?:export\s+)?(?:async\s+)?function\s*\*?\s*(\w+)|^\s*(?:(?:public|private|protected|static|async)\s+)*(\w+)\s*\([^;]*\)\s*(?::[^{=]+)?\{\s*$|^\s*(?:export\s+)?(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?(?:function|\()"
+    ),
 }
 DECLARATION["typescript"] = DECLARATION["javascript"]
 KEYWORDS = {"if", "for", "while", "switch", "catch", "return", "function", "else"}

benchmarks/archive/verify_batched.py

@@ -101,13 +101,22 @@ def triage(root: Path, path: str, candidates: list[list], family: str, model: st
     except OSError:
         return {}
     names = sorted({fn for _, fn, _ in candidates})
-    prompt = TRIAGE.format(operation=OPERATIONS[family], functions=json.dumps(names), path=path, code=_relevant_code(text, [ln for _, _, ln in candidates], chunk_lines))
+    prompt = TRIAGE.format(
+        operation=OPERATIONS[family],
+        functions=json.dumps(names),
+        path=path,
+        code=_relevant_code(text, [ln for _, _, ln in candidates], chunk_lines),
+    )
     response = client.complete(model=model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True)
     try:
         items = json.loads(response.content or "{}").get("functions", [])
     except (json.JSONDecodeError, AttributeError):
         return {}
-    return {str(i.get("function")): str(i.get("data_origin", "")).strip().lower() for i in items if isinstance(i, dict) and i.get("function") in names}
+    return {
+        str(i.get("function")): str(i.get("data_origin", "")).strip().lower()
+        for i in items
+        if isinstance(i, dict) and i.get("function") in names
+    }
 
 
 def hunt(root: Path, path: str, group: list[list], family: str, model: str, steps: int) -> tuple[list[dict], dict]:
@@ -120,8 +129,30 @@ def hunt(root: Path, path: str, group: list[list], family: str, model: str, step
         f"Operations to judge, all in `{path}`, each of which {OPERATIONS[family]}:\n{listing}\n"
         "The file is above. For each, trace where the operation's data comes from and whether it is guarded."
     )
-    spots = [Hotspot(path=path, function_name=fn, score=1.0, band="candidate", signals={}, rationale="model-classified sink", test_hint=None, inventory_finding_ids=()) for _, fn, _ in group]
+    spots = [
-    found = tool_hunter.run_tool_hunter(root, spots, client=recording, model=model, max_steps=steps, system_prompt=SYSTEM, user_prompt=prompt, context_files=[path], tags=("sink-verifier",))
+        Hotspot(
+            path=path,
+            function_name=fn,
+            score=1.0,
+            band="candidate",
+            signals={},
+            rationale="model-classified sink",
+            test_hint=None,
+            inventory_finding_ids=(),
+        )
+        for _, fn, _ in group
+    ]
+    found = tool_hunter.run_tool_hunter(
+        root,
+        spots,
+        client=recording,
+        model=model,
+        max_steps=steps,
+        system_prompt=SYSTEM,
+        user_prompt=prompt,
+        context_files=[path],
+        tags=("sink-verifier",),
+    )
     names = {fn for _, fn, _ in group}
     rows = []
     for f in found:
@@ -131,7 +162,16 @@ def hunt(root: Path, path: str, group: list[list], family: str, model: str, step
         # The verdict is about the candidate operation, so the site is the candidate's file and function; the
         # model's own location is kept beside it (it often names the file where the input enters instead).
         line = f.line if f.path == path else next(c[2] for c in group if c[1] == fn)
-        rows.append({"candidate": f"{path}::{fn}", "site": f"{path}:{line or 0}:{fn}", "reported_at": f"{f.path}:{f.line or 0}", "family": family, "title": f.title, "witness": f.rationale[:600]})
+        rows.append(
+            {
+                "candidate": f"{path}::{fn}",
+                "site": f"{path}:{line or 0}:{fn}",
+                "reported_at": f"{f.path}:{f.line or 0}",
+                "family": family,
+                "title": f.title,
+                "witness": f.rationale[:600],
+            }
+        )
     return rows, recording.usage()
 
 
@@ -143,7 +183,12 @@ def verify_file(root: str, path: str, candidates: list[list], family: str, args_
     try:
         if not args_["no_triage"]:
             record["triage"] = triage(Path(root), path, candidates, family, args_["model"], args_["chunk_lines"])
-        kept = [c for c in candidates if record["triage"].get(c[1], "unclassified") in VERIFY | {"unclassified"} or record["triage"].get(c[1]) not in {"internal", "constant"}]
+        kept = [
+            c
+            for c in candidates
+            if record["triage"].get(c[1], "unclassified") in VERIFY | {"unclassified"}
+            or record["triage"].get(c[1]) not in {"internal", "constant"}
+        ]
         record["kept"] = [c[1] for c in kept]
         usage: dict[str, int] = {}
         for _pass in range(args_["passes"]):
@@ -169,14 +214,16 @@ def agreement(record: dict) -> tuple[list[dict], list[dict]]:
     some = set.union(*(set(b) for b in by_pass))
     first = by_pass[0]
     pick = lambda c: next(b[c] for b in by_pass if c in b)  # noqa: E731
-    return [dict(first[c], passes=len(by_pass)) for c in sorted(every)], [dict(pick(c), passes=sum(c in b for b in by_pass)) for c in sorted(some - every)]
+    return [dict(first[c], passes=len(by_pass)) for c in sorted(every)], [
+        dict(pick(c), passes=sum(c in b for b in by_pass)) for c in sorted(some - every)
+    ]
 
 
 def main() -> int:
     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
     parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
     parser.add_argument("--only", default="")
-    parser.add_argument("--subset", type=Path, help='JSON {case_id: [[path, function], ...]}: verify only these candidates')
+    parser.add_argument("--subset", type=Path, help="JSON {case_id: [[path, function], ...]}: verify only these candidates")
     parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
     parser.add_argument("--max-steps", type=int, default=6)
     parser.add_argument("--per-hunt", type=int, default=6)
@@ -206,7 +253,11 @@ def main() -> int:
         if out.is_file():
             continue
         log = out_dir / f"{case['id']}--{label}.jsonl"
-        done = {r["path"]: r for r in (json.loads(row) for row in log.read_text().splitlines()) if "unverified" not in r} if log.is_file() else {}
+        done = (
+            {r["path"]: r for r in (json.loads(row) for row in log.read_text().splitlines()) if "unverified" not in r}
+            if log.is_file()
+            else {}
+        )
         spent += sum(float(r.get("usd", 0.0)) for r in done.values())
         record = json.loads((candidates_dir / f"{case['id']}.json").read_text())
         unique = list({(p, fn): [p, fn, ln] for p, fn, ln in sorted(record["candidates"])}.values())
@@ -265,19 +316,41 @@ def main() -> int:
             agreed += a
             disputed += d
         triaged_out = sum(len(r["candidates"]) - len(r.get("kept", [])) for r in done.values())
-        usage = {k: sum(int(r.get("usage", {}).get(k, 0)) for r in done.values()) for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens", "calls")}
+        usage = {
+            k: sum(int(r.get("usage", {}).get(k, 0)) for r in done.values())
+            for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens", "calls")
+        }
         complete = len(done) >= len(by_file) and not account_error
         result = {
-            "root": "/case", "families": [case["family"]], "questions": len(unique), "completed": sum(len(r["candidates"]) for r in done.values()),
+            "root": "/case", "families": [case["family"]],
+            "questions": len(unique), "completed": sum(len(r["candidates"]) for r in done.values()),
             "completed_regions": sorted(f"{c[0]}:{c[1]}" for r in done.values() for c in r["candidates"]),
-            "triaged_out": triaged_out, "passes": args.passes, "usage": usage, "usd": round(sum(float(r.get("usd", 0.0)) for r in done.values()), 4),
+            "triaged_out": triaged_out, "passes": args.passes, "usage": usage,
-            "seconds": round(time.monotonic() - started, 1), "findings": sorted(agreed, key=lambda f: f["site"]), "disputed": sorted(disputed, key=lambda f: f["site"]),
+            "usd": round(sum(float(r.get("usd", 0.0)) for r in done.values()), 4),
+            "seconds": round(time.monotonic() - started, 1),
+            "findings": sorted(agreed, key=lambda f: f["site"]),
+            "disputed": sorted(disputed, key=lambda f: f["site"]),
         }  # fmt: skip
         if args.fixed:
             result["removed_by_fix"] = removed
         if complete:
             out.write_text(json.dumps(result, indent=1) + "\n")
-        print(json.dumps({"case": case["id"], "files": f"{len(done)}/{len(by_file)}", "candidates": len(unique), "triaged_out": triaged_out, "agreed": len(agreed), "disputed": len(disputed), "usd": result["usd"], "total_usd": round(spent, 4), "complete": complete}), flush=True)
+        print(
+            json.dumps(
+                {
+                    "case": case["id"],
+                    "files": f"{len(done)}/{len(by_file)}",
+                    "candidates": len(unique),
+                    "triaged_out": triaged_out,
+                    "agreed": len(agreed),
+                    "disputed": len(disputed),
+                    "usd": result["usd"],
+                    "total_usd": round(spent, 4),
+                    "complete": complete,
+                }
+            ),
+            flush=True,
+        )
         if account_error:
             print(json.dumps({"stopped": "account", "error": account_error}), flush=True)
             return 2

benchmarks/archive/verify_sinks.py

@@ -104,7 +104,16 @@ def verify(root: str, candidate: list, family: str, model: str, max_steps: int,
     client = _client()
     before = float(client.cost_usd())
     path, function, line = candidate
-    spot = Hotspot(path=path, function_name=function, score=1.0, band="candidate", signals={}, rationale="model-classified sink", test_hint=None, inventory_finding_ids=())
+    spot = Hotspot(
+        path=path,
+        function_name=function,
+        score=1.0,
+        band="candidate",
+        signals={},
+        rationale="model-classified sink",
+        test_hint=None,
+        inventory_finding_ids=(),
+    )
     prompt = (
         f"Operation to judge: in `{path}`, function `{function}`, around line {line}, code that {OPERATIONS[family]}. "
         "The file is above. Trace where the operation's data comes from and whether it is guarded."
@@ -120,9 +129,19 @@ def verify(root: str, candidate: list, family: str, model: str, max_steps: int,
             second = recording.usage()
             usage = {k: usage.get(k, 0) + second.get(k, 0) for k in set(usage) | set(second)}
     except Exception as error:  # noqa: BLE001 -- recorded; an unverified candidate is not a clean one
-        return {"candidate": candidate, "unverified": f"{type(error).__name__}: {str(error)[:120]}", "usd": float(client.cost_usd()) - before}
+        return {
+            "candidate": candidate,
+            "unverified": f"{type(error).__name__}: {str(error)[:120]}",
+            "usd": float(client.cost_usd()) - before,
+        }
     findings = [
-        {"site": f"{f.path}:{f.line or line}:{f.function_name or function}", "family": family, "title": f.title, "witness": f.rationale[:600], "candidate": f"{path}::{function}"}
+        {
+            "site": f"{f.path}:{f.line or line}:{f.function_name or function}",
+            "family": family,
+            "title": f.title,
+            "witness": f.rationale[:600],
+            "candidate": f"{path}::{function}",
+        }
         for f in found
     ]
     return {"candidate": candidate, "findings": findings, "usd": float(client.cost_usd()) - before, "escalated": escalated, "usage": usage}
@@ -134,7 +153,12 @@ def main() -> int:
     parser.add_argument("--only", default="")
     parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
     parser.add_argument("--max-steps", type=int, default=6)
-    parser.add_argument("--escalate-steps", type=int, default=0, help="rerun a hunt that used every step without a verdict; 0 disables (measured: 88% escalate, +45% cost)")
+    parser.add_argument(
+        "--escalate-steps",
+        type=int,
+        default=0,
+        help="rerun a hunt that used every step without a verdict; 0 disables (measured: 88% escalate, +45% cost)",
+    )
     parser.add_argument("--workers", type=int, default=4)
     parser.add_argument("--budget-usd", type=float, default=40.0)
     parser.add_argument("--fixed", action="store_true", help="recheck detected sites at the fixed pin")
@@ -185,7 +209,9 @@ def main() -> int:
             with concurrent.futures.ProcessPoolExecutor(max_workers=args.workers) as pool, log.open("a") as sink:
                 futures = []
                 for candidate in pending:
-                    futures.append(pool.submit(verify, str(checkout), list(candidate), case["family"], args.model, args.max_steps, args.escalate_steps))
+                    futures.append(
+                        pool.submit(verify, str(checkout), list(candidate), case["family"], args.model, args.max_steps, args.escalate_steps)
+                    )
                 for future in concurrent.futures.as_completed(futures):
                     if future.cancelled():  # cancelled at the ceiling: never run, nothing to record
                         continue
@@ -219,9 +245,29 @@ def main() -> int:
         }  # fmt: skip
         if complete:
             out.write_text(json.dumps(result, indent=1) + "\n")
-        tokens = {k: sum(int(d.get("usage", {}).get(k, 0)) for d in done) for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens")}
+        tokens = {
+            k: sum(int(d.get("usage", {}).get(k, 0)) for d in done)
+            for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens")
+        }
         result["usage"] = tokens | {"escalated": sum(1 for d in done if d.get("escalated"))}
-        print(json.dumps({"case": case["id"], "pin": label, "candidates": len(todo), "verified": len(verified), "findings": len(findings), "unverified": len(unverified), "escalated": result["usage"]["escalated"], "cache_share": round(tokens["prompt_cache_hit_tokens"] / max(tokens["prompt_tokens"], 1), 2), "usd": result["usd"], "total_usd": round(spent, 4), "complete": complete}), flush=True)
+        print(
+            json.dumps(
+                {
+                    "case": case["id"],
+                    "pin": label,
+                    "candidates": len(todo),
+                    "verified": len(verified),
+                    "findings": len(findings),
+                    "unverified": len(unverified),
+                    "escalated": result["usage"]["escalated"],
+                    "cache_share": round(tokens["prompt_cache_hit_tokens"] / max(tokens["prompt_tokens"], 1), 2),
+                    "usd": result["usd"],
+                    "total_usd": round(spent, 4),
+                    "complete": complete,
+                }
+            ),
+            flush=True,
+        )
         if account_error:
             print(json.dumps({"stopped": "account", "error": account_error, "spent_usd": round(spent, 4)}), flush=True)
             return 2

benchmarks/ax/batch.py

@@ -0,0 +1,434 @@
+"""Host-only AX Task transport; credentials never enter task manifests."""
+
+from __future__ import annotations
+
+import hashlib
+import ipaddress
+import json
+import os
+import queue
+import re
+import shutil
+import subprocess
+import threading
+import time
+import uuid
+from collections import Counter, deque
+from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
+from dataclasses import dataclass, field
+from datetime import timedelta
+from pathlib import Path
+from urllib.parse import urlsplit
+
+import yaml
+
+from openultrasast.config import load_dotenv
+from openultrasast.plane.memory import PRESIGNED_ARCHIVE_SUFFIX, S3Store, open_store
+
+
+class EgressError(ValueError):
+    """Safe diagnostic without presigned URL contents."""
+
+
+class SandboxFailure(Exception):
+    """A terminal AX task without an authoritative result object."""
+
+
+def public_endpoint(url):
+    """Reject endpoints the actor's hostname-based HTTPS egress policy cannot reach."""
+    parts = urlsplit(url)
+    host = parts.hostname or ""
+    try:
+        ipaddress.ip_address(host)
+        is_ip = True
+    except ValueError:
+        is_ip = False
+    if (
+        parts.scheme != "https"
+        or not host
+        or is_ip
+        or "." not in host
+        or host.endswith((".localhost", ".local", ".internal"))
+        or parts.port not in (None, 443)
+        or parts.username
+        or parts.password
+    ):
+        raise EgressError(
+            "AX requires a public HTTPS DNS hostname in S3_ENDPOINT and presigned URLs: "
+            "the egress policy allows the store hostname on port 443 only (no IP or plain HTTP)"
+        )
+    return host.lower()
+
+
+def task_state(text, name):
+    """Accept AX YAML/JSON output and its human-readable tasks table."""
+    try:
+        doc = yaml.safe_load(text)
+    except yaml.YAMLError:
+        doc = None
+    rows = doc.get("items", doc.get("tasks", [doc])) if isinstance(doc, dict) else doc if isinstance(doc, list) else []
+    for row in rows:
+        if not isinstance(row, dict) or row.get("metadata", {}).get("name", row.get("name")) != name:
+            continue
+        status = row.get("status", {})
+        ip = next(
+            (status.get(key) or row.get(key) for key in ("workerIP", "workerIp", "worker_ip", "ip") if status.get(key) or row.get(key)),
+            None,
+        )
+        worker = status.get("worker")
+        if not ip and isinstance(worker, dict):
+            ip = worker.get("ip") or worker.get("address")
+        return str(status.get("phase", row.get("phase", ""))), ip
+    lines = [line.split() for line in text.splitlines() if line.strip()]
+    if lines:
+        headers = [re.sub(r"[^a-z]", "", cell.lower()) for cell in lines[0]]
+        for cells in lines[1:]:
+            if name not in cells:
+                continue
+            row = dict(zip(headers, cells, strict=False))
+            phase = row.get("phase", row.get("status", ""))
+            ip = row.get("workerip") or row.get("ip")
+            if not ip:
+                for cell in cells:
+                    try:
+                        ipaddress.ip_address(cell)
+                        ip = cell
+                        break
+                    except ValueError:
+                        pass
+            return phase, None if ip in {"<none>", "-", ""} else ip
+    listed = isinstance(doc, (dict, list)) or bool(lines and "name" in headers and {"phase", "status"}.intersection(headers))
+    return "Missing" if listed else "", None
+
+
+def bounded_read(read, timeout):
+    """Do not let SDK retries postpone task teardown beyond the pin deadline.
+
+    Only the read runs in a daemon thread; it cannot publish a late result to disk.
+    No executor shutdown waits for an SDK call whose socket has stopped responding.
+    """
+    if timeout <= 0:
+        raise TimeoutError("AX result deadline")
+    replies = queue.Queue(maxsize=1)
+
+    def fetch():
+        try:
+            replies.put((True, read()))
+        except Exception as exc:
+            replies.put((False, exc))
+
+    threading.Thread(target=fetch, daemon=True, name="engine-ax-result-read").start()
+    try:
+        ok, result = replies.get(timeout=timeout)
+    except queue.Empty:
+        raise TimeoutError("AX result deadline") from None
+    if not ok:
+        raise result
+    return result
+
+
+class AXLane:
+    def __init__(self, args, workload, store=None, pause=time.sleep, clock=time.monotonic):
+        self.workload = workload
+        self.args = args
+        self.pause, self.clock = pause, clock
+        self.run = uuid.uuid4().hex
+        load_dotenv()
+        self.host = public_endpoint(os.environ.get("S3_ENDPOINT", ""))
+        # Validate the image before opening a store, whose constructor probes the bucket.
+        validate_image(workload.image)
+        self.store = store if store is not None else open_store()
+        if store is None and not isinstance(self.store, S3Store):
+            raise ValueError("AX requires OUSAST_MEMORY=s3://bucket[/prefix]")
+
+    def __call__(self, item, output, attempt):
+        attempts = []
+        try:
+            done, worker = self._execute_attempt(item, output, attempt, attempts)
+        except SandboxFailure:
+            return {"status": "sandbox_failure", "ax_attempts": attempts}
+        if done is None or done.returncode:
+            return {"status": "instrument_failure", "ax_attempts": attempts}
+        record = json.loads((output / "result.json").read_text())
+        return {**record, "worker_ip": worker, "ax_attempts": attempts}
+
+    def ax(self, *args, manifest=None, timeout=30):
+        env = dict(os.environ)
+        if self.args.kubeconfig:
+            env["KUBECONFIG"] = str(Path(self.args.kubeconfig).expanduser())
+        done = subprocess.run(
+            [str(Path(self.args.ax_bin).expanduser()), *args, *(["--atespace", self.args.atespace] if args[0] != "apply" else [])],
+            input=json.dumps(manifest) if manifest is not None else None,
+            env=env,
+            capture_output=True,
+            text=True,
+            timeout=max(0.01, timeout),
+        )
+        if done.returncode:
+            # AX errors may echo manifest URLs; report only verb and exit code.
+            raise ValueError(f"ax {args[0]} failed (exit {done.returncode})")
+        return done
+
+    def _execute_attempt(self, item, output, attempt, attempts):
+        pin_id = item.id
+        digest = hashlib.sha256(f"{self.run}/{pin_id}/attempt-{attempt}".encode()).hexdigest()[:12]
+        slug = re.sub(r"[^a-z0-9]+", "-", str(pin_id).lower()).strip("-")[:22]
+        name = f"ousast-engine-{slug or 'pin'}-{digest}"
+        key_id = hashlib.sha256(str(pin_id).encode()).hexdigest()
+        prefix = f"engine-queue/{self.run}/{key_id}/attempt-{attempt}/"
+        keys = [prefix + item.object_names.get(key, key.lower() + PRESIGNED_ARCHIVE_SUFFIX) for key in item.inputs]
+        keys.append(prefix + item.result_object)
+        expires = timedelta(seconds=self.workload.deadline + 600)
+        worker_ip = None
+        submitted = False
+        started = self.clock()
+        phase = ""
+        try:
+            for key, data in zip(keys, item.inputs.values(), strict=False):
+                self.store._put(key, data)
+            urls = {env: self.store.presign_get(key, expires) for env, key in zip(item.inputs, keys, strict=False)}
+            urls["RESULT_URL"] = self.store.presign_put(keys[-1], expires)
+            if any(public_endpoint(url) != self.host for url in urls.values()):
+                raise EgressError("AX presign hostname must match S3_ENDPOINT for the store egress policy")
+            document = manifest(name, self.workload, item, urls, self.args.atespace)
+            end = self.clock() + self.workload.deadline
+            # An apply timeout can still have created the task; cleanup must cover it.
+            submitted = True
+            self.ax("apply", "-f", "-", manifest=document, timeout=min(30, self.workload.deadline))
+            while self.clock() < end:
+                result = bounded_read(lambda: self.store._get(keys[-1]), end - self.clock())
+                if self.clock() >= end:
+                    return None, worker_ip
+                phase = ""
+                try:
+                    state = self.ax("get", "tasks", timeout=min(30, end - self.clock())).stdout
+                    phase, ip = task_state(state, name)
+                    worker_ip = ip or worker_ip
+                except (ValueError, OSError, subprocess.SubprocessError):
+                    # The object is authoritative even when AX status is temporarily unavailable.
+                    pass
+                if self.clock() >= end:
+                    return None, worker_ip
+                if result is not None:
+                    self.workload.validate(result[0], output)
+                    return subprocess.CompletedProcess([], 0, "", ""), worker_ip
+                if phase.lower() in {"failed", "missing"}:
+                    # The worker may have uploaded while we were reading AX status.
+                    result = bounded_read(lambda: self.store._get(keys[-1]), end - self.clock())
+                    if self.clock() >= end:
+                        return None, worker_ip
+                    if result is not None:
+                        self.workload.validate(result[0], output)
+                        return subprocess.CompletedProcess([], 0, "", ""), worker_ip
+                    raise SandboxFailure
+                self.pause(min(2, max(0, end - self.clock())))
+            return None, worker_ip
+        except (subprocess.TimeoutExpired, TimeoutError):
+            return None, worker_ip
+        except SandboxFailure:
+            raise
+        except Exception as exc:
+            # Never expose exception strings which may contain object-scoped bearer URLs.
+            detail = str(exc) if isinstance(exc, EgressError) else type(exc).__name__
+            return subprocess.CompletedProcess([], 1, "", "AX transport failed: " + detail), worker_ip
+        finally:
+            attempts.append({"name": name, "phase": phase, "seconds": self.clock() - started})
+            errors = []
+            if submitted:
+                try:
+                    self.ax("delete", "task", name)
+                except Exception as exc:
+                    # A successful list already confirmed a disappeared task is gone.
+                    if phase != "Missing":
+                        errors.append("task: " + type(exc).__name__)
+            for key in keys:
+                try:
+                    self.store._delete(key)
+                except Exception as exc:
+                    errors.append("object: " + type(exc).__name__)
+            if errors:
+                raise RuntimeError("AX cleanup failed; stop dispatching: " + ", ".join(errors))
+
+
+@dataclass
+class Item:
+    id: str
+    inputs: dict[str, bytes]
+    result_object: str
+    command: tuple[str, ...]
+    extra_env: dict[str, str]
+    input_digest: str
+    object_names: dict[str, str] = field(default_factory=dict)
+
+
+@dataclass(frozen=True)
+class Workload:
+    image: str
+    url_env: frozenset[str]
+    deadline: float
+    validate: object
+
+
+def validate_image(image):
+    if not re.fullmatch(r"[^\s@]+@sha256:[0-9a-f]{64}", image):
+        raise ValueError("AX --image must be pinned by sha256 digest")
+
+
+def manifest(name, workload, item, urls, atespace="default", *, validate_name=True):
+    validate_image(workload.image)
+    if validate_name and (not re.fullmatch(r"ousast-engine-[a-z0-9-]+", name) or len(name.encode()) > 49):
+        raise ValueError("invalid AX task name")
+    if set(urls) != workload.url_env:
+        raise ValueError("unexpected URL environment names")
+    # Extra env is deliberately a small numeric execution contract, never inherited credentials.
+    if set(item.extra_env) - {"DEADLINE", "QUESTION_DEADLINE"} or any(
+        not re.fullmatch(r"[0-9]+(?:\.[0-9]+)?", str(v)) for v in item.extra_env.values()
+    ):
+        raise ValueError("unexpected task environment")
+    for url in urls.values():
+        public_endpoint(url)
+    env = [{"name": key, "value": str(value)} for key, value in {**urls, **item.extra_env}.items()]
+    if len(json.dumps(env).encode()) >= 32768:
+        raise ValueError("AX Task env exceeds the 32 KB limit")
+    return {
+        "apiVersion": "ax.io/v1alpha1",
+        "kind": "Task",
+        "metadata": {"name": name, "atespace": atespace},
+        "spec": {"image": workload.image, "env": env, "command": list(item.command)},
+    }
+
+
+def write_json(path, record):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(".tmp")
+    temporary.write_text(json.dumps(record, sort_keys=True) + "\n")
+    temporary.replace(path)
+
+
+def schedule(pending, lanes, out, execute, complete, *, fallbacks=(), overflow=True, progress=None):
+    """The sole progress writer; callbacks carry workload-specific records, never transport."""
+    if sum(lane in {"ax", "kind"} for lane in lanes) > 2 or lanes.count("docker") > 1:
+        raise ValueError("at most two AX lanes and one VM lane")
+    pending, fallbacks = deque(pending), deque(fallbacks)
+    progress = progress if progress is not None else {"pins_done": 0, "pins_total": len(pending) + len(fallbacks)}
+    progress["lanes"] = dict(Counter(lanes))
+    active = {}
+    failures = False
+
+    def persist():
+        progress.update(pins_in_flight=len(active), active_lanes=dict(Counter(lane for _, lane in active.values())))
+        if any(lane in {"ax", "kind"} for lane in lanes):
+            progress["fallbacks_pending"] = len(fallbacks)
+        write_json(out / "progress.json", progress)
+
+    with ThreadPoolExecutor(max_workers=len(lanes)) as pool:
+
+        def launch(lane):
+            if (out / "STOP").exists():
+                return
+            if lane == "docker" and fallbacks:
+                item, previous = fallbacks.popleft()
+            elif pending and (lane != "docker" or overflow):
+                item, previous = pending.popleft(), None
+            else:
+                return
+            active[pool.submit(execute, item, lane, previous)] = item, lane
+
+        for lane in lanes:
+            launch(lane)
+        persist()
+        try:
+            while active:
+                finished, _ = wait(active, return_when=FIRST_COMPLETED)
+                # Resolve ALL completed futures before launching; cleanup exceptions stop launches.
+                results = [(active.pop(future), future.result()) for future in finished]
+                for (item, lane), result in results:
+                    failed, fallback = complete(item, lane, result, progress)
+                    if fallback is not None:
+                        fallbacks.append((item, fallback))
+                    else:
+                        failures |= failed
+                        progress["pins_done"] += 1
+                for lane, count in (Counter(lanes) - Counter(lane for _, lane in active.values())).items():
+                    for _ in range(count):
+                        launch(lane)
+                persist()
+        finally:
+            persist()
+    return failures
+
+
+def dispatch(items, workload, out, executors, *, lanes=("ax", "ax", "docker"), overflow=True):
+    """Digest checkpoints, one instrument rerun, sandbox retry, priority VM fallback."""
+    out.mkdir(parents=True, exist_ok=True)
+    pending, fallbacks = [], []
+
+    def path(item):
+        return out / (hashlib.sha256(item.id.encode()).hexdigest() + ".json")
+
+    for item in items:
+        previous = json.loads(path(item).read_text()) if path(item).exists() else {}
+        if previous.get("input_digest") == item.input_digest:
+            if previous.get("fallback_pending"):
+                fallbacks.append((item, previous))
+                continue
+            if previous.get("done"):
+                continue
+        pending.append(item)
+
+    def execute(item, lane, previous):
+        output = out / hashlib.sha256(item.id.encode()).hexdigest()
+        output.mkdir(exist_ok=True)
+        attempts = []
+        for attempt in range(2):
+            attempt_output = output / ("attempt-" + uuid.uuid4().hex)
+            attempt_output.mkdir()
+            record = executors[lane](item, attempt_output, attempt)
+            for artifact in attempt_output.iterdir():
+                if artifact.is_file():
+                    shutil.copyfile(artifact, output / artifact.name)
+            attempts.append({"lane": lane, "status": record.get("status"), "attempt": attempt, "transport": record.get("ax_attempts", [])})
+            if record.get("status") not in {"sandbox_failure", "instrument_failure"}:
+                break
+        return {**record, "attempts": (previous or {}).get("attempts", []) + attempts, "input_digest": item.input_digest, "done": True}
+
+    def complete(item, lane, record, progress):
+        fallback = lane in {"ax", "kind"} and record.get("status") in {"sandbox_failure", "oom"} and "docker" in lanes
+        record["fallback_pending"] = fallback
+        write_json(path(item), record)
+        progress["last_item"] = hashlib.sha256(item.id.encode()).hexdigest()
+        return record.get("status") not in {"ok", "unanalysable", "quick_only", "engine_covered"}, record if fallback else None
+
+    return schedule(pending, lanes, out, execute, complete, fallbacks=fallbacks, overflow=overflow)
+
+
+class DockerLane(AXLane):
+    """VM lane uses the same presigned-object protocol and pinned replay image."""
+
+    def _execute_attempt(self, item, output, attempt, attempts):
+        # Reuse all object/validation/cleanup logic, replacing only the task runner.
+        import copy
+
+        lane = copy.copy(self)
+        lane.ax = lane._docker_command
+        lane._docker_name = None
+        return AXLane._execute_attempt(lane, item, output, attempt, attempts)
+
+    def _docker_command(self, *args, manifest=None, timeout=30):
+        if args[0] == "apply":
+            self._docker_name = manifest["metadata"]["name"]
+            command = ["docker", "run", "-d", "--name", self._docker_name, "--entrypoint", manifest["spec"]["command"][0]]
+            for env in manifest["spec"]["env"]:
+                command += ["-e", env["name"] + "=" + env["value"]]
+            command += [manifest["spec"]["image"], *manifest["spec"]["command"][1:], "--heap-profile", "vm"]
+        elif args[0] == "get":
+            command = ["docker", "inspect", "--format", "{{.State.Running}} {{.State.ExitCode}}", self._docker_name]
+        else:
+            command = ["docker", "rm", "-f", self._docker_name or args[2]]
+        done = subprocess.run(command, capture_output=True, text=True, timeout=timeout)
+        if done.returncode:
+            raise ValueError("docker command failed")
+        if args[0] == "get":
+            phase = "Running" if done.stdout.startswith("true") else "Failed"
+            done.stdout = json.dumps({"metadata": {"name": self._docker_name}, "status": {"phase": phase}})
+        return done

benchmarks/dev/bulk_read.py

@@ -36,9 +36,9 @@ def main() -> int:
     parser.add_argument("files", nargs="+", type=Path)
     parser.add_argument("--max-usd", type=float, default=0.05)
     args = parser.parse_args()
+    from openultrasast import tool_hunter
     from openultrasast.config import load_config
     from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL
-    from openultrasast import tool_hunter
 
     load_config()
     client = tool_hunter.resolve_hunter_client()
@@ -51,7 +51,12 @@ def main() -> int:
         if float(client.cost_usd()) >= args.max_usd:
             answers.append(f"[stopped at the ${args.max_usd} ceiling before: {chunk.splitlines()[0]}]")
             break
-        response = client.complete(model=DEFAULT_DETECTOR_MODEL, messages=[{"role": "user", "content": PROMPT.format(question=args.question, files=chunk)}], tools=[], timeout_seconds=90)
+        response = client.complete(
+            model=DEFAULT_DETECTOR_MODEL,
+            messages=[{"role": "user", "content": PROMPT.format(question=args.question, files=chunk)}],
+            tools=[],
+            timeout_seconds=90,
+        )
         answers.append((response.content or "").strip())
     print("\n\n".join(a for a in answers if a))
     print(f"\n[bulk_read: {len(chunks)} chunk(s), ${float(client.cost_usd()):.4f}]", file=sys.stderr)

benchmarks/dev/read_guard.py

@@ -37,7 +37,11 @@ def main() -> int:
         f"{path} has {lines} lines (limit {LIMIT} for a whole-file read). Read a range with offset/limit, or ask "
         f"the bulk reader for what you need: .venv/bin/python benchmarks/dev/bulk_read.py '<question>' {path}"
     )
-    print(json.dumps({"hookSpecificOutput": {"hookEventName": "PreToolUse", "permissionDecision": "deny", "permissionDecisionReason": reason}}))
+    print(
+        json.dumps(
+            {"hookSpecificOutput": {"hookEventName": "PreToolUse", "permissionDecision": "deny", "permissionDecisionReason": reason}}
+        )
+    )
     return 0
 
 

benchmarks/experiments/exp-001-known-callers/registration.json

@@ -0,0 +1,11 @@
+{
+  "experiment": "exp-001-known-callers",
+  "registered": "2026-10-02",
+  "store": "s3://sast-memory",
+  "manifest": "plane/experiments/exp-001-known-callers.yaml",
+  "manifest_digest": "76b54aa5ae908a4cd7797f3b4654acf03b90e5ad5d068216036e722104313929",
+  "pin": "fbc3220bbe1c8a4f8afa0a724fd41a51b390dd22",
+  "units_status": "pending",
+  "units_note": "model-free candidate generation on the 41 development repositories is not built; the plane's experiment-outcome task and Run annotation checks are not written",
+  "status": "registered, not run"
+}

benchmarks/experiments/exp-002-retrieval-ensemble/registration.json

@@ -0,0 +1,23 @@
+{
+  "experiment": "exp-002-retrieval-ensemble",
+  "registered": "2026-10-02",
+  "store": "s3://sast-memory",
+  "manifest": "plane/experiments/exp-002-retrieval-ensemble.yaml",
+  "manifest_digest": "f5004fb8618b1f7cc7dcf298d42b4ac1b6c0c1aa72a97299c77f937bd6175148",
+  "pin": "9213e5ffd978b387439e5568d32f4269dd81c021",
+  "units_status": "frozen",
+  "units_file": "plane/experiments/exp-002-retrieval-ensemble.units.jsonl",
+  "units_digest": "0de66bfa3ff844825e06f4f4227a14210d1b77f1cc08c0441c6d23e23fa5d91f",
+  "units": {
+    "total": 160,
+    "paired": 108,
+    "groups": 56,
+    "by_family": {
+      "injection": {"units": 119, "paired": 76, "pairs": 38, "groups": 34, "labels": {"1": 63, "0": 56}},
+      "access_control": {"units": 41, "paired": 32, "pairs": 16, "groups": 24, "labels": {"1": 25, "0": 16}}
+    },
+    "note": "frozen on the 937-example static memory (slice 2 evaluated 110 + 37 candidates on the 904-example memory of the same day; the recovered negatives of ef6945e moved the seeded compile split and the outer folds). Freezing needed two repairs first: example rows carried the plane-only features as null in x and the renderer refused them (1f525ff), and the RustFS store held none of the 788 embedding vectors the file store had (copied over with put_blob, no embedding call)."
+  },
+  "first_registration": {"registered": "2026-10-02", "pin": "fbc3220bbe1c8a4f8afa0a724fd41a51b390dd22", "units_status": "pending"},
+  "status": "run and analysed 2026-10-02: reject (result.json; benchmarks/measurements/2026-10-02-exp-002/record.json)"
+}

benchmarks/experiments/exp-002-retrieval-ensemble/result.json

@@ -0,0 +1,241 @@
+{
+  "alpha": 0.05,
+  "arms": {
+    "A": {
+      "budget_usd": 1.0,
+      "label": "today's sampling -- k = 5, sample 1 at temperature 0 and samples 2..5 at 0.7, one retrieved example set",
+      "repeats": 2,
+      "replay_only": false
+    },
+    "B": {
+      "budget_usd": 3.0,
+      "label": "retrieval ensemble -- k = 5 at temperature 0, each sample over a disjoint retrieved example set",
+      "repeats": 2,
+      "replay_only": false
+    }
+  },
+  "code_commit": "780caaa558f2cc091fd6b0fd5c618c3b488030c2",
+  "decision": "reject",
+  "experiment": "exp-002-retrieval-ensemble",
+  "hypothesis": "Drawing the k = 5 samples from five disjoint retrieved example sets at temperature 0 (a retrieval ensemble) makes the decision more repeatable than five temperature samples over one example set, without hurting the ranking within a candidate pair, at no higher cost per candidate.\n",
+  "looks": [
+    {
+      "boundary": 2.797,
+      "crossed": false,
+      "estimate": {
+        "a": 0.8611111111111112,
+        "b": 0.8888888888888888,
+        "ci": [
+          -0.08333333333333326,
+          0.1325301204819277
+        ],
+        "diff": 0.02777777777777768,
+        "groups": 28,
+        "metric": "canary_agreement",
+        "resamples": 10000,
+        "se": 0.05449719877526111,
+        "units": 72
+      },
+      "fraction": 0.5,
+      "groups": 28,
+      "z": 0.5097101943226363
+    },
+    {
+      "boundary": 1.977,
+      "crossed": false,
+      "estimate": {
+        "a": 0.875,
+        "b": 0.90625,
+        "ci": [
+          -0.03448275862068961,
+          0.0972972972972973
+        ],
+        "diff": 0.03125,
+        "groups": 56,
+        "metric": "canary_agreement",
+        "resamples": 10000,
+        "se": 0.0337315848833629,
+        "units": 160
+      },
+      "fraction": 1.0,
+      "groups": 56,
+      "z": 0.9264314175588332
+    }
+  ],
+  "manifest_digest": "f5004fb8618b1f7cc7dcf298d42b4ac1b6c0c1aa72a97299c77f937bd6175148",
+  "mde": 0.1,
+  "primary": [
+    {
+      "name": "canary_agreement",
+      "target": 0.9
+    },
+    {
+      "name": "within_pair_auc"
+    }
+  ],
+  "reason": "the CI of within_pair_auc excludes 0 against B",
+  "resamples": 10000,
+  "secondary": [
+    {
+      "name": "usd_per_candidate"
+    },
+    {
+      "name": "auc"
+    },
+    {
+      "name": "advisory_recall"
+    },
+    {
+      "name": "advisory_precision"
+    }
+  ],
+  "spend_usd": {
+    "A": 0.682094,
+    "B": 1.632737
+  },
+  "tests": {
+    "advisory_precision": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 0.8979591836734694,
+        "b": 0.9069767441860465,
+        "ci": [
+          -0.02020202020202022,
+          0.05081826012058566
+        ],
+        "diff": 0.009017560512577072,
+        "groups": 56,
+        "metric": "advisory_precision",
+        "resamples": 10000,
+        "se": 0.018550028052620342,
+        "units": 160
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "advisory_recall": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 0.5,
+        "b": 0.4431818181818182,
+        "ci": [
+          -0.1235955056179775,
+          0.01010101010101011
+        ],
+        "diff": -0.05681818181818182,
+        "groups": 56,
+        "metric": "advisory_recall",
+        "resamples": 10000,
+        "se": 0.033343898686966876,
+        "units": 160
+      },
+      "mcnemar": {
+        "b": 7,
+        "c": 2,
+        "concordant": 79,
+        "p_value": 0.1796875
+      },
+      "target": null
+    },
+    "auc": {
+      "ci_vs_zero": "against",
+      "direction": 1,
+      "estimate": {
+        "a": 0.8012941919191919,
+        "b": 0.7802241161616161,
+        "ci": [
+          -0.04139254385964919,
+          -0.0034279184917158645
+        ],
+        "diff": -0.0210700757575758,
+        "groups": 56,
+        "metric": "auc",
+        "resamples": 10000,
+        "se": 0.009565373841115053,
+        "units": 160
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "canary_agreement": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 0.875,
+        "b": 0.90625,
+        "ci": [
+          -0.03448275862068961,
+          0.0972972972972973
+        ],
+        "diff": 0.03125,
+        "groups": 56,
+        "metric": "canary_agreement",
+        "resamples": 10000,
+        "se": 0.0337315848833629,
+        "units": 160
+      },
+      "mcnemar": {
+        "b": 10,
+        "c": 15,
+        "concordant": 135,
+        "p_value": 0.42435622215270996
+      },
+      "target": {
+        "b_meets": true,
+        "level": 0.9
+      }
+    },
+    "usd_per_candidate": {
+      "ci_vs_zero": "against",
+      "direction": -1,
+      "estimate": {
+        "a": 0.002131543750000001,
+        "b": 0.005102303125,
+        "ci": [
+          0.002724568181818181,
+          0.00317872222222222
+        ],
+        "diff": 0.002970759374999999,
+        "groups": 56,
+        "metric": "usd_per_candidate",
+        "resamples": 10000,
+        "se": 0.00011577410791736543,
+        "units": 160
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "within_pair_auc": {
+      "ci_vs_zero": "against",
+      "direction": 1,
+      "estimate": {
+        "a": 0.8148148148148148,
+        "b": 0.7407407407407407,
+        "ci": [
+          -0.12857142857142867,
+          -0.015151515151515138
+        ],
+        "diff": -0.07407407407407407,
+        "groups": 56,
+        "metric": "within_pair_auc",
+        "resamples": 10000,
+        "se": 0.029550669251750233,
+        "units": 160
+      },
+      "mcnemar": null,
+      "target": null
+    }
+  },
+  "tost": null,
+  "units": {
+    "by_label": {
+      "0": 72,
+      "1": 88
+    },
+    "groups": 56,
+    "outcomes": 640,
+    "paired": 160
+  }
+}

benchmarks/experiments/exp-003-precision-bound-blocking/registration.json

@@ -0,0 +1,39 @@
+{
+  "experiment": "exp-003-precision-bound-blocking",
+  "registered": "2026-10-03",
+  "store": "s3://sast-memory",
+  "manifest": "plane/experiments/exp-003-precision-bound-blocking.yaml",
+  "manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
+  "pin": "dd7ebc119b5ad8e77c05d418eed9cbd46e193587",
+  "units_status": "frozen",
+  "units_file": "plane/experiments/exp-003-precision-bound-blocking.units.jsonl",
+  "units_digest": "338de998b1cfa36857cd793dbc4cf7fd0a13ab517a91f6790335881e941b3006",
+  "units": {
+    "total": 353,
+    "paired": 244,
+    "groups": 146,
+    "by_family": {
+      "access_control": {"units": 41, "positives": 25, "paired": 32, "paired_positives": 16, "groups": 24},
+      "deserialization": {"units": 48, "positives": 31, "paired": 34, "paired_positives": 17, "groups": 31},
+      "injection": {"units": 119, "positives": 63, "paired": 76, "paired_positives": 38, "groups": 34},
+      "output_encoding": {"units": 45, "positives": 29, "paired": 32, "paired_positives": 16, "groups": 27},
+      "path": {"units": 39, "positives": 23, "paired": 32, "paired_positives": 16, "groups": 23},
+      "untrusted_destination": {"units": 61, "positives": 37, "paired": 38, "paired_positives": 19, "groups": 36}
+    },
+    "note": "frozen model-free on the 937-example static memory; the outer folds (k 5, seed 0, compile fraction 0.25) are the incumbent evaluation's; injection and access_control equal exp-002's 160 units"
+  },
+  "kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
+  "zero_cost": "responses replayed from the response cache only, behind a meter with zero USD and call ceilings around a refusing client; a unit with any uncached response is excluded and counted; the run is void unless the meter reads 0 calls and $0",
+  "arms": {
+    "A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds (slope CI contains 1, intercept CI contains 0, ECE <= 0.05); BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
+    "B": "selective blocking: per family and outer fold, threshold chosen on the family's paired units in the other four folds as the lowest raw score t with Wilson 95% LB of flagged precision >= 0.95 (unsure verdicts never flag); none -> no BLOCK on that fold",
+    "C": "secondary, non-deciding: as B with point precision >= 0.95 over at least 10 training flags"
+  },
+  "primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision, per family, arm B, paired population",
+  "populations": {"primary": "paired (both sides of the candidate present; excludes every positives-only group and any unpaired unit)", "secondary": ["pooled (every unit)"]},
+  "secondary_metrics": ["flagged_precision", "coverage (flagged true positives / positives)", "flagged", "repository-cluster bootstrap 95% CI of flagged precision"],
+  "decision_rule": "REJECT B if in any family B flags held-out paired units at a point precision < 0.95; else ADOPT B if in some family B's held-out paired Wilson LB >= 0.95; else INCONCLUSIVE (the incumbent rule stays)",
+  "m4_reading": "a family reaches the M4 precision bar on held-out repositories when its held-out paired Wilson LB >= 0.95; point precision reported beside it",
+  "power": "a Wilson LB of 0.95 needs >= 73 flagged units with no false flag; the largest family holds 38 paired positives over all five folds, so arm B cannot offer a BLOCK in any family by construction; expected verdict inconclusive",
+  "status": "run and analysed 2026-10-03: inconclusive -- no arm flags any held-out unit (result.json); meter 0 client calls, $0.0; 142 of 353 units excluded for an uncached response"
+}

benchmarks/experiments/exp-003-precision-bound-blocking/result.json

@@ -0,0 +1,981 @@
+{
+  "arms": {
+    "A": {
+      "label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
+      "rule": {
+        "name": "calibrated_block"
+      }
+    },
+    "B": {
+      "label": "selective blocking -- per-family raw-score threshold from the other folds' paired units, lowest t with Wilson 95% LB of flagged precision >= 0.95",
+      "rule": {
+        "bound": "wilson",
+        "name": "precision_bound_block",
+        "precision": 0.95,
+        "select_on": "paired"
+      }
+    },
+    "C": {
+      "label": "secondary, non-deciding -- as B with point precision >= 0.95 over at least 10 training flags",
+      "rule": {
+        "bound": "point",
+        "min_flagged": 10,
+        "name": "precision_bound_block",
+        "precision": 0.95,
+        "select_on": "paired"
+      }
+    }
+  },
+  "code_commit": "7bb97ad0bca62b5f986c5768297d8b8fca50fae4",
+  "decision": "inconclusive",
+  "experiment": "exp-003-precision-bound-blocking",
+  "families": {
+    "access_control": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.077,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 16,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 32
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 24,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 41
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 16,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 32
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 24,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 41
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 16,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 32
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 24,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 41
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 41,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 12,
+          "outer-1": 13,
+          "outer-2": 13,
+          "outer-3": 13,
+          "outer-4": 13
+        }
+      }
+    },
+    "deserialization": {
+      "A": {
+        "calibration_holds": false,
+        "ece": null,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 3,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 2,
+          "true_flagged": 0,
+          "units": 5
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 12,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 11,
+          "true_flagged": 0,
+          "units": 14
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 3,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 2,
+          "true_flagged": 0,
+          "units": 5
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 12,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 11,
+          "true_flagged": 0,
+          "units": 14
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 3,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 2,
+          "true_flagged": 0,
+          "units": 5
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 12,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 11,
+          "true_flagged": 0,
+          "units": 14
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 14,
+        "excluded_replay_miss": 34,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 0,
+          "outer-2": 2,
+          "outer-3": 2,
+          "outer-4": 2
+        }
+      }
+    },
+    "injection": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.1067,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 26,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 76
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 34,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 63,
+          "true_flagged": 0,
+          "units": 119
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 26,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 76
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 34,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 63,
+          "true_flagged": 0,
+          "units": 119
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 26,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 76
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 34,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 63,
+          "true_flagged": 0,
+          "units": 119
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 119,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 29,
+          "outer-1": 33,
+          "outer-2": 33,
+          "outer-3": 26,
+          "outer-4": 31
+        }
+      }
+    },
+    "output_encoding": {
+      "A": {
+        "calibration_holds": false,
+        "ece": null,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 8,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 8,
+          "true_flagged": 0,
+          "units": 16
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 15,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 24
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 8,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 8,
+          "true_flagged": 0,
+          "units": 16
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 15,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 24
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 8,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 8,
+          "true_flagged": 0,
+          "units": 16
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 15,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 16,
+          "true_flagged": 0,
+          "units": 24
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "capacity": {
+        "evaluated": 24,
+        "excluded_replay_miss": 21,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 7,
+          "outer-1": 6,
+          "outer-2": 6,
+          "outer-3": 7,
+          "outer-4": 6
+        }
+      }
+    },
+    "path": {
+      "A": {
+        "paired": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        },
+        "pooled": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        },
+        "pooled": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        },
+        "pooled": {
+          "coverage": null,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 0,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 0,
+          "true_flagged": 0,
+          "units": 0
+        }
+      },
+      "capacity": {
+        "evaluated": 0,
+        "excluded_replay_miss": 39,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {}
+      }
+    },
+    "untrusted_destination": {
+      "A": {
+        "calibration_holds": false,
+        "ece": null,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 6,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 4,
+          "true_flagged": 0,
+          "units": 9
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 9,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 7,
+          "true_flagged": 0,
+          "units": 13
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null
+        }
+      },
+      "B": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 6,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 4,
+          "true_flagged": 0,
+          "units": 9
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 9,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 7,
+          "true_flagged": 0,
+          "units": 13
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null
+        }
+      },
+      "C": {
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 6,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 4,
+          "true_flagged": 0,
+          "units": 9
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 9,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 7,
+          "true_flagged": 0,
+          "units": 13
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null
+        }
+      },
+      "capacity": {
+        "evaluated": 13,
+        "excluded_replay_miss": 48,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 4,
+          "outer-1": 3,
+          "outer-2": 3,
+          "outer-3": 2
+        }
+      }
+    }
+  },
+  "hypothesis": "The calibrated-probability BLOCK rule (Platt map + calibration gate, ECE <= 0.05) is the wrong instrument for M4, which asks for precision on what is flagged. Selective blocking -- a per-family raw-score threshold chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95 -- reaches the M4 precision bar on held-out repositories where the calibrated rule offers no BLOCK.\n",
+  "instrument_check": {
+    "families": {
+      "access_control": {
+        "distinct_s": 30,
+        "paired": {
+          "blockable": 28,
+          "positives_before_first_negative": 10
+        },
+        "pooled": {
+          "blockable": 37,
+          "positives_before_first_negative": 19
+        },
+        "verdicts": {
+          "not_vulnerable": 19,
+          "unsure": 4,
+          "vulnerable": 18
+        }
+      },
+      "deserialization": {
+        "distinct_s": 5,
+        "paired": {
+          "blockable": 4,
+          "positives_before_first_negative": 1
+        },
+        "pooled": {
+          "blockable": 13,
+          "positives_before_first_negative": 10
+        },
+        "verdicts": {
+          "not_vulnerable": 2,
+          "unsure": 1,
+          "vulnerable": 11
+        }
+      },
+      "injection": {
+        "distinct_s": 48,
+        "paired": {
+          "blockable": 50,
+          "positives_before_first_negative": 3
+        },
+        "pooled": {
+          "blockable": 77,
+          "positives_before_first_negative": 11
+        },
+        "verdicts": {
+          "not_vulnerable": 46,
+          "unsure": 42,
+          "vulnerable": 31
+        }
+      },
+      "output_encoding": {
+        "distinct_s": 14,
+        "paired": {
+          "blockable": 11,
+          "positives_before_first_negative": 0
+        },
+        "pooled": {
+          "blockable": 19,
+          "positives_before_first_negative": 8
+        },
+        "verdicts": {
+          "not_vulnerable": 9,
+          "unsure": 5,
+          "vulnerable": 10
+        }
+      },
+      "untrusted_destination": {
+        "distinct_s": 9,
+        "paired": {
+          "blockable": 4,
+          "positives_before_first_negative": 1
+        },
+        "pooled": {
+          "blockable": 8,
+          "positives_before_first_negative": 4
+        },
+        "verdicts": {
+          "not_vulnerable": 3,
+          "unsure": 5,
+          "vulnerable": 5
+        }
+      }
+    },
+    "note": "descriptive, all folds together, ranking by raw score s among non-unsure verdicts; not nested, chooses nothing. It shows the scores are real and the zero is the rule's, not an unread input."
+  },
+  "manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
+  "meter": {
+    "client_calls": 0,
+    "decided": 211,
+    "excluded_replay_miss": 142,
+    "folds_digest": "a1f78e141a572a4adcc9cd57eb8a784c440c177f0b60d5ef5bed9e94c8753c5f",
+    "metered_usd": 0.0,
+    "replay_priced_usd_not_spent": 0.582876,
+    "replayed_responses": 1056
+  },
+  "min_flagged_for_bound": 73,
+  "overall": {
+    "A": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 59,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 68,
+        "true_flagged": 0,
+        "units": 138
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 89,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 122,
+        "true_flagged": 0,
+        "units": 211
+      }
+    },
+    "B": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 59,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 68,
+        "true_flagged": 0,
+        "units": 138
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 89,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 122,
+        "true_flagged": 0,
+        "units": 211
+      }
+    },
+    "C": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 59,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 68,
+        "true_flagged": 0,
+        "units": 138
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 89,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 122,
+        "true_flagged": 0,
+        "units": 211
+      }
+    }
+  },
+  "populations": {
+    "primary": "paired",
+    "secondary": [
+      "pooled"
+    ]
+  },
+  "primary": [
+    {
+      "name": "flagged_precision_lb",
+      "target": 0.95
+    }
+  ],
+  "reason": "B offers no BLOCK on held-out paired units in any family; the incumbent stays",
+  "resamples": 2000,
+  "secondary": [
+    {
+      "name": "flagged_precision"
+    },
+    {
+      "name": "coverage"
+    },
+    {
+      "name": "flagged"
+    }
+  ],
+  "spend_usd": {
+    "score": 0.0
+  },
+  "target_precision": 0.95,
+  "units": {
+    "by_label": {
+      "0": 89,
+      "1": 122
+    },
+    "evaluated": 211,
+    "excluded_replay_miss": 142,
+    "frozen": 353,
+    "groups": 89,
+    "not_run": 0,
+    "paired_evaluated": 138,
+    "unsure": 57
+  }
+}

benchmarks/experiments/exp-004-pooled-block-gate/instrument_check.py

@@ -0,0 +1,53 @@
+"""Instrument check for exp-004: are the scores real, and how clean is the pooled top of the ranking?
+
+Reads the run's arm_outcome rows from the memory store (OUSAST_MEMORY) and the frozen units file; makes no model call.
+usage: python benchmarks/experiments/exp-004-pooled-block-gate/instrument_check.py   (from the repository root)
+"""
+
+from __future__ import annotations
+
+import collections
+import json
+from pathlib import Path
+
+from openultrasast.config import load_dotenv
+from openultrasast.learn import experiments as ex
+from openultrasast.plane.memory import open_store
+
+EXPERIMENT = "exp-004-pooled-block-gate"
+UNITS = Path(f"plane/experiments/{EXPERIMENT}.units.jsonl")
+
+
+def main() -> None:
+    load_dotenv(Path(".env"))
+    rows = [r.row for r in open_store(None).rows(repo=ex.experiment_repo(EXPERIMENT), kind="arm_outcome")]
+    with UNITS.open() as handle:
+        units = {u["unit"]: u for u in (json.loads(line) for line in handle)}
+    print("arm_outcome rows", len(rows), "units file", len(units))
+    by_family: dict[str, list[tuple[float | None, int, str | None, str]]] = collections.defaultdict(list)
+    for row in rows:
+        unit = units.get(row.get("unit"))
+        if unit:
+            by_family[unit["family"]].append((row.get("s"), unit["label"], unit.get("pair"), str(row.get("verdict") or "")))
+    for family, xs in sorted(by_family.items()):
+        scored = [x for x in xs if x[0] is not None]
+        pairs: dict[str, dict[int, float]] = collections.defaultdict(dict)
+        for score, label, pair, _ in scored:
+            if pair:
+                pairs[pair][label] = score
+        full = [p for p in pairs.values() if 0 in p and 1 in p]
+        wins = sum(1 for p in full if p[1] > p[0])
+        ties = sum(1 for p in full if p[1] == p[0])
+        top = sorted((x for x in scored if x[2]), key=lambda x: -(x[0] or 0.0))
+        first_negative = next((i for i, x in enumerate(top) if x[1] == 0), len(top))
+        print(
+            f"{family}: n={len(xs)} distinct_scores={len({round(x[0] or 0.0, 4) for x in scored})} "
+            f"unsure={sum(1 for x in xs if x[3] == 'unsure')} pairs={len(full)} within_pair_win={wins} tie={ties} "
+            f"pos_above_first_neg={first_negative}"
+        )
+    pooled = sorted((x for xs in by_family.values() for x in xs if x[0] is not None and x[2]), key=lambda x: -(x[0] or 0.0))
+    print("pooled raw top-30 labels:", "".join(str(x[1]) for x in pooled[:30]))
+
+
+if __name__ == "__main__":
+    main()

benchmarks/experiments/exp-004-pooled-block-gate/record.json

@@ -0,0 +1,81 @@
+{
+  "experiment": "exp-004-pooled-block-gate",
+  "date": "2026-10-03",
+  "fill": {
+    "client_calls": 2073,
+    "usd": 1.238223,
+    "ceiling_usd": 2.0,
+    "still_missing": 0,
+    "dry_run_estimate_usd": 5.582,
+    "note": "the dry-run estimator over-priced 4.5x (no prompt-cache pricing); measured spend is the meter's"
+  },
+  "run": {
+    "units": 501,
+    "client_calls": 0,
+    "usd": 0.0,
+    "replayed": 2509,
+    "replay_miss": 0
+  },
+  "verdict": "inconclusive",
+  "reason": "no t_block on any outer fold, pooled or per family; B offers no BLOCK",
+  "instrument_check": {
+    "per_family": {
+      "access_control": {
+        "n": 66,
+        "distinct_scores": 41,
+        "unsure": 5,
+        "pairs": 28,
+        "within_pair_wins": 22,
+        "ties": 1,
+        "positives_above_first_negative": 12
+      },
+      "deserialization": {
+        "n": 64,
+        "distinct_scores": 38,
+        "unsure": 7,
+        "pairs": 25,
+        "within_pair_wins": 18,
+        "ties": 4,
+        "positives_above_first_negative": 5
+      },
+      "injection": {
+        "n": 149,
+        "distinct_scores": 52,
+        "unsure": 54,
+        "pairs": 53,
+        "within_pair_wins": 30,
+        "ties": 17,
+        "positives_above_first_negative": 6
+      },
+      "output_encoding": {
+        "n": 69,
+        "distinct_scores": 31,
+        "unsure": 32,
+        "pairs": 28,
+        "within_pair_wins": 18,
+        "ties": 6,
+        "positives_above_first_negative": 2
+      },
+      "path": {
+        "n": 66,
+        "distinct_scores": 43,
+        "unsure": 15,
+        "pairs": 29,
+        "within_pair_wins": 25,
+        "ties": 3,
+        "positives_above_first_negative": 13
+      },
+      "untrusted_destination": {
+        "n": 87,
+        "distinct_scores": 36,
+        "unsure": 33,
+        "pairs": 32,
+        "within_pair_wins": 18,
+        "ties": 4,
+        "positives_above_first_negative": 3
+      }
+    },
+    "pooled_raw_top30_labels": "111111111110111110111111110111",
+    "reading": "scores are real (31-52 distinct values per family); pooled raw top 30 holds 27 positives (0.90), first negative at rank 12; the Wilson bound needs >= 73 flags with no error, so precision at the top, not data volume alone, now binds"
+  }
+}

benchmarks/experiments/exp-004-pooled-block-gate/registration.json

@@ -0,0 +1,35 @@
+{
+  "experiment": "exp-004-pooled-block-gate",
+  "registered": "2026-10-03",
+  "store": "s3://sast-memory",
+  "manifest": "plane/experiments/exp-004-pooled-block-gate.yaml",
+  "manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
+  "pin": "2ad9c1db7ae673c2a2e48a0afadcaa795dfb1ced",
+  "units_status": "pending",
+  "units_file": "plane/experiments/exp-004-pooled-block-gate.units.jsonl",
+  "units_digest": "",
+  "units": "frozen at run time by `ousast learn experiment units` from the memory after the negatives harvest (merged in 5b2bd9b, 1133 static examples), committed, and registered again before the run; until then the run refuses (check_registered: the units file does not exist)",
+  "requirement": "learned-decision-engine Req 6.4, amended 2026-10-03 by maintainer decision (refines the BLOCK rule of Req 7.3 and 7.4)",
+  "kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
+  "zero_cost": "the run replays the response cache behind a zero-ceiling meter; a unit with any uncached response is excluded and counted. The harvest changed the memory the programs retrieve from, so a paid cache fill, approved separately with its own ceiling, must precede the run",
+  "normalisation": {
+    "choice": "per-family Platt scaling: learn.calibrate.fit_map with its existing floors (own slope and intercept per family at >= 20 positive and negative repository groups, a pooled slope with a per-family intercept at >= 10, no map below that and so no BLOCK for that family on that fold), fitted on the training folds' paired units only",
+    "why": "a rank-to-quantile map gives every family the same flag rate whatever its precision, so one pooled threshold would flag the top slice of a weak family as readily as of a strong one; Platt puts each family on the scale of its own training precision, which a pooled precision bound needs. The map is a normaliser only: precision is shown by the held-out bound, not by calibration (exp-003: ECE <= 0.05 does not hold per family)"
+  },
+  "arms": {
+    "A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds; BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
+    "B": "pooled gate: per outer fold j, training units normalised by an inner cross-fit; t_block = the lowest normalised score whose Wilson 95% LB of precision over the training paired flags of all families together is >= 0.95 (unsure never flags); t_quiet per family = the highest normalised score keeping >= 90% of that family's training paired positives; fold j's units BLOCK at p >= t_block, ADVISORY at p >= t_quiet (or without a map or t_quiet), silent below"
+  },
+  "family_floor": {
+    "F": 10,
+    "rule": "a family may block only if its own held-out paired flags (all five folds) number >= 10 with zero errors; otherwise its flags are reported as ADVISORY. The floor uses held-out labels to decide which families block and moves no threshold",
+    "why": "zero errors in 10 flags is unlikely for a family much worse than the bar (probability 0.056 at precision 0.75, 0.107 at 0.80); a larger F is out of reach of the smaller families (16-19 paired positives in exp-003's units over all folds); 10 is exp-003 arm C's training minimum"
+  },
+  "primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision over arm B's paired flags of all families together, before the family floor (so dropping a failing family cannot raise it)",
+  "populations": {"primary": "paired (both sides of the candidate present)", "secondary": ["pooled (every unit)"]},
+  "secondary_metrics": ["per-family flagged precision and Wilson LB", "the family-floor table", "quiet_recall per family (held-out positives at or above t_quiet / positives) against 0.90", "after-floor totals (what users would get)", "repository-cluster bootstrap 95% CI of flagged precision", "coverage", "flagged"],
+  "decision_rule": "on arm B's held-out paired flags of all families together, before the floor: REJECT B if their point precision < 0.95; else ADOPT B if their Wilson 95% LB >= 0.95 and at least one family passes the floor (>= 10 held-out paired flags, zero errors) -- those families block, every other family stays advisory; else INCONCLUSIVE (the incumbent rule stays)",
+  "power": "the pooled bound needs >= 73 flags with no error (110 with one, 142 with two). On exp-003's units (all 353 scored, result-complete-cache.json) few paired positives rank above the first negative per family on the raw score (access control 10, deserialization 7, injection 2, output encoding 0, path 8, untrusted destination 5); normalisation keeps each family's order, so the pooled top set reaches 73 only if the harvest's pairs separate better. An inconclusive verdict is a plausible outcome",
+  "tests": "tests/test_learn_rules.py: test_one_pooled_threshold_blocks_where_no_family_alone_reaches_the_bound, test_no_held_out_label_moves_a_threshold_or_the_map, test_a_family_with_one_held_out_error_cannot_block, test_quiet_threshold_keeps_ninety_percent_of_positives_and_nothing_is_silenced_blind, test_the_pooled_verdict_follows_the_registered_rule, test_the_exp004_manifest_registers_the_procedure_before_its_units",
+  "status": "registered 2026-10-03, units pending, not run"
+}

benchmarks/experiments/exp-004-pooled-block-gate/result.json

@@ -0,0 +1,822 @@
+{
+  "arms": {
+    "A": {
+      "label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
+      "rule": {
+        "name": "calibrated_block"
+      }
+    },
+    "B": {
+      "label": "pooled gate -- one t_block over all families on a per-family Platt score (training folds, inner cross-fit), t_quiet per family at 90% recall, family floor 10 clean held-out flags",
+      "rule": {
+        "floor": 10,
+        "name": "pooled_block_gate",
+        "normalise": "platt",
+        "precision": 0.95,
+        "recall": 0.9,
+        "select_on": "paired"
+      }
+    }
+  },
+  "code_commit": "90a2642019df414cb67b5dcea6e4ec235f35650a",
+  "decision": "inconclusive",
+  "experiment": "exp-004-pooled-block-gate",
+  "families": {
+    "access_control": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0715,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 38,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "quiet_recall": {
+          "paired": 0.8929,
+          "pooled": 0.9211
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.44900641988143913,
+          "outer-1": 0.3723384028044276,
+          "outer-2": 0.4202772526614184,
+          "outer-3": 0.4481849709797749,
+          "outer-4": 0.36346174463707154
+        }
+      },
+      "capacity": {
+        "evaluated": 66,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 22,
+          "outer-1": 22,
+          "outer-2": 22,
+          "outer-3": 23,
+          "outer-4": 23
+        }
+      }
+    },
+    "deserialization": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0725,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 25,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 50
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 39,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 25,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 25,
+          "true_flagged": 0,
+          "units": 50
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 39,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "quiet_recall": {
+          "paired": 0.88,
+          "pooled": 0.9231
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.17507194490500488,
+          "outer-1": 0.22766488322270723,
+          "outer-2": 0.2163745870743959,
+          "outer-3": 0.3112201878893598,
+          "outer-4": 0.363243722721776
+        }
+      },
+      "capacity": {
+        "evaluated": 64,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 20,
+          "outer-1": 20,
+          "outer-2": 20,
+          "outer-3": 20,
+          "outer-4": 20
+        }
+      }
+    },
+    "injection": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.1042,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 41,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 53,
+          "true_flagged": 0,
+          "units": 106
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 78,
+          "true_flagged": 0,
+          "units": 149
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 41,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 53,
+          "true_flagged": 0,
+          "units": 106
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 78,
+          "true_flagged": 0,
+          "units": 149
+        },
+        "quiet_recall": {
+          "paired": 0.8679,
+          "pooled": 0.8974
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.287983504097999,
+          "outer-1": 0.17524866355153343,
+          "outer-2": 0.12918182665453967,
+          "outer-3": 0.14521040470628938,
+          "outer-4": 0.23005714687697673
+        }
+      },
+      "capacity": {
+        "evaluated": 149,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 38,
+          "outer-1": 44,
+          "outer-2": 45,
+          "outer-3": 43,
+          "outer-4": 42
+        }
+      }
+    },
+    "output_encoding": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0736,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 41,
+          "true_flagged": 0,
+          "units": 69
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 28,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 28,
+          "true_flagged": 0,
+          "units": 56
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 39,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 41,
+          "true_flagged": 0,
+          "units": 69
+        },
+        "quiet_recall": {
+          "paired": 0.8571,
+          "pooled": 0.9024
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.3723166587726901,
+          "outer-1": 0.5329072757249385,
+          "outer-2": 0.3785066578507705,
+          "outer-3": 0.4389011844848602,
+          "outer-4": 0.43855966643440264
+        }
+      },
+      "capacity": {
+        "evaluated": 69,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 23,
+          "outer-1": 22,
+          "outer-2": 22,
+          "outer-3": 22,
+          "outer-4": 23
+        }
+      }
+    },
+    "path": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.0621,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 29,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 29,
+          "true_flagged": 0,
+          "units": 58
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 37,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 29,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 29,
+          "true_flagged": 0,
+          "units": 58
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 37,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 37,
+          "true_flagged": 0,
+          "units": 66
+        },
+        "quiet_recall": {
+          "paired": 0.8621,
+          "pooled": 0.8919
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.4680507207548809,
+          "outer-1": 0.3955717127313657,
+          "outer-2": 0.43694703358760895,
+          "outer-3": 0.4450583743716199,
+          "outer-4": 0.46396657833309796
+        }
+      },
+      "capacity": {
+        "evaluated": 66,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 23,
+          "outer-1": 23,
+          "outer-2": 24,
+          "outer-3": 23,
+          "outer-4": 23
+        }
+      }
+    },
+    "untrusted_destination": {
+      "A": {
+        "calibration_holds": false,
+        "ece": 0.1143,
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 31,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 32,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 50,
+          "true_flagged": 0,
+          "units": 87
+        },
+        "reason_by_fold": {
+          "outer-0": "calibration",
+          "outer-1": "calibration",
+          "outer-2": "calibration",
+          "outer-3": "calibration",
+          "outer-4": "calibration"
+        },
+        "thresholds_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        }
+      },
+      "B": {
+        "floor": {
+          "errors": 0,
+          "flagged": 0,
+          "floor": 10,
+          "may_block": false
+        },
+        "paired": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 31,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 32,
+          "true_flagged": 0,
+          "units": 64
+        },
+        "pooled": {
+          "coverage": 0.0,
+          "flagged": 0,
+          "flagged_precision": null,
+          "flagged_precision_ci": null,
+          "flagged_precision_lb": null,
+          "groups": 49,
+          "meets_target_lb": false,
+          "meets_target_point": false,
+          "positives": 50,
+          "true_flagged": 0,
+          "units": 87
+        },
+        "quiet_recall": {
+          "paired": 1.0,
+          "pooled": 1.0
+        },
+        "t_block_by_fold": {
+          "outer-0": null,
+          "outer-1": null,
+          "outer-2": null,
+          "outer-3": null,
+          "outer-4": null
+        },
+        "t_quiet_by_fold": {
+          "outer-0": 0.13676746287025585,
+          "outer-1": 0.11255775703980551,
+          "outer-2": 0.13763698150768486,
+          "outer-3": 0.09787938647374639,
+          "outer-4": 0.13763698150768486
+        }
+      },
+      "capacity": {
+        "evaluated": 87,
+        "excluded_replay_miss": 0,
+        "not_run": 0,
+        "paired_positives_by_training_folds": {
+          "outer-0": 26,
+          "outer-1": 26,
+          "outer-2": 24,
+          "outer-3": 26,
+          "outer-4": 26
+        }
+      }
+    }
+  },
+  "hypothesis": "One BLOCK threshold pooled across families on a per-family Platt-normalised score, chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95, reaches that bound on held-out repositories' paired units, where per-family thresholds (exp-003) could not for lack of flags; a per-family floor (>= 10 held-out flags, none wrong) keeps a family that the pooling masks from blocking.\n",
+  "manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
+  "min_flagged_for_bound": 73,
+  "overall": {
+    "A": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 182,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 195,
+        "true_flagged": 0,
+        "units": 390
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 220,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 283,
+        "true_flagged": 0,
+        "units": 501
+      }
+    },
+    "B": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 182,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 195,
+        "true_flagged": 0,
+        "units": 390
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 220,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 283,
+        "true_flagged": 0,
+        "units": 501
+      }
+    }
+  },
+  "overall_after_floor": {
+    "B": {
+      "paired": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 182,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 195,
+        "true_flagged": 0,
+        "units": 390
+      },
+      "pooled": {
+        "coverage": 0.0,
+        "flagged": 0,
+        "flagged_precision": null,
+        "flagged_precision_ci": null,
+        "flagged_precision_lb": null,
+        "groups": 220,
+        "meets_target_lb": false,
+        "meets_target_point": false,
+        "positives": 283,
+        "true_flagged": 0,
+        "units": 501
+      }
+    }
+  },
+  "populations": {
+    "primary": "paired",
+    "secondary": [
+      "pooled"
+    ]
+  },
+  "primary": [
+    {
+      "name": "flagged_precision_lb",
+      "target": 0.95
+    }
+  ],
+  "reason": "B offers no BLOCK on held-out paired units; the incumbent stays",
+  "resamples": 2000,
+  "secondary": [
+    {
+      "name": "flagged_precision"
+    },
+    {
+      "name": "coverage"
+    },
+    {
+      "name": "flagged"
+    },
+    {
+      "name": "quiet_recall"
+    }
+  ],
+  "spend_usd": {
+    "score": 0.0
+  },
+  "target_precision": 0.95,
+  "units": {
+    "by_label": {
+      "0": 218,
+      "1": 283
+    },
+    "evaluated": 501,
+    "excluded_replay_miss": 0,
+    "frozen": 501,
+    "groups": 220,
+    "not_run": 0,
+    "paired_evaluated": 390,
+    "unsure": 146
+  }
+}

benchmarks/experiments/exp-005-graph-slice/fill-B.json

@@ -0,0 +1,84 @@
+{
+  "arms": [
+    "B"
+  ],
+  "client_calls": 2394,
+  "experiment": "exp-005-graph-slice",
+  "families": {
+    "access_control": {
+      "cached": 0,
+      "conditional_retries": 330,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 0.9956892000000007,
+      "filled": 330,
+      "missing": 330,
+      "requests": 330,
+      "still_missing": 0,
+      "units": 66
+    },
+    "deserialization": {
+      "cached": 0,
+      "conditional_retries": 310,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 0.8843295999999988,
+      "filled": 310,
+      "missing": 310,
+      "requests": 310,
+      "still_missing": 0,
+      "units": 64
+    },
+    "injection": {
+      "cached": 0,
+      "conditional_retries": 675,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 2.270278999999999,
+      "filled": 675,
+      "missing": 675,
+      "requests": 675,
+      "still_missing": 0,
+      "units": 149
+    },
+    "output_encoding": {
+      "cached": 0,
+      "conditional_retries": 325,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 1.0014180000000004,
+      "filled": 328,
+      "missing": 325,
+      "requests": 328,
+      "still_missing": 0,
+      "units": 69
+    },
+    "path": {
+      "cached": 0,
+      "conditional_retries": 320,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 0.9399192000000005,
+      "filled": 325,
+      "missing": 320,
+      "requests": 325,
+      "still_missing": 0,
+      "units": 66
+    },
+    "untrusted_destination": {
+      "cached": 0,
+      "conditional_retries": 425,
+      "estimate_method": "estimated",
+      "estimated_output_tokens": 300,
+      "estimated_usd": 1.174316,
+      "filled": 426,
+      "missing": 425,
+      "requests": 426,
+      "still_missing": 0,
+      "units": 87
+    }
+  },
+  "reason": "",
+  "status": "done",
+  "usd": 1.5890382240000016
+}

benchmarks/experiments/exp-005-graph-slice/result.json

@@ -0,0 +1,165 @@
+{
+  "alpha": 0.05,
+  "arms": {
+    "A": {
+      "budget_usd": 0.0,
+      "label": "six slice-2 incumbents unchanged, replaying exp-004's response cache",
+      "repeats": 1,
+      "replay_only": true
+    },
+    "B": {
+      "budget_usd": 6.0,
+      "label": "the same six programs with the in-function slice added to their prompts",
+      "repeats": 1,
+      "replay_only": false
+    }
+  },
+  "code_commit": "0acbc7ec41dcaeca805c4720bc3f6d02cb940ee7",
+  "decision": "inconclusive",
+  "experiment": "exp-005-graph-slice",
+  "hypothesis": "Adding the in-function def-use slice to the prompt improves within-pair ranking of the six slice-2 incumbent programs on held-out repositories, at no more than 1.5 times the cost per candidate.\n",
+  "looks": [
+    {
+      "boundary": 1.959964,
+      "crossed": false,
+      "estimate": {
+        "a": 0.7615384615384615,
+        "b": 0.7461538461538462,
+        "ci": [
+          -0.0628019323671497,
+          0.03155339805825241
+        ],
+        "diff": -0.01538461538461533,
+        "groups": 220,
+        "metric": "within_pair_auc",
+        "resamples": 10000,
+        "se": 0.024030780235707972,
+        "units": 501
+      },
+      "fraction": 1.0,
+      "groups": 220,
+      "z": -0.6402045723740141
+    }
+  ],
+  "manifest_digest": "1b1ec814cde12801491376d2086c8c2d6db70f4a86fc308c0adc5a530398e554",
+  "mde": 0.0,
+  "primary": [
+    {
+      "name": "within_pair_auc"
+    }
+  ],
+  "reason": "the CI of within_pair_auc spans 0; the incumbent stays",
+  "resamples": 10000,
+  "secondary": [
+    {
+      "name": "top_30_precision"
+    },
+    {
+      "name": "unsure_rate"
+    },
+    {
+      "name": "usd_per_candidate"
+    }
+  ],
+  "spend_usd": {
+    "A": 1.444503,
+    "B": 1.589038
+  },
+  "tests": {
+    "top_30_precision": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 1.0,
+        "b": 1.0,
+        "ci": [
+          0.0,
+          0.0
+        ],
+        "diff": 0.0,
+        "groups": 220,
+        "metric": "top_30_precision",
+        "resamples": 10000,
+        "se": 0.0006666666666666664,
+        "units": 501
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "unsure_rate": {
+      "ci_vs_zero": "favour",
+      "direction": -1,
+      "estimate": {
+        "a": 0.29141716566866266,
+        "b": 0.23952095808383234,
+        "ci": [
+          -0.08583690987124465,
+          -0.01996007984031939
+        ],
+        "diff": -0.05189620758483032,
+        "groups": 220,
+        "metric": "unsure_rate",
+        "resamples": 10000,
+        "se": 0.01685991406098136,
+        "units": 501
+      },
+      "mcnemar": {
+        "b": 47,
+        "c": 21,
+        "concordant": 433,
+        "p_value": 0.0021862431776861705
+      },
+      "target": null
+    },
+    "usd_per_candidate": {
+      "ci_vs_zero": "against",
+      "direction": -1,
+      "estimate": {
+        "a": 0.002883239520958083,
+        "b": 0.0031717325349301397,
+        "ci": [
+          0.0002458205128205199,
+          0.0003319800399201574
+        ],
+        "diff": 0.0002884930139720565,
+        "groups": 220,
+        "metric": "usd_per_candidate",
+        "resamples": 10000,
+        "se": 2.1746161270931252e-05,
+        "units": 501
+      },
+      "mcnemar": null,
+      "target": null
+    },
+    "within_pair_auc": {
+      "ci_vs_zero": "spans",
+      "direction": 1,
+      "estimate": {
+        "a": 0.7615384615384615,
+        "b": 0.7461538461538462,
+        "ci": [
+          -0.0628019323671497,
+          0.03155339805825241
+        ],
+        "diff": -0.01538461538461533,
+        "groups": 220,
+        "metric": "within_pair_auc",
+        "resamples": 10000,
+        "se": 0.024030780235707972,
+        "units": 501
+      },
+      "mcnemar": null,
+      "target": null
+    }
+  },
+  "tost": null,
+  "units": {
+    "by_label": {
+      "0": 218,
+      "1": 283
+    },
+    "groups": 220,
+    "outcomes": 1002,
+    "paired": 501
+  }
+}

benchmarks/independent/evaluate.py

@@ -258,7 +258,7 @@ def score_v2() -> int:
     per_case, pool, unstable_pool = [], [], 0
     changing, total_questions, total_completed = [0, 0], 0, 0
     for case in cases():
-        path = lambda label: RESULTS / "scans" / f"{case['id']}--{label}.json"  # noqa: E731
+        path = lambda label: RESULTS / "scans" / f"{case['id']}--{label}.json"  # noqa: E731, B023 -- called only inside this iteration
         scans = {label: json.loads(path(label).read_text()) for label in pins(case) if path(label).is_file()}
         if len(scans) < 6:
             per_case.append({"id": case["id"], "status": "incomplete", "scans": sorted(scans)})
@@ -279,7 +279,9 @@ def score_v2() -> int:
         ]
         site_files = {site.partition("::")[0] for site in case.get("sites", [])}
         covered = set(old) | site_files
-        answered = any(r.split(":")[0] in covered for label in ("vulnerable_a", "vulnerable_b") for r in scans[label].get("completed_regions", []))
+        answered = any(
+            r.split(":")[0] in covered for label in ("vulnerable_a", "vulnerable_b") for r in scans[label].get("completed_regions", [])
+        )
         diff_v, union_v = stability(va, vb)
         diff_f, union_f = stability(fa, fb)
         changing[0] += diff_v + diff_f

benchmarks/independent/freeze.py

@@ -44,7 +44,7 @@ EXTENSIONS = {
 # scopes to API tokens" -- access control and input validation, which is exactly what a benign control must not be.
 SECURITY_WORDS = re.compile(
     r"secur|cve|ghsa|vuln|inject|xss|csrf|ssrf|sanitiz|escap|exploit|attack|auth|permission|cors|access|restrict|harden"
-    r"|token|scope|valid|filter|privileg|role|owner|password|secret|redirect|sql|query|login|log in|ldap|session|ssl|tls|verif|cert|crypt|hash|cookie",
+    r"|token|scope|valid|filter|privileg|role|owner|password|secret|redirect|sql|query|login|log in|ldap|session|ssl|tls|verif|cert|crypt|hash|cookie",  # noqa: E501
     re.I,
 )
 MAX_BENIGN_LINES = 200

benchmarks/learn/build_features.py

@@ -7,7 +7,7 @@ from:
 - quick rules and deterministic roles, run here on the unit's tree: a pair side materialised from its excerpt, a
   population or recipe pin exported from its clone or checkout (one at a time, deleted after, disk-guarded);
 - the engine: ``benchmarks/learn/engine_pairs.py`` records for pairs, the recorded protocol scans
-  (``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>.json``) for populations;
+  (``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>[_a|_b].json``) for populations;
 - verify passes a/b and ``agree`` from the harvest Run (``harvest-verify``) and, for population v2, the recorded
   ``validation-46`` Run; model roles from ``harvest-roles``;
 - callers and entry-point distance on a repository (``not_applicable`` on a pair excerpt: no repository exists).
@@ -21,6 +21,11 @@ An instrument that has not run for a unit yet (the engine job still running, a R
 memory store (``OUSAST_MEMORY``, default ``~/ousast-results/plane/memory``) as kind ``features``; a rerun replaces
 them by id. ``--counts`` writes the counts-only coverage record (no candidate, path or repository name).
 
+``--dry-run`` instead compares existing local store examples with the corrected engine features, printing
+per-source transitions from not-run/none to ran (and recorded failures separately). It reads only the spent
+v1/v2 manifests, saved scans and local Git objects; it writes neither the store nor ``--counts`` and does not
+rebuild other instruments. No label snapshot, harvest units or scratch directory is needed in this mode.
+
 Usage::
 
     python benchmarks/learn/build_features.py --labels labels-<sha>.jsonl --units <harvest plane>/units.json --counts counts.json
@@ -30,9 +35,11 @@ from __future__ import annotations
 
 import argparse
 import json
+import os
 import shutil
 import subprocess
 import tempfile
+import tomllib
 from collections import Counter, defaultdict
 from collections.abc import Mapping
 from pathlib import Path
@@ -61,6 +68,7 @@ from openultrasast.learn.features import (
     source_part,
     verify_part,
 )
+from openultrasast.learn.labels import Clone
 from openultrasast.learn.roles import RoleSet, from_model_roles, infer_for_checkout
 from openultrasast.pairs import DEFAULT_CATALOG, _materialize_side, _targets, load_pair_catalog
 from openultrasast.plane import memory
@@ -89,6 +97,110 @@ def read_json(path: Path) -> Any:
     return json.loads(path.read_text(encoding="utf-8")) if path.is_file() else None
 
 
+def resolve_scan(scans: Path, case: str, side: str) -> dict[str, Any] | None:
+    """Read the v1 single scan, otherwise combine the v2 repeats for this side."""
+    single = read_json(scans / f"{case}--{side}.json")
+    if single is not None:
+        return dict(single)
+    repeats = [read_json(scans / f"{case}--{side}_{repeat}.json") for repeat in ("a", "b")]
+    available = [r for r in repeats if r is not None]
+    if not available:
+        return None
+    # Identical rule for vulnerable and fixed: a finding in either repeat counts;
+    # exact duplicates count once. engine_part takes the highest rung and minimum
+    # witness steps across the union. Completion is sum(completed)/sum(questions).
+    # A successful repeat survives a failed/missing peer, which marks degradation.
+    ran = [r for r in available if engine_for(r) is None]
+    findings = {json.dumps(f, sort_keys=True): f for r in ran for f in r.get("findings", [])}
+    degradations = sorted({d for r in available for d in r.get("degradations", [])})
+    if len(ran) != len(repeats):
+        degradations.append("repeat_missing_or_failed")
+    return {
+        "state": "ran" if ran else "none" if all(engine_for(r) == NONE for r in available) else "failed",
+        "questions": sum(int(r.get("questions") or 0) for r in available),
+        "completed": sum(int(r.get("completed") or 0) for r in available),
+        "findings": list(findings.values()),
+        "degradations": degradations,
+        "image": "+".join(sorted({str(r.get("image") or "recorded") for r in available})),
+    }
+
+
+def recorded_engine(language: str, findings: list, result: Mapping | None, repository: bool) -> Part:
+    # The language list describes standalone scans. Repository scans also analyze
+    # TypeScript; recorded function findings take precedence over that list.
+    if language not in ENGINE_LANGUAGES and not findings and not (repository and language == "typescript" and result is not None):
+        return NONE
+    state = engine_for(result)
+    if state is not None:
+        return state
+    result = result or {}
+    questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
+    return engine_part(
+        findings, completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
+        vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
+    )  # fmt: skip
+
+
+class PinSourceIndex(SourceIndex):
+    """Resolve findings with the same matcher as a checkout, without exporting it."""
+
+    def __init__(self, clone: Path, pin: str) -> None:
+        super().__init__(clone)
+        self.clone, self.pin = Clone(clone), pin
+
+    def lines(self, path: str) -> list[str]:
+        if path not in self._lines:
+            self._lines[path] = self.clone.git("show", f"{self.pin}:{path}").splitlines()
+        return self._lines[path] or []
+
+
+def dry_run(store: memory.FileStore, results: Path, cache: Path) -> dict[str, Any]:
+    """Count existing examples whose engine becomes available; never write rows or blobs.
+
+    Only spent v1/v2 populations are read. Counts are examples, not the two
+    static/plane feature records. Other sources are reported with zero changes.
+    """
+    root = Path(__file__).resolve().parents[2]
+    cases = {}
+    for population in ("population-v1", "population-v2"):
+        manifest = root / "benchmarks" / "independent" / f"{population}.toml"
+        for case in tomllib.loads(manifest.read_text(encoding="utf-8"))["case"]:
+            for side in ("vulnerable", "fixed"):
+                cases[(population, memory.repo_key(case["repo"]), case[side], side)] = case["id"]
+    rows = [r.row for r in store.rows(kind="example")]
+    if not rows:
+        raise ValueError("dry run read no examples from the store")
+    counts: dict[str, Counter[str]] = defaultdict(Counter)
+    for row in rows:
+        count = counts[row["source"]]
+        count["examples"] += 1
+        old = row["instruments"]["engine"]
+        reason = "not_run" if old.get("version") == "missing:not-run" else "none" if old["state"] == "none" else None
+        if reason is None:
+            continue
+        key = (row["split"], row["repo"], row["pin"], row["pin_role"])
+        case_id = cases.get(key)
+        if case_id is None:
+            continue
+        scans = results / row["split"].replace("population", "independent") / "scans"
+        result = resolve_scan(scans, case_id, row["pin_role"])
+        path, _, function = row["candidate"].partition("::")
+        engine = {}
+        if engine_for(result) is None:
+            index = PinSourceIndex(cache / "independent" / case_id, row["pin"])
+            lines = index.lines(path)  # fail loudly if the candidate's input cannot be read
+            count["source_bytes_read"] += len("\n".join(lines).encode("utf-8"))
+            selected = dict(result or {})
+            selected["findings"] = [f for f in selected.get("findings", []) if f["site"].partition(":")[0] == path]
+            _, engine = _signals(index, [], selected, {})
+        part = recorded_engine(row["language"], engine.get((path, function, row["family"]), []), result, True)
+        if part.state == "ran":
+            count[f"{reason}_to_ran"] += 1
+        elif result is not None and part.state == "failed":
+            count[f"{reason}_to_recorded_failed"] += 1
+    return {source: {**{"not_run_to_ran": 0, "none_to_ran": 0}, **dict(count)} for source, count in sorted(counts.items())}
+
+
 def done(run: Path, entry: str) -> bool:
     state = (read_json(run / "state.json") or {}).get("tasks", {})
     return state.get(entry, {}).get("status") == "done"
@@ -188,17 +300,7 @@ def static_parts(root: Path, index: SourceIndex, quick: Mapping, engine: Mapping
     key = (path, function, family)
     parts: dict[str, Part] = {}
     parts["quick"] = quick_part(quick.get(key, []), rules, quick_version) if covers(QUICK_LANGUAGES, language) else NONE
-    if language not in ENGINE_LANGUAGES:
+    parts["engine"] = recorded_engine(language, engine.get(key, []), engine_result, repository)
-        parts["engine"] = NONE
-    elif engine_state is not None:
-        parts["engine"] = engine_state
-    else:
-        result = engine_result or {}
-        questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
-        parts["engine"] = engine_part(
-            engine.get(key, []), completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
-            vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
-        )  # fmt: skip
     lines = index.lines(path)
     parts["source"] = source_part(lines, language, function)
     parts["roles"] = roles_part(function_of(lines, path, language, function), inferred, language)
@@ -237,21 +339,34 @@ def export(clone: Path, pin: str, target: Path) -> bool:
 
 def main() -> int:
     parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
-    parser.add_argument("--labels", type=Path, required=True)
+    parser.add_argument("--labels", type=Path)
-    parser.add_argument("--units", type=Path, required=True, help="units.json of the harvest plane")
+    parser.add_argument("--units", type=Path, help="units.json of the harvest plane")
     parser.add_argument("--catalog", type=Path, default=DEFAULT_CATALOG)
     parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "openultrasast")
     parser.add_argument("--engine", type=Path, default=RESULTS / "plane" / "harvest-engine")
     parser.add_argument("--verify-run", type=Path, default=RESULTS / "plane" / "harvest-verify")
     parser.add_argument("--roles-run", type=Path, default=RESULTS / "plane" / "harvest-roles")
     parser.add_argument("--v46", type=Path, default=RESULTS / "plane" / "validation-46")
-    parser.add_argument("--scratch", type=Path, required=True)
+    parser.add_argument("--scratch", type=Path)
     parser.add_argument("--min-free-gb", type=float, default=1.2)
-    parser.add_argument("--counts", type=Path, required=True)
+    parser.add_argument("--counts", type=Path)
     parser.add_argument("--store", help="OUSAST_MEMORY spec (default: the environment's)")
     parser.add_argument("--limit", type=int, default=0, help="stop after this many targets (a test)")
     parser.add_argument("--kinds", default="pairs,pin", help="which targets: pairs, pin (population and recipe pins)")
+    parser.add_argument("--results-root", type=Path, default=RESULTS)
+    parser.add_argument("--dry-run", action="store_true", help="read local stored examples and print engine changes by source; no writes")
     args = parser.parse_args()
+    if args.dry_run:
+        spec = args.store or os.environ.get("OUSAST_MEMORY") or ""
+        if spec and not spec.startswith("file://"):
+            parser.error("--dry-run requires a local file:// store (no network)")
+        store = memory.open_store(spec)
+        if not isinstance(store, memory.FileStore):
+            parser.error("--dry-run requires a local file:// store (no network)")
+        print(json.dumps({"dry_run": True, "by_source": dry_run(store, args.results_root, args.cache)}, sort_keys=True))
+        return 0
+    if any(getattr(args, name) is None for name in ("labels", "units", "scratch", "counts")):
+        parser.error("--labels, --units, --scratch and --counts are required unless --dry-run is used")
 
     labels = [r for r in jsonl(args.labels) if r["unit"] == "pin" and r["source"] not in CONDITIONAL_SOURCES and not r.get("conditional")]
     units = json.loads(args.units.read_text(encoding="utf-8"))
@@ -275,6 +390,7 @@ def main() -> int:
     skipped: Counter[str] = Counter()
     absent: Counter[str] = Counter()  # family -> labelled pair candidates their side does not declare
     dropped = 0
+    reingested = 0  # rows the index remembered from an earlier build but a later one had dropped
     args.scratch.mkdir(parents=True, exist_ok=True)
     chosen = [(k, r) for k, r in sorted(targets.items()) if k[0] in args.kinds.split(",")]
     for number, ((kind, ref, role_or_pin), rows) in enumerate(chosen):
@@ -313,7 +429,7 @@ def main() -> int:
                         continue
                     split = {r["split"] for r in rows} | {unit["source"]}
                     population = "independent-v1" if "population-v1" in split else "independent-v2"
-                    engine_result = read_json(RESULTS / population / "scans" / f"{case_id}--{rows[0]['pin_role']}.json")
+                    engine_result = resolve_scan(args.results_root / population / "scans", case_id, rows[0]["pin_role"])
                 stems = [(args.verify_run, n) for n in names] + [(args.v46, unit["ref"])]
                 owner = next(
                     (n for n in names if f"{n}-roles" in {e for e in (read_json(args.roles_run / "state.json") or {}).get("tasks", {})}),
@@ -357,15 +473,24 @@ def main() -> int:
                         for inst, info in rec["instruments"].items():
                             state = "missing" if info.get("version") == "missing:not-run" else info["state"]
                             coverage[f"{family}:{inst}"][state] += 1
-            store.ingest_rows("harvest", f"features:{owner}", out_rows)
+            ingested = store.ingest_rows("harvest", f"features:{owner}", out_rows)
             built = {r["id"] for r in out_rows}
             stored = store.rows(repo=memory.repo_key(repo), pin=pin, kind="features", where={"run": "harvest", "task": owner})
+            present = {r.row["id"] for r in stored}
+            restored = 0
+            if ingested.skipped and not built <= present:
+                # the index remembers these very rows from an earlier build, but a later build dropped them as stale
+                # (the side did not declare the function then); an index hit is not evidence the rows are there
+                store.put_rows(out_rows)
+                restored = len(built - present)
+                reingested += restored
             stale = [r.row["id"] for r in stored if r.row["id"] not in built]
             dropped += store.drop_rows(memory.repo_key(repo), pin, stale)
-            print(json.dumps({"target": owner, "rows": len(out_rows), "dropped": len(stale)}), flush=True)
+            print(json.dumps({"target": owner, "rows": len(out_rows), "dropped": len(stale), "restored": restored}), flush=True)
     counts = {
         "absent_on_side": dict(sorted(absent.items())),
         "dropped_stale_rows": dropped,
+        "restored_after_drop": reingested,
         "records": {f"{p}:{f}": n for (p, f), n in sorted(tally.items())},
         "instrument_states_plane": {k: dict(sorted(v.items())) for k, v in sorted(coverage.items())},
         "skipped_label_rows": dict(skipped),

benchmarks/learn/engine_task_entry.py

@@ -0,0 +1,173 @@
+"""AX task entry: object-scoped transfers and stdlib only; result PUT is completion."""
+
+from __future__ import annotations
+
+import http.client
+import json
+import os
+import shutil
+import subprocess
+import sys
+import tarfile
+import time
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+
+class TaskEntryError(RuntimeError):
+    """A task-entry diagnostic containing only locally controlled text."""
+
+
+def transfer(request, destination=None, *, readiness_budget=None):
+    """Wait for actor egress on startup; bound later transfers to three attempts."""
+    deadline = time.monotonic() + readiness_budget if readiness_budget is not None else None
+    delay = 0.5
+    attempt = 0
+    while True:
+        remaining = deadline - time.monotonic() if deadline is not None else 60
+        if remaining <= 0:
+            raise TaskEntryError("egress not open after 60 s")
+        attempt += 1
+        try:
+            with urllib.request.urlopen(request, timeout=min(60, remaining)) as response:
+                if destination is not None:
+                    with destination.open("wb") as output:
+                        shutil.copyfileobj(response, output)
+            return
+        except urllib.error.HTTPError as exc:
+            exc.close()
+            if exc.code != 403 and not 500 <= exc.code < 600:
+                raise
+            if deadline is None and attempt >= 3:
+                raise
+        except (urllib.error.URLError, OSError, http.client.HTTPException):
+            if deadline is None and attempt >= 3:
+                raise
+        if deadline is not None:
+            remaining = deadline - time.monotonic()
+            if remaining <= 0:
+                raise TaskEntryError("egress not open after 60 s") from None
+            time.sleep(min(delay, remaining))
+            delay = min(delay * 2, 8)
+        else:
+            time.sleep(attempt)
+
+
+def download(url, destination, *, readiness_budget=None):
+    transfer(url, destination, readiness_budget=readiness_budget)
+
+
+def unpack(source, destination):
+    """Reject links and traversal; source archives contain only regular files/directories."""
+    with tarfile.open(source) as archive:
+        for member in archive.getmembers():
+            target = (destination / member.name).resolve()
+            if not target.is_relative_to(destination.resolve()) or not (member.isfile() or member.isdir()):
+                raise ValueError("unsafe source archive member")
+        archive.extractall(destination, filter="data")
+
+
+def failure_record(pin, reason):
+    return {
+        **pin,
+        "done": True,
+        "status": "failed",
+        "reason": reason,
+        "units": [
+            {
+                **unit,
+                "status": "failed" if unit.get("supported", True) else "unsupported",
+                "reason": reason,
+                "instrument": {"files": [], "bytes": 0, "jvm": []},
+                "questions": {},
+                "questions_asked": [],
+                "questions_completed": [],
+                "witness_rows": [],
+                "traces": [],
+            }
+            for unit in pin.get("units", [])
+        ],
+    }
+
+
+def main(*, work_root=Path("/tmp"), worker_script=None, readiness_budget=60):
+    pin = {}
+    failed = False
+    output = work_root / "out"
+    result = output / "result.json"
+    step = "prepare output"
+    try:
+        output.mkdir(parents=True, exist_ok=True)
+        result.unlink(missing_ok=True)
+        questions = work_root / "questions.json"
+        step = "GET questions"
+        download(os.environ["QUESTIONS_URL"], questions, readiness_budget=readiness_budget)
+        step = "parse questions"
+        pin = json.loads(questions.read_text())
+        if not isinstance(pin, dict):
+            pin = {}
+            raise ValueError("questions object must contain a prepared pin")
+        source = work_root / "source.tar"
+        step = "GET source"
+        download(os.environ["SOURCE_URL"], source)
+        step = "unpack source"
+        case = work_root / "case"
+        case.mkdir(parents=True, exist_ok=True)
+        unpack(source, case)
+        step = "configure worker"
+        deadline = int(os.environ.get("DEADLINE", "900"))
+        command = [
+            sys.executable,
+            str(worker_script or Path(__file__).with_name("engine_trace_worker.py")),
+            "--worker",
+            str(questions),
+            "--root",
+            str(case),
+            "--output",
+            str(result),
+            "--heap-profile",
+            "cluster",
+            "--deadline",
+            str(deadline),
+            "--question-deadline",
+            os.environ.get("QUESTION_DEADLINE", "120"),
+        ]
+        step = "run worker"
+        completed = subprocess.run(command, timeout=deadline + 5, check=False)
+        if completed.returncode:
+            raise TaskEntryError(f"worker exited {completed.returncode}")
+        step = "read worker result"
+        if not result.is_file() or not json.loads(result.read_text()).get("done"):
+            raise TaskEntryError("worker did not write a completed result")
+    except Exception as exc:
+        failed = True
+        # Avoid including exception text containing a presigned URL in durable output.
+        if isinstance(exc, urllib.error.HTTPError):
+            detail = f"HTTP {exc.code}"
+        else:
+            detail = str(exc) if isinstance(exc, TaskEntryError) else type(exc).__name__
+        reason = f"{step}: {detail}"
+        try:
+            output.mkdir(parents=True, exist_ok=True)
+            result.write_text(json.dumps(failure_record(pin, reason)) + "\n")
+        except Exception:
+            print("task entry could not write failure result", file=sys.stderr)
+    step = "pack result"
+    try:
+        archive = work_root / "result.tar"
+        with tarfile.open(archive, "w") as tar:
+            for path in sorted(output.iterdir()):
+                tar.add(path, arcname=path.name)
+        data = archive.read_bytes()
+        step = "PUT result"
+        request = urllib.request.Request(os.environ["RESULT_URL"], data=data, method="PUT", headers={"Content-Type": "application/x-tar"})
+        transfer(request, readiness_budget=readiness_budget if failed else None)
+    except Exception as exc:
+        detail = f"HTTP {exc.code}" if isinstance(exc, urllib.error.HTTPError) else type(exc).__name__
+        print(f"{step}: {detail}; dispatcher deadline remains authoritative", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())