spec(ai-service-plane): switch to ax completely; routing and token attribution requirements

Maintainer decisions 2026-09-29: ax (kind + Agent Substrate + ax controller on
this host) is the only executor; the Run layer carries dependencies, artifacts,
budgets and attribution until ax has them. Requirement 7 declares routing per
Task (one Model per Task, no runtime switching) and a token attribution table
from provider usage fields. Task 0 (bring-up, doctor), runner artifact delivery,
ax-backed reconciler with a fake-ax test seam. harnessx-removal brief records
the HarnessX inventory; removal is gated on the increment's measurement.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

.kiro/specs/ai-service-plane/design.md

@@ -2,9 +2,10 @@
 
 ## Overview
 
-A thin service plane under `src/openultrasast/plane/`: manifest schemas in ax's shape plus a `Run` kind, a local
+A thin service plane under `src/openultrasast/plane/`: manifest schemas in ax's shape plus a `Run` kind, an
-reconciler that executes a Run's tasks in dependency order with persisted status and resume, a runner that
+ax-backed reconciler that submits a Run's tasks to ax on this host's kind cluster in dependency order with
-honours ax's contract, and task entrypoints. The pipeline's logic lives only in the task entrypoints; the
+persisted status and resume, a runner that honours ax's contract and delivers artifacts, and task entrypoints.
+ax (with Agent Substrate) is the only executor; there is no local subprocess path. The pipeline's logic lives only in the task entrypoints; the
 reconciler and runner know nothing about prompts, models or scoring (Requirement 3.4). The first increment moves
 `repo-facts` and `verify` (plus a model-free `agree` step) and measures them on the 46-candidate set
 (Requirement 6); the other concerns stay as scripts until that measurement is recorded.
@@ -56,13 +57,19 @@ into the bound path (locally a `git archive` from the case cache, as `evaluate.e
 (`openultrasast.plane.tasks.<name>`). With `AX_RUNNER_HTTP=1` it serves `/healthz` and `/readyz` (503 until
 workspaces are ready) on port 80 in a thread; locally the flag is unset (Requirement 4.2).
 
-### Reconciler (`reconciler.py`)
+### Reconciler (`reconciler.py`, ax-backed)
 
-`ousast plane run <Run.yaml>` and `ousast plane status <run>`. State: `state.json` with per-task status in
+`ousast plane run <Run.yaml>`, `ousast plane status <run>` and `ousast plane doctor`. State: `state.json` with
-`pending | running | suspended | done | failed | unfinished`, timestamps, spend. Execution: topological order;
+per-task status in `pending | running | suspended | done | failed | unfinished`, timestamps, spend. Execution:
-tasks sharing a `serialize` label never overlap; everything else may run in parallel up to `--workers`. A task's
+topological order; for each ready task the reconciler renders the Task manifest with the run's env
-outcome is read from its `summary.json`: `done` only when the task says every unit finished; `unfinished` on a
+(`OUSAST_BUDGET_*`, `OUSAST_INPUT_*`, `OUSAST_OUTPUT_DIR`, `OUSAST_ARTIFACT_URL`), applies it and its
-budget stop; `failed` on an account or authentication error or a crash. On rerun, `done` tasks are skipped and
+Workspaces through the `ax` CLI (`ax apply`, `ax get`, `ax delete`; the CLI path is injectable so tests use a
+fake), and polls ax's status. Artifacts: the reconciler serves a small HTTP receiver on the host (reachable from
+kind pods via the node's host gateway); the runner posts the output directory there at exit. Tasks sharing a
+`serialize` label never overlap; everything else may run in parallel up to `--workers`. A task's outcome is
+read from its delivered `summary.json`: `done` only when the task says every unit finished; `unfinished` on a
+budget stop; `failed` on an account or authentication error, a crash, or a missing delivery. `status` prints
+the token attribution table (Requirement 7.3) from the delivered summaries and writes `attribution.json`. On rerun, `done` tasks are skipped and
 the others re-executed; per-unit resume is the task's own duty (Requirement 2.2). No prompts, scoring or model
 calls anywhere in this module; a test counts its lines.
 
@@ -125,9 +132,11 @@ plus the per-candidate cost, turns and the declared-site match (`evaluate.matche
 
 - `tests/test_plane_manifests.py`: valid ax manifests load; an unknown field, a bad provider without the
   annotation, and a Run with a cycle are rejected with the field named.
-- `tests/test_plane_reconciler.py`: stub tasks (shell commands writing `summary.json`) in a DAG: order,
+- `tests/test_plane_reconciler.py`: a fake `ax` CLI (a script recording `apply`/`get`/`delete` calls and
-  `serialize` non-overlap, skip-on-rerun, `unfinished` and `failed` propagation, the lock; a test asserts the
+  posting scripted `summary.json` deliveries to the receiver) drives a DAG: order, `serialize` non-overlap,
-  module is under 500 lines.
+  skip-on-rerun, `unfinished` and `failed` propagation, missing delivery, the lock, the attribution table; a
+  test asserts the module is under 500 lines. `tests/test_plane_ax_live.py` (marker `ax`, skipped unless
+  `ousast plane doctor` passes) applies one trivial Task on the real cluster.
 - `tests/test_plane_runner.py`: the env contract on a fixture workspace; `/readyz` 503 then 200 with the flag.
 - `tests/test_plane_repo_facts.py`: a three-file tree; callers found across files, none from tests or vendor;
   identical output on a second run.

.kiro/specs/ai-service-plane/requirements.md

@@ -7,20 +7,28 @@ one long conversation, each with its own resume, budget and pass logic. Measured
 tokens are tool I/O around those scripts, and verification re-derives the same per-repository facts in every
 hunt and every pass. This feature restructures the pipeline as a Kubernetes-style service plane at the model
 level, in the shape of google/ax: declarative `Task`, `Workspace` and `Model` manifests, a reconciler that
-executes them in isolation with resume, and a runner contract. ax's software (Kubernetes, Agent Substrate) is a
+executes them in isolation with resume, and a runner contract. The plane runs on ax itself: a `kind` cluster
-later deployment target; its design is adopted now (`brief.md`).
+with Agent Substrate and the ax controller on this host (maintainer decision 2026-09-29: "switch to ax
+completely now"). What ax does not yet carry -- task dependencies, artifacts, token budgets (on ax's roadmap),
+a DeepSeek provider -- is carried by our thin `Run` layer and dissolves into ax as ax gains it.
 
 ## Boundary Context
 
-- **In scope**: the manifest kinds and a `Run` kind for dependencies, artifacts and budgets; a local reconciler
+- **In scope**: the manifest kinds and a `Run` kind for dependencies, artifacts and budgets; bring-up of ax on
-  for this host (one engine container at a time); the runner contract; the first two tasks, `repo-facts` and
+  this host (kind + Agent Substrate + ax controller) with a doctor command; an ax-backed reconciler that
+  submits Tasks to ax (one engine-bearing task at a time) and is the only executor; the runner contract and a
+  runner image; the first two tasks, `repo-facts` and
   `verify`, with the existing scripts as reference behaviour; measurement on the 46-candidate validation set.
-- **Out of scope**: running on ax or Kubernetes; changing what the model is asked (prompts, families, pass rules)
+- **Out of scope**: a remote or multi-node cluster; contributing providers or budgets to ax upstream (a
+  follow-on once the increment is measured); changing what the model is asked (prompts, families, pass rules)
   except as required to consume `repo-facts`; the judge stage, the config-family question and population v3
   (they become tasks later, under their own specs); any change to `evaluate.py`'s protocol scoring.
 - **Adjacent expectations**: the independent-population protocols and records stay as they are; `finding_dump`
   and the engine path are untouched; `vulnerabilities-over-plumbing` applies -- this is plumbing that must pay
   for itself in measured tokens and unchanged or better recall.
+- **Target state (maintainer, 2026-09-29)**: this plane replaces HarnessX as the agentic plane; HarnessX is
+  removed by its own spec (`harnessx-removal`) once Requirement 6's measurement shows value; the `Run` layer
+  shrinks as ax gains dependencies, budgets and artifacts. No new HarnessX feature is added meanwhile.
 
 ## Requirements
 
@@ -55,19 +63,25 @@ without ever recording partial work as a result.
 4. A task's spend is metered from the provider's usage fields and stops the task at its budget; a task with no
    priced model reports "unpriced", not zero.
 
-### Requirement 3: A local reconciler executes a Run on this host
+### Requirement 3: An ax-backed reconciler executes a Run on this host's ax
 
-**User Story:** As the maintainer, I want to run a whole pipeline with one command on the laptop, with the same
+**User Story:** As the maintainer, I want to run a whole pipeline with one command on the laptop, with every
-manifests a cluster would take.
+task executed by ax on the local cluster, so that the same manifests move to a bigger cluster unchanged.
 
 #### Acceptance Criteria
 
-1. The reconciler executes tasks in dependency order, at most one engine-bearing task at a time, and passes
+1. The reconciler submits each task to ax as a `Task` (with its bound `Workspace`s and `Model`) in dependency
-   artifacts by path.
+   order, at most one engine-bearing task at a time, reads task outcome from ax's status plus the task's
+   delivered `summary.json`, and passes artifacts between tasks by placing a producer's outputs where the
+   consumer's manifest expects them. It never executes a task as a local subprocess; ax is the only executor.
 2. Task status (`pending`, `running`, `suspended`, `done`, `failed`, `unfinished`) is persisted and shown by
    one status command.
 3. A suspended or interrupted run resumes from persisted status (Requirement 2.2) with the same command.
 4. The reconciler is under 500 lines and holds no pipeline logic: no prompts, no scoring, no model calls.
+5. `ousast plane doctor` verifies the host's ax in one command: kind cluster reachable, Agent Substrate control
+   API up, ax controller ready, runner image loaded; each check names what is missing and how to bring it up.
+6. Bring-up on this host (kind + Agent Substrate + ax) is scripted, documented with its measured memory and
+   CPU footprint, and smoke-tested with one trivial Task before any pipeline task runs.
 
 ### Requirement 4: The runner contract is ax's
 
@@ -78,8 +92,12 @@ later.
 
 1. An entrypoint receives its task and workspace manifests as YAML in `AX_TASK_YAML` and `AX_WORKSPACES_YAML`
    and takes nothing else from the command line except what the manifest's `command` states.
-2. The default image serves `/healthz` and `/readyz` when run as a container; locally these may be no-ops.
+2. The runner image serves `/healthz` and `/readyz` on port 80, `/readyz` returning 503 until workspaces are
+   materialised, as ax's runner contract requires; the image contains this package and is loaded into kind.
 3. One fixture test per task type runs the entrypoint against a small workspace and checks its artifacts.
+4. Because ax has no artifact channel, the runner delivers the task's output directory (including
+   `summary.json` and `units.jsonl`) to the reconciler's artifact endpoint named in `OUSAST_ARTIFACT_URL`
+   before it reports completion; a delivery failure is reported as `failed`, never as done.
 
 ### Requirement 5: `repo-facts` is the memory layer
 
@@ -101,9 +119,29 @@ before the remaining concerns are converted.
 
 #### Acceptance Criteria
 
-1. `repo-facts` + `verify` run on the 46-candidate validation set through the reconciler, two passes, and
+1. `repo-facts` + `verify` run on the 46-candidate validation set through the reconciler on ax, two passes, and
    report cost per candidate, tool turns per hunt and declared sites agreed.
 2. Acceptance: declared sites agreed at least 16 of 20 (the current value) and cost per candidate below the
    current $0.022 for two passes; otherwise the increment is reported as not met and the design revisited.
 3. The development transcript measure (`benchmarks/dev/token_report.py`) is run at the end of the increment and
    recorded next to the result.
+
+### Requirement 7: Routing is a declared concern of the plane
+
+**User Story:** As the maintainer, I want which model serves which concern to be a manifest decision with a
+budget, not a prompt trick inside one conversation, so that token spend is structured by component.
+
+#### Acceptance Criteria
+
+1. Every `Task` in a `Run` binds exactly one `Model`; a Run may bind different Models to different tasks
+   (a cheap model for reads and triage, a stronger one for verification and judgement), and the reconciler
+   passes the bound Model's identity and prices to the task.
+2. A task never chooses or switches its model at runtime; the Model it was bound to is recorded in its
+   `summary.json`.
+3. `ousast plane status <run>` prints a token attribution table: one row per task and Model binding with
+   calls, input tokens, cache-hit tokens, output tokens and USD, a total row, and a per-Model subtotal; every
+   number comes from the provider's usage fields recorded in the tasks' `summary.json`, never estimated. A
+   `--units` flag breaks a task down per unit (file or candidate) from `units.jsonl`. The same table is
+   written to `attribution.json` in the run directory so runs can be compared.
+4. A task that needs a second model for a sub-step (a judge over a verifier's output) is a separate task with
+   its own Model binding, never a second client inside one task.

.kiro/specs/ai-service-plane/spec.json

@@ -1,7 +1,7 @@
 {
   "feature_name": "ai-service-plane",
   "created_at": "2026-09-29T15:40:00Z",
-  "updated_at": "2026-09-29T16:40:00Z",
+  "updated_at": "2026-09-29T17:55:00Z",
   "language": "en",
   "phase": "implementation",
   "approvals": {

.kiro/specs/ai-service-plane/tasks.md

@@ -1,9 +1,20 @@
 # Implementation Plan
 
-Order follows the dependency chain: manifests -> budget -> runner -> reconciler -> tasks -> increment run ->
+Order follows the dependency chain: ax bring-up (parallel with 1-2) -> manifests -> budget -> runner ->
-measurement. Every task ships with its tests; a task is done when its tests pass in the host suite, ruff and
+ax-backed reconciler -> tasks -> increment run on ax -> measurement. Every task ships with its tests; a task is done when its tests pass in the host suite, ruff and
 mypy are clean, and the commit is gated on pytest's own exit code. Script-level pipeline work stays paused.
 
+- [ ] 0. ax bring-up on this host
+  - Install `kind`, `kubectl`, the `ax` CLI and `ko` (user-local). `ops/ax/up.sh`: create the kind cluster and
+    install Agent Substrate with its `hack/create-kind-cluster.sh` and `hack/install-ate-kind.sh
+    --deploy-ate-system`, deploy the ax control plane with `ko` to the kind registry, apply one trivial Task
+    and wait for it to finish; `ops/ax/down.sh` tears it down. Record the measured idle footprint (RSS, CPU)
+    in `ops/ax/README.md` next to the 7 GB host constraint.
+  - `ousast plane doctor` (in `reconciler.py` or a sibling `doctor.py`): the four checks of Requirement 3.5.
+  - Stop rule: if the cluster does not come up within one working session, report the exact failure and stop;
+    the decision to work around it is the maintainer's.
+  - _Requirements: 3.5, 3.6_
+
 - [ ] 1. Manifest schemas and validation
   - Add `pyyaml` as a declared dependency in `pyproject.toml`.
   - `src/openultrasast/plane/manifests.py`: dataclasses and loaders for `Task`, `Workspace`, `Model`
@@ -23,20 +34,29 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level
 - [ ] 3. Runner contract
   - `src/openultrasast/plane/runner.py`: read `AX_TASK_YAML` / `AX_WORKSPACES_YAML`; materialise `git`
     entries at the pinned commit from the local case cache and write `spec.files`; execute `spec.command`
-    by importing `openultrasast.plane.tasks.<name>`; optional `/healthz` `/readyz` server under
+    by importing `openultrasast.plane.tasks.<name>`; `/healthz` `/readyz` server on port 80 (503 until
-    `AX_RUNNER_HTTP=1`; exit codes 0 done, 2 failed, 3 unfinished.
+    workspaces are ready; disabled only under `AX_RUNNER_HTTP=0` for unit tests); at exit, POST the output
-  - `tests/test_plane_runner.py`: the env contract on a fixture workspace; readyz 503 then 200.
+    directory as a tar stream to `OUSAST_ARTIFACT_URL` and report `failed` if delivery fails; exit codes
-  - _Requirements: 4.1, 4.2, 4.3_
+    0 done, 2 failed, 3 unfinished. `plane/Dockerfile.runner`: this package on Python 3.12, installed to
+    `/usr/local/bin/ax-task-runner`, loaded into kind by `ops/ax/up.sh`.
+  - `tests/test_plane_runner.py`: the env contract on a fixture workspace; readyz 503 then 200; delivery to a
+    local receiver; delivery failure yields `failed`.
+  - _Requirements: 4.1, 4.2, 4.3, 4.4_
 
-- [ ] 4. Local reconciler and CLI
+- [ ] 4. ax-backed reconciler and CLI
-  - `src/openultrasast/plane/reconciler.py` (under 500 lines, no pipeline logic): `state.json`, statuses,
+  - `src/openultrasast/plane/reconciler.py` (under 500 lines, no pipeline logic, no local subprocess
-    topological execution, `serialize` labels never overlapping, `--workers`, budgets and artifact paths
+    executor): `state.json`, statuses, topological submission through the `ax` CLI (path injectable), status
-    passed as task env, outcome read from `summary.json`, skip `done` on rerun, PID lock per run.
+    polling, the artifact HTTP receiver, `serialize` labels never overlapping, `--workers`, budgets and
-  - `ousast plane run <Run.yaml>`, `ousast plane status <run>`, `ousast plane workspaces <population.toml>`
+    artifact env passed in the rendered Task manifest, outcome read from the delivered `summary.json`, skip
-    (generates Workspace manifests per case pin).
+    `done` on rerun, PID lock per run, `ax delete` of finished Tasks.
-  - `tests/test_plane_reconciler.py`: stub tasks in a DAG: order, non-overlap, skip-on-rerun, `unfinished`
+  - `ousast plane run <Run.yaml>`, `ousast plane status <run>` (token attribution table per task and Model
-    and `failed` propagation, the lock, and the line-count assertion.
+    from the tasks' `summary.json` usage; `--units` from `units.jsonl`; also written to `attribution.json`),
-  - _Requirements: 3.1, 3.2, 3.3, 3.4, 2.2_
+    `ousast plane workspaces <population.toml>` (generates Workspace manifests per case pin).
+  - `tests/test_plane_reconciler.py`: a fake `ax` CLI drives a DAG: order, non-overlap, skip-on-rerun,
+    `unfinished` and `failed` propagation, missing delivery, the lock, the attribution table, and the
+    line-count assertion. `tests/test_plane_ax_live.py` (marker `ax`, skipped without a live cluster): one
+    trivial Task end to end.
+  - _Requirements: 3.1, 3.2, 3.3, 3.4, 2.2, 7.1, 7.3_
 
 - [ ] 5. `repo-facts` task
   - `src/openultrasast/plane/tasks/repo_facts.py`: product files, functions per file (per-language
@@ -50,23 +70,24 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level
   - `src/openultrasast/plane/tasks/verify.py`: the batched hunt moved from `verify_batched.py` (one hunt per
     file, up to 6 candidates, 6 steps, candidate-anchored sites) with known callers from `facts.json` in the
     prompt, tool turns and usage per hunt in `units.jsonl`, per-unit resume, `pass` from env, budget wrapper
-    from task 2; `summary.json` never `done` unless every unit finished.
+    from task 2; `summary.json` never `done` unless every unit finished, and records the bound Model and
+    the summed usage fields (prompt, cache-hit, output tokens, calls, usd).
   - `src/openultrasast/plane/tasks/agree.py`: agreement across pass outputs, per-candidate cost and turns,
     declared-site matching via `evaluate.matches_v2`.
   - `tests/test_plane_verify.py`: scripted client; prompt carries the callers; resume asks only the
     unfinished file; a scripted 402 yields `failed`; agree on two scripted passes.
-  - _Requirements: 5.3, 2.1, 2.2, 2.3_
+  - _Requirements: 5.3, 2.1, 2.2, 2.3, 7.2_
 
 - [ ] 7. Manifests for the increment
   - `plane/models/deepseek-flash.yaml` (annotated extension, prices in parameters);
     `plane/tasks/repo-facts.yaml`, `plane/tasks/verify.yaml`, `plane/tasks/agree.yaml`;
     `plane/runs/validation-46.yaml`: repo-facts -> verify pass a, verify pass b -> agree, budgets per task,
     inputs from the recorded candidate and triage artifacts.
-  - Workspaces generated for the 15 validation cases' vulnerable pins.
+  - Workspaces generated for the 15 validation cases' vulnerable pins; the runner image rebuilt and loaded.
   - _Requirements: 1.1, 1.2, 6.1_
 
 - [ ] 8. Measurement of the first increment
-  - Run `validation-46` through the reconciler (needs DeepSeek credit); record cost per candidate, tool turns
+  - Run `validation-46` through the reconciler on ax (needs DeepSeek credit); record cost per candidate, tool turns
     per hunt, declared sites agreed, against `verifier-batched-check-2026-09-29.json`.
   - Run `benchmarks/dev/token_report.py` on the increment's development session.
   - Record `benchmarks/independent/plane-increment-1.json` with the gate result (met or not met) and commit;

.kiro/specs/harnessx-removal/brief.md

@@ -0,0 +1,29 @@
+# Brief: remove HarnessX after the plane proves value
+
+Status: brief only, gated. Not started until `ai-service-plane` Requirement 6 records a met gate.
+
+## Inventory (read-only, 2026-09-29)
+
+- Code: `src/openultrasast/harness_ext.py` (97 lines) and `hunter_harness.py` (197) exist only for HarnessX;
+  guarded branches in `cli.py` (372-512), `fusion.py` (LLM panels, ~100 lines), `verify_judge.py` (judge path,
+  ~90), `stage_processors.py` (`host_under_harnessx`, ~40), `config.py` (`HarnessxConfig`). About 575 of
+  30,243 lines (1.9%). Every path falls back deterministically when the extra is absent and records a
+  degradation (`harnessx_extra_unavailable`; fallbacks `run_hunter_pool`, `structural_verifier`,
+  `deterministic_panels`). `evolve`, `regress`, `benchmark`, `gate` and quick mode do not depend on it.
+- Packaging: `pyproject.toml` extra `harnessx` pinned to a git commit; mypy override; `uv.lock` entry.
+- Tests: 14 files touch it; dedicated `test_harness_ext.py`, `test_hunter_harness.py`, `test_cli_hx_dispatch.py`,
+  `test_verify_judge.py`; `tests/conftest.py` fixture `assert_cold_of_harnessx`.
+- Docs: README sections "Configuring the HarnessX agentic plane" and "The HarnessX self-improving cycle";
+  `docs/examples.md` section 6; `docs/threat-model.md` egress and spend caps.
+- Specs: `harnessx-self-improving-rulesets` (approved, 43/43 done, Requirement 1 is direct adoption);
+  `learning-harness` (30/33); `three-stage-scan` framed as its follow-up; `openrouter-sast-harness` (no
+  spec.json); mentions in four more; steering `overview.md:175`.
+
+## Shape of the removal
+
+1. Replace the three guarded paths with plane tasks (hunter pool -> `verify`; judge -> `judge`; panels ->
+   `judge`/`agree`) so their fallbacks become the only path, then delete the two modules, the config block,
+   the extra, the override and the fixture.
+2. Retire the four specs' HarnessX requirements with a dated note, not a rewrite; update README and docs.
+3. Gate: the plane increment met (recall and cost), and the replaced features measured no worse on the
+   development pins before deletion.