diff --git a/.kiro/specs/ai-service-plane/tasks.md b/.kiro/specs/ai-service-plane/tasks.md index f2905b1..68b6356 100644 --- a/.kiro/specs/ai-service-plane/tasks.md +++ b/.kiro/specs/ai-service-plane/tasks.md @@ -4,7 +4,7 @@ Order follows the dependency chain: ax bring-up (parallel with 1-2) -> manifests ax-backed reconciler -> tasks -> increment run on ax -> measurement. Every task ships with its tests; a task is done when its tests pass in the host suite, ruff and mypy are clean, and the commit is gated on pytest's own exit code. Script-level pipeline work stays paused. -- [ ] 0. ax bring-up on this host +- [x] 0. ax bring-up on this host - Install `kind`, `kubectl`, the `ax` CLI and `ko` (user-local). `ops/ax/up.sh`: create the kind cluster and install Agent Substrate with its `hack/create-kind-cluster.sh` and `hack/install-ate-kind.sh --deploy-ate-system`, deploy the ax control plane with `ko` to the kind registry, apply one trivial Task @@ -21,7 +21,7 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level the decision to work around it is the maintainer's. - _Requirements: 3.5, 3.6_ -- [ ] 1. Manifest schemas and validation +- [x] 1. Manifest schemas and validation - Add `pyyaml` as a declared dependency in `pyproject.toml`. - `src/openultrasast/plane/manifests.py`: dataclasses and loaders for `Task`, `Workspace`, `Model` (`ax.io/v1alpha1`, ax's fields only) and `Run` (`openultrasast.io/v1alpha1`: tasks, dependsOn, inputs, @@ -30,14 +30,14 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level - `tests/test_plane_manifests.py`: valid manifests load; each rejection names the field. - _Requirements: 1.1, 1.2, 1.3, 1.4_ -- [ ] 2. Budget metering +- [x] 2. Budget metering - `src/openultrasast/plane/budget.py`: a client wrapper that sums provider usage per call, prices from the Model's parameters, raises `BudgetExhausted` at the ceiling and `AccountError` on 401/402 or an "insufficient balance" body; an unpriced model reports `unpriced` and refuses a `usd` budget. - Tests with a scripted client: ceiling stop, account error, unpriced refusal, cache-hit pricing. - _Requirements: 2.3, 2.4_ -- [ ] 3. Runner contract +- [x] 3. Runner contract - `src/openultrasast/plane/runner.py`: read `AX_TASK_YAML` / `AX_WORKSPACES_YAML`; materialise `git` entries at the pinned commit from the local case cache and write `spec.files` (first boot only); run `spec.command` as a child `python -m openultrasast.plane.tasks.` in its own process group, cwd the @@ -62,7 +62,7 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level with identical env only the addressed one runs (the golden scenario). - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6_ -- [ ] 4. ax-backed reconciler and CLI +- [x] 4. ax-backed reconciler and CLI - `src/openultrasast/plane/reconciler.py` (under 500 lines, no pipeline logic, no local subprocess executor): `state.json`, statuses, topological submission through the `ax` CLI (path injectable), `ax resume task` after `ax apply` (a created Task is `Suspended`), retrying DeadlineExceeded/Unavailable @@ -91,7 +91,7 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level trivial Task end to end. - _Requirements: 3.1, 3.2, 3.3, 3.4, 2.2, 4.5, 4.6, 7.1, 7.3_ -- [ ] 5. `repo-facts` task +- [x] 5. `repo-facts` task - `src/openultrasast/plane/tasks/repo_facts.py`: product files, functions per file (per-language declaration patterns), call sites per candidate function across product files (path, line, enclosing function, cap 40), sorted and deterministic; `summary.json` per the task contract. @@ -99,7 +99,7 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level identical output on a second run. - _Requirements: 5.1, 5.2, 2.1_ -- [ ] 6. `verify` and `agree` tasks +- [x] 6. `verify` and `agree` tasks - `src/openultrasast/plane/tasks/verify.py`: the batched hunt moved from `verify_batched.py` (one hunt per file, up to 6 candidates, 6 steps, candidate-anchored sites) with known callers from `facts.json` in the prompt, tool turns and usage per hunt in `units.jsonl`, per-unit resume, `pass` from env, budget wrapper @@ -111,7 +111,7 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level unfinished file; a scripted 402 yields `failed`; agree on two scripted passes. - _Requirements: 5.3, 2.1, 2.2, 2.3, 7.2_ -- [ ] 7. Manifests for the increment +- [x] 7. Manifests for the increment - `plane/models/deepseek-flash.yaml` (annotated extension, prices in parameters); `plane/tasks/repo-facts.yaml`, `plane/tasks/verify.yaml`, `plane/tasks/agree.yaml`; `plane/runs/validation-46.yaml`: repo-facts -> verify pass a, verify pass b -> agree, budgets per task, diff --git a/ops/ax/README.md b/ops/ax/README.md new file mode 100644 index 0000000..822c6ce --- /dev/null +++ b/ops/ax/README.md @@ -0,0 +1,49 @@ +# ax on this host + +The service plane (`.kiro/specs/ai-service-plane/`) runs every task on google/ax over Agent Substrate in a +single-node kind cluster. ax is the only executor; `ousast plane run` submits, resumes, starts and collects. + +## Bring-up + + ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway, + # receiver Service, runner image (digest pinned), ax CLI, smoke Task + ousast plane doctor # kind, Substrate, ax controller, runner image + ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table + ops/ax/down.sh # deletes the cluster and its registry + +Tools live in `~/go/bin` (kind, ax, ko, kubectl-ate) and `~/.local/bin` (kubectl); sources and rendered files +in `~/.cache/ousast/ax-src/` (the runner digest pin is `runner-image` there). + +## Measured footprint (2026-09-29, idle, two workers) + +| Item | Value | +|---|---| +| kind node container memory | 1.7 GiB (27% of the 7.7 GiB host) | +| registry container memory | 34 MiB | +| node volume (images, containerd) | 4.9 GB on disk | +| worker pool | 2 gVisor workers, 1 CPU / 1.5 GiB limit each | + +Disk is the binding constraint: the Substrate install and `ko` builds need several GB of Go caches. Clear them +afterwards (`go clean -cache -modcache`); rerunning the full Substrate install on a near-full disk fails in +`ko resolve`. + +## What this host taught (each one cost a failed live run) + +- Substrate installs no WorkerPool ("no free workers") -> `workerpool.yaml.tmpl`. +- Actor images must be pinned by digest; ax's default runner image on gcr.io needs credentials. +- A release `ax` CLI skews from the server; build it from the deployed checkout. +- ax's API rejects unknown fields; metadata is `name` + `atespace`; Git entries have no `commit`. +- `ax apply` leaves a Task Suspended until `ax resume`; the first resume may time out while Substrate builds the + template's golden snapshot. ax has no Completed phase: completion is our artifact delivery. +- Substrate boots each template once as a golden actor with the task's env, so the runner waits for a start + request sent through `atenet-router`; the provider key travels only in that request. +- Actor egress goes through the `atenet-egress` gateway, deny by default, one EgressPolicy per actor, hostnames + only; the gateway dials the address the actor connected to, so the receiver is a Service dialled by ClusterIP + with its name as Host. The Envoy gateway needs a Rust build; the agentgateway variant needs none. +- Apply a Task after the Workspaces it binds; ax copies them at create time. + +## What a larger ax deployment needs + +A Kubernetes cluster with Agent Substrate (and its egress gateway), the ax control plane, a registry the workers +can pull from, the runner image pinned by digest, the receiver reachable as a Service, and the provider host +allowed per task. The manifests under `plane/` move unchanged; only `OUSAST_ARTIFACT_*` and the context change.