Open_Ultra_SAST/.kiro/specs/ai-service-plane/tasks.md

Implementation Plan

Order follows the dependency chain: ax bring-up (parallel with 1-2) -> manifests -> budget -> runner -> ax-backed reconciler -> tasks -> increment run on ax -> measurement. Every task ships with its tests; a task is done when its tests pass in the host suite, ruff and mypy are clean, and the commit is gated on pytest's own exit code. Script-level pipeline work stays paused.

• 0. ax bring-up on this host

  • Install kind, kubectl, the ax CLI and ko (user-local). ops/ax/up.sh: create the kind cluster and install Agent Substrate with its hack/create-kind-cluster.sh and hack/install-ate-kind.sh --deploy-ate-system, deploy the ax control plane with ko to the kind registry, apply one trivial Task and wait for it to finish; ops/ax/down.sh tears it down. Record the measured idle footprint (RSS, CPU) in ops/ax/README.md next to the 7 GB host constraint.
  • ousast plane doctor (in reconciler.py or a sibling doctor.py): the four checks of Requirement 3.5.
  • Stop rule: if the cluster does not come up within one working session, report the exact failure and stop; the decision to work around it is the maintainer's.
  • Requirements: 3.5, 3.6

• 1. Manifest schemas and validation

  • Add pyyaml as a declared dependency in pyproject.toml.
  • src/openultrasast/plane/manifests.py: dataclasses and loaders for Task, Workspace, Model (ax.io/v1alpha1, ax's fields only) and Run (openultrasast.io/v1alpha1: tasks, dependsOn, inputs, outputs, budget, serialize); reject unknown fields, a provider outside ax's two without the openultrasast.io/provider-extension annotation, and a Run with a cycle or an input naming no task.
  • tests/test_plane_manifests.py: valid manifests load; each rejection names the field.
  • Requirements: 1.1, 1.2, 1.3, 1.4

• 2. Budget metering

  • src/openultrasast/plane/budget.py: a client wrapper that sums provider usage per call, prices from the Model's parameters, raises BudgetExhausted at the ceiling and AccountError on 401/402 or an "insufficient balance" body; an unpriced model reports unpriced and refuses a usd budget.
  • Tests with a scripted client: ceiling stop, account error, unpriced refusal, cache-hit pricing.
  • Requirements: 2.3, 2.4

• 3. Runner contract

  • src/openultrasast/plane/runner.py: read AX_TASK_YAML / AX_WORKSPACES_YAML; materialise git entries at the pinned commit from the local case cache and write spec.files; execute spec.command by importing openultrasast.plane.tasks.<name>; /healthz /readyz server on port 80 (503 until workspaces are ready; disabled only under AX_RUNNER_HTTP=0 for unit tests); at exit, POST the output directory as a tar stream to OUSAST_ARTIFACT_URL and report failed if delivery fails; exit codes 0 done, 2 failed, 3 unfinished. plane/Dockerfile.runner: this package on Python 3.12, installed to /usr/local/bin/ax-task-runner, loaded into kind by ops/ax/up.sh.
  • tests/test_plane_runner.py: the env contract on a fixture workspace; readyz 503 then 200; delivery to a local receiver; delivery failure yields failed.
  • Requirements: 4.1, 4.2, 4.3, 4.4

• 4. ax-backed reconciler and CLI

  • src/openultrasast/plane/reconciler.py (under 500 lines, no pipeline logic, no local subprocess executor): state.json, statuses, topological submission through the ax CLI (path injectable), status polling, the artifact HTTP receiver, serialize labels never overlapping, --workers, budgets and artifact env passed in the rendered Task manifest, outcome read from the delivered summary.json, skip done on rerun, PID lock per run, ax delete of finished Tasks.
  • ousast plane run <Run.yaml>, ousast plane status <run> (token attribution table per task and Model from the tasks' summary.json usage; --units from units.jsonl; also written to attribution.json), ousast plane workspaces <population.toml> (generates Workspace manifests per case pin).
  • tests/test_plane_reconciler.py: a fake ax CLI drives a DAG: order, non-overlap, skip-on-rerun, unfinished and failed propagation, missing delivery, the lock, the attribution table, and the line-count assertion. tests/test_plane_ax_live.py (marker ax, skipped without a live cluster): one trivial Task end to end.
  • Requirements: 3.1, 3.2, 3.3, 3.4, 2.2, 7.1, 7.3

• 5. repo-facts task

  • src/openultrasast/plane/tasks/repo_facts.py: product files, functions per file (per-language declaration patterns), call sites per candidate function across product files (path, line, enclosing function, cap 40), sorted and deterministic; summary.json per the task contract.
  • tests/test_plane_repo_facts.py: three-file tree; cross-file callers found, tests and vendor excluded, identical output on a second run.
  • Requirements: 5.1, 5.2, 2.1

• 6. verify and agree tasks

  • src/openultrasast/plane/tasks/verify.py: the batched hunt moved from verify_batched.py (one hunt per file, up to 6 candidates, 6 steps, candidate-anchored sites) with known callers from facts.json in the prompt, tool turns and usage per hunt in units.jsonl, per-unit resume, pass from env, budget wrapper from task 2; summary.json never done unless every unit finished, and records the bound Model and the summed usage fields (prompt, cache-hit, output tokens, calls, usd).
  • src/openultrasast/plane/tasks/agree.py: agreement across pass outputs, per-candidate cost and turns, declared-site matching via evaluate.matches_v2.
  • tests/test_plane_verify.py: scripted client; prompt carries the callers; resume asks only the unfinished file; a scripted 402 yields failed; agree on two scripted passes.
  • Requirements: 5.3, 2.1, 2.2, 2.3, 7.2

• 7. Manifests for the increment

  • plane/models/deepseek-flash.yaml (annotated extension, prices in parameters); plane/tasks/repo-facts.yaml, plane/tasks/verify.yaml, plane/tasks/agree.yaml; plane/runs/validation-46.yaml: repo-facts -> verify pass a, verify pass b -> agree, budgets per task, inputs from the recorded candidate and triage artifacts.
  • Workspaces generated for the 15 validation cases' vulnerable pins; the runner image rebuilt and loaded.
  • Requirements: 1.1, 1.2, 6.1

• 8. Measurement of the first increment

  • Run validation-46 through the reconciler on ax (needs DeepSeek credit); record cost per candidate, tool turns per hunt, declared sites agreed, against verifier-batched-check-2026-09-29.json.
  • Run benchmarks/dev/token_report.py on the increment's development session.
  • Record benchmarks/independent/plane-increment-1.json with the gate result (met or not met) and commit; if not met, revise the design before task 9.
  • Requirements: 6.1, 6.2, 6.3

• 9. Handover

  • ops/README.md: how to run a Run locally and what an ax deployment would need (Kubernetes, Agent Substrate, the runner image, the provider extension); memory note updated; scripts that the tasks replaced marked as reference-only in their docstrings.
  • Requirements: 4.2