A third verify pass on the 10 disputed candidates, 2-of-3 agreement (Req 6.4),
run on ax by extending the same Run: 16/20 declared sites agreed (reference 16),
$0.0205 per candidate with recorded triage (reference $0.022). Caveats recorded:
a design change on the same set; 17 vs 19 sites flagged in any pass; three
non-declared pgadmin candidates promoted.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The design revisit after plane-increment-1.json (14/20 agreed, 19/20 in either
pass): a third verify pass on the candidates passes a and b dispute, and agree
decides 2-of-3 where pass c asked (the a/b rule elsewhere; c alone never decides).
- verify: optional OUSAST_INPUT_ONLY restricts the hunt to the named candidates
(agree's disputed.json); an empty list is done with 0 units and no model client.
- agree: optional OUSAST_INPUT_PASS_C; metrics keep every key and add tiebreak
{asked, flipped_to_agreed, flipped_to_rejected, rule}; usd_total, hunts and
declared-site metrics are on the final decision.
- generate: per case <case>-vc and <case>-final appended to validation-46; the
first 60 entries stay byte-identical, so a rerun keeps their done state.
- Offline replay over the recorded outputs with an empty pass c reproduces the
recorded agree summaries exactly (14/20 agreed, 23 agreed, 10 disputed).
- Spec: Requirement 6.4, design Measurement, task 8.1.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
validation-46 ran done on google/ax: 60 tasks, 357 DeepSeek calls, $0.83 with
recorded triage ($0.0194 per candidate, reference $0.022: cost met). Declared
sites agreed 14/20 against the reference's 16/20: recall criterion not met.
Both runs find 19/20 sites in at least one pass; the two-pass agreement
flicker moved three sites. Design revisit before task 9.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A producer's artifact rendered as spec.files of a generated <task>-inputs
Workspace ends up in ax's AX_WORKSPACES_YAML env value; Agent Substrate
rejects values over 32768 characters, so a 182 KB facts.json broke ax apply
on the live cluster. The reconciler now passes OUSAST_INPUTS instead, the
receiver serves a running task its declared inputs over GET /inputs/..., and
the runner fetches them after the start (failed fetch -> failed summary).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The end-to-end smoke Run finished done on the local ax: submit, egress policy,
resume, router start, repo-facts in gVisor, artifacts delivered through the
egress gateway, attribution table printed, task and policy cleaned up.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Agent Substrate's egress gateway denies an actor without an EgressPolicy, so
no task could deliver its artifacts or reach a model. egress.py grants each
task exactly: http/80 to ousast-receiver.ax-system.svc.cluster.local, and
TLS passthrough/443 to the bound Workspaces' https Git hosts and the bound
Model's openultrasast.io/egress-hosts (api.deepseek.com for deepseek-flash).
No IPs, no wildcards, no https (MITM) rules.
The reconciler writes the policy with kubectl-ate between ax apply and
resume, reads it back, waits OUSAST_EGRESS_SETTLE (11 s, the gateway's
cached deny), and after ax delete task confirms the policy went with its
actor (kubectl-ate has no delete verb; the store cascades).
The receiver is a selector-less Service (ops/ax/receiver-service.yaml.tmpl,
applied by up.sh together with the no-build agentgateway egress gateway)
on port 18090 of the host. Measured live: the gateway checks the policy on
Host but connects to the address the actor dialled, so the runner dials the
Service's ClusterIP (OUSAST_ARTIFACT_DIAL) with the receiver's name as Host.
The runner now labels its delivery with the Run's task name (OUSAST_TASK),
which is what the receiver accepts; the ax Task name was refused with 403.
The Receiver moved to egress.py to keep reconciler.py at 460 lines.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Agent Substrate's golden-snapshot boot runs the same image with the same task env,
so the runner no longer starts its command by itself: after the workspaces are
ready it waits for POST /ousast/v1/start (run/task checked, once per boot, refused
after a delivered run). The reconciler sends it through svc/atenet-router with
ate-target-actor, which never addresses the golden actor (plane/router.py), and the
bound Model's secretKey variable travels only in that body, into the child's env.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The first live run on the local ax cluster and ax's source (docs/runner.md,
docs/sandbox.md, internal/controller/reconciler.go, internal/server/server.go)
showed a lifecycle the task-3/4 code did not model:
- `ax apply` creates a Task Suspended (Ready=False, TaskSuspended); it runs only
after `ax resume task`. The first resume of a new image can fail with
DeadlineExceeded while Agent Substrate builds the golden snapshot, and ax's
server then stores phase Failed; a later resume succeeds.
- ax has no Completed phase and never reads a command's exit back.
- The runner is PID 1 and must keep serving after the command exits; stop and
suspend deliver SIGTERM, to be forwarded to the command's process group.
Our runner imported the task in-process and exited, which stopped the
container ("Container guest is stopped") and made every resume time out.
- ax's controller probes /readyz?check=workspace; the query string made our
handler answer 404.
Runner (Req 4.1-4.4): spec.command runs as a child
`python -m openultrasast.plane.tasks.<name>` in its own process group, cwd the
first workspace, AX_METADATA_URL set; ax's metadata paths are served. After the
child exits the runner reads summary.json (a crash without one becomes failed
with the exit code and stderr tail), delivers the output directory, writes
/workspace/.ousast-state/<task>.done.json and serves until SIGTERM (forwarded
to the group, SIGKILL after OUSAST_TERM_GRACE, exit 0). A boot with the marker
never reruns the command; a failed delivery is retried alone. Workspaces are
prepared on the first boot only. AX_RUNNER_EXIT_AFTER_COMMAND=1 is the
unit-test escape.
Reconciler (Req 2.2, 2.3, 3.1-3.3): `ax resume task` after apply, retrying
DeadlineExceeded/Unavailable with backoff up to OUSAST_RESUME_TIMEOUT (900 s);
completion is the artifact delivery; Failed after a resume went through, a
vanished task, or no delivery within OUSAST_TASK_TIMEOUT (7200 s) fail the
task with ax's condition message; `ax delete task` follows. Ax gains resume()
and phase() returns the condition message. doctor moves to plane/doctor.py
(tasks.md task 0 allows the sibling) to keep reconciler.py under 500 lines.
Tests: the fake ax follows the real lifecycle (Suspended, a failing first
resume that leaves Failed, Running forever, delivery only after a resume went
through), plus Failed-with-message, vanish, resume-timeout and task-timeout
scenarios; runner tests cover the child's cwd/group/metadata URL, stay-up and
SIGTERM in a real subprocess, SIGKILL of a SIGTERM-ignoring group, and the
marker. The opt-in live test now checks resume, Running, suspend and a second
resume instead of a Completed phase. design.md and tasks.md state the
lifecycle; the module audit manifest counts plane.doctor.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Maintainer decisions 2026-09-29: ax (kind + Agent Substrate + ax controller on
this host) is the only executor; the Run layer carries dependencies, artifacts,
budgets and attribution until ax has them. Requirement 7 declares routing per
Task (one Model per Task, no runtime switching) and a token attribution table
from provider usage fields. Task 0 (bring-up, doctor), runner artifact delivery,
ax-backed reconciler with a fake-ax test seam. harnessx-removal brief records
the HarnessX inventory; removal is gated on the increment's measurement.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Nine tasks in dependency order: manifests, budget metering, runner contract,
reconciler and CLI, repo-facts, verify and agree, the increment's manifests, the
measurement against the recorded batched check, handover.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
