fix(plane): ax task lifecycle -- resume, stay-up runner, delivery as completion

The first live run on the local ax cluster and ax's source (docs/runner.md,
docs/sandbox.md, internal/controller/reconciler.go, internal/server/server.go)
showed a lifecycle the task-3/4 code did not model:

- `ax apply` creates a Task Suspended (Ready=False, TaskSuspended); it runs only
  after `ax resume task`. The first resume of a new image can fail with
  DeadlineExceeded while Agent Substrate builds the golden snapshot, and ax's
  server then stores phase Failed; a later resume succeeds.
- ax has no Completed phase and never reads a command's exit back.
- The runner is PID 1 and must keep serving after the command exits; stop and
  suspend deliver SIGTERM, to be forwarded to the command's process group.
  Our runner imported the task in-process and exited, which stopped the
  container ("Container guest is stopped") and made every resume time out.
- ax's controller probes /readyz?check=workspace; the query string made our
  handler answer 404.

Runner (Req 4.1-4.4): spec.command runs as a child
`python -m openultrasast.plane.tasks.<name>` in its own process group, cwd the
first workspace, AX_METADATA_URL set; ax's metadata paths are served. After the
child exits the runner reads summary.json (a crash without one becomes failed
with the exit code and stderr tail), delivers the output directory, writes
/workspace/.ousast-state/<task>.done.json and serves until SIGTERM (forwarded
to the group, SIGKILL after OUSAST_TERM_GRACE, exit 0). A boot with the marker
never reruns the command; a failed delivery is retried alone. Workspaces are
prepared on the first boot only. AX_RUNNER_EXIT_AFTER_COMMAND=1 is the
unit-test escape.

Reconciler (Req 2.2, 2.3, 3.1-3.3): `ax resume task` after apply, retrying
DeadlineExceeded/Unavailable with backoff up to OUSAST_RESUME_TIMEOUT (900 s);
completion is the artifact delivery; Failed after a resume went through, a
vanished task, or no delivery within OUSAST_TASK_TIMEOUT (7200 s) fail the
task with ax's condition message; `ax delete task` follows. Ax gains resume()
and phase() returns the condition message. doctor moves to plane/doctor.py
(tasks.md task 0 allows the sibling) to keep reconciler.py under 500 lines.

Tests: the fake ax follows the real lifecycle (Suspended, a failing first
resume that leaves Failed, Running forever, delivery only after a resume went
through), plus Failed-with-message, vanish, resume-timeout and task-timeout
scenarios; runner tests cover the child's cwd/group/metadata URL, stay-up and
SIGTERM in a real subprocess, SIGKILL of a SIGTERM-ignoring group, and the
marker. The opt-in live test now checks resume, Running, suspend and a second
resume instead of a Completed phase. design.md and tasks.md state the
lifecycle; the module audit manifest counts plane.doctor.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

tests/plane_stub_task.py

@@ -1,34 +1,57 @@
 """A stub task module for the runner tests: records its contract, writes one artifact and ``summary.json``.
 
-Registered by the tests as ``openultrasast.plane.tasks.stub_task``; ``STUB_MODE`` picks the outcome
-(``done``, ``failed``, ``unfinished``, ``crash``, ``nosummary``, ``exit``).
+The tests copy it into a throwaway package (``AX_RUNNER_TASK_PACKAGE``) so the runner starts it the way it starts
+a real task, ``python -m <package>.stub_task``. ``STUB_MODE`` picks the outcome (``done``, ``failed``,
+``unfinished``, ``crash``, ``nosummary``, ``exit``, ``sleep``); ``STUB_COUNTER`` names a file that gets one line
+per run.
 """
 
 from __future__ import annotations
 
 import json
 import os
+import signal
+import subprocess
+import sys
+import time
 import urllib.request
 from pathlib import Path
 
 
+def _get(url: str) -> tuple[int, str]:
+    with urllib.request.urlopen(url, timeout=5) as response:
+        return int(response.status), response.read().decode("utf-8")
+
+
 def main(argv: list[str]) -> None:
     out = Path(os.environ["OUSAST_OUTPUT_DIR"])
     out.mkdir(parents=True, exist_ok=True)
     mode = os.environ.get("STUB_MODE", "done")
+    counter = os.environ.get("STUB_COUNTER")
+    if counter:
+        with open(counter, "a", encoding="utf-8") as handle:
+            handle.write(f"{os.getpid()}\n")
     workspace = Path(os.environ["OUSAST_WORKSPACE_DIR"])
-    readyz: int | None = None
     port = os.environ.get("AX_RUNNER_BOUND_PORT")
-    if port:
-        with urllib.request.urlopen(f"http://127.0.0.1:{port}/readyz", timeout=5) as response:
-            readyz = int(response.status)
+    readyz = _get(f"http://127.0.0.1:{port}/readyz")[0] if port else None
+    metadata = os.environ.get("AX_METADATA_URL")
     record = {
         "argv": argv,
         "env": {k: v for k, v in os.environ.items() if k.startswith("OUSAST_")},
         "workspace_files": sorted(p.relative_to(workspace).as_posix() for p in workspace.rglob("*") if p.is_file()),
         "readyz": readyz,
+        "cwd": os.getcwd(),
+        "pid": os.getpid(),
+        "pgid": os.getpgid(0),
+        "metadata_url": metadata,
+        "metadata_task": _get(f"{metadata}/metadata/v1alpha1/ax/task")[1] if metadata else None,
     }
     (out / "facts.json").write_text(json.dumps(record, indent=2, sort_keys=True), encoding="utf-8")
+    if mode == "sleep":  # ignore SIGTERM so only the runner's SIGKILL ends the group; the grandchild inherits it
+        signal.signal(signal.SIGTERM, signal.SIG_IGN)
+        grandchild = subprocess.Popen(["sleep", "60"])
+        (out / "pids.json").write_text(json.dumps([os.getpid(), grandchild.pid]), encoding="utf-8")
+        time.sleep(60)
     if mode == "crash":
         raise RuntimeError("stub exploded")
     if mode == "exit":
@@ -37,3 +60,7 @@ def main(argv: list[str]) -> None:
         return
     summary = {"status": mode, "units_done": 1, "units_total": 1, "usd": 0.0, "calls": 0, "usage": {}}
     (out / "summary.json").write_text(json.dumps(summary), encoding="utf-8")
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])