diff --git a/.kiro/specs/ai-service-plane/tasks.md b/.kiro/specs/ai-service-plane/tasks.md index a1b2dd8..f7cabba 100644 --- a/.kiro/specs/ai-service-plane/tasks.md +++ b/.kiro/specs/ai-service-plane/tasks.md @@ -139,7 +139,7 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level `benchmarks/independent/plane-increment-2.json` next to the first. - _Requirements: 6.2, 6.4_ -- [ ] 9. Handover +- [x] 9. Handover - `ops/README.md`: how to run a Run locally and what an ax deployment would need (Kubernetes, Agent Substrate, the runner image, the provider extension); memory note updated; scripts that the tasks replaced marked as reference-only in their docstrings. diff --git a/benchmarks/independent/score_batched.py b/benchmarks/independent/score_batched.py index 40c8b8c..bf9a9cb 100644 --- a/benchmarks/independent/score_batched.py +++ b/benchmarks/independent/score_batched.py @@ -1,4 +1,8 @@ -"""Exploratory score of a batched-verifier run against the population's cases (protocol v2 matching). +"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax +through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/ +plane-increment-2.json). It stays as the reference behaviour the plane was measured against. + +Exploratory score of a batched-verifier run against the population's cases (protocol v2 matching). The batched verifier writes one vulnerable-pin result per case (and a fixed-pin recheck with --fixed), not the six pins protocol v2 scores, so this applies the same matching rule (`evaluate.matches_v2`) to what exists: diff --git a/benchmarks/independent/verify_batched.py b/benchmarks/independent/verify_batched.py index 01dc3ea..d406994 100644 --- a/benchmarks/independent/verify_batched.py +++ b/benchmarks/independent/verify_batched.py @@ -1,4 +1,8 @@ -"""Stage 2, second design: triage per file, verify per file, report only what two passes agree on. +"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax +through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/ +plane-increment-2.json). It stays as the reference behaviour the plane was measured against. + +Stage 2, second design: triage per file, verify per file, report only what two passes agree on. The first verifier (`verify_sinks.py`) asked one 6-step tool hunt per candidate function. Measured on pgAdmin (verifier-check-2026-09-29.json): $0.014-0.020 per candidate, and on 180 candidates verified twice the two runs diff --git a/ops/README.md b/ops/README.md index 7f3e6b8..fd42e0d 100644 --- a/ops/README.md +++ b/ops/README.md @@ -369,3 +369,7 @@ recomputed after admission and never reuses a cached model decision. Pre-push does not implicitly read the working tree's `.env`. Supply any configured credential variable in the Git process environment. Explicit model configuration is read under the same analysis deadline. + +## The agentic service plane on ax + +Model-driven pipeline stages run as tasks on google/ax over Agent Substrate (`.kiro/specs/ai-service-plane/`). Bring-up, the smoke Run, the measured footprint and the lessons from the live cluster are in [`ops/ax/README.md`](ax/README.md); `ousast plane run|status|doctor|workspaces` drives it.