diff --git a/.kiro/specs/plane-on-kubernetes/design.md b/.kiro/specs/plane-on-kubernetes/design.md index d5e7277..f34fab5 100644 --- a/.kiro/specs/plane-on-kubernetes/design.md +++ b/.kiro/specs/plane-on-kubernetes/design.md @@ -254,6 +254,13 @@ containerd; `Dockerfile:1-7` already argues against it). a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays `alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such. + **Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.** + With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor + requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS + 1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so + group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether + Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the + k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path. - **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and `docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose `engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`. diff --git a/.kiro/specs/plane-on-kubernetes/tasks.md b/.kiro/specs/plane-on-kubernetes/tasks.md index d394d63..e0e4317 100644 --- a/.kiro/specs/plane-on-kubernetes/tasks.md +++ b/.kiro/specs/plane-on-kubernetes/tasks.md @@ -10,7 +10,7 @@ its kind proof and the record it writes under `benchmarks/measurements/-k8 go-ahead** with a ceiling; steps that need a hand on a cluster that does not exist yet are marked **manual**. - [ ] 1. Profile and literals (design section 1; open-question checks 2, 3, 7) -- [ ] 1.1 `PlaneProfile` +- [x] 1.1 `PlaneProfile` - `src/openultrasast/plane/profile.py`: frozen dataclass with the fields of design section 1; `load_profile()` reads `ops/k8s/profiles/.toml` (a name or a path; default `kind`), then applies `OUSAST_` overrides; rejects an unknown key, a secret-looking key (`*_KEY`, `*_SECRET`), an `exec` @@ -19,7 +19,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi - `tests/test_plane_profile.py`: defaults; file then env precedence; each rejection names the field; both committed profiles load; no field holds a value matching a credential pattern. - _Requirements: 1.1_ -- [ ] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py` +- [x] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py` - `doctor(profile)`: context from `profile.kube_context`; the registry check resolves the `images` file's runner reference (`docker manifest inspect` for a remote registry, `/v2/_catalog` for a local one); `open_store` verifies `profile.memory`; prints every configured address (Req 1.2). `Egress(context=...)`, @@ -29,46 +29,46 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi profile's context, never a built one. - Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile. - _Requirements: 1.1, 1.2_ -- [ ] 1.3 Host-literal scan test +- [x] 1.3 Host-literal scan test - `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does) over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`, `localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged `ax-tunnel`; fails listing `path:line`. - Evidence: the test passes on this commit and fails when a literal is reinserted (shown once in the record). - _Requirements: 1.3_ -- [ ] 1.4 `up.sh` by profile +- [x] 1.4 `up.sh` by profile - `ops/ax/up.sh`: `KO_DOCKER_REPO` and the context from `kind.toml` (read with `python -m openultrasast.plane.profile --print registry|kube_context`); writes `kind-images.json` instead of `runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3). - Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes. - _Requirements: 1.1, 2.4_ -- [ ] 1.5 Placement probe (open question 2, decided before the engine work) +- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any` - `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after. - Evidence: `record.json` says whether Substrate places by resource fit (`placement: fit|any`). `any` selects the design's fallback for group 5 and is written into `design.md` section 4 before group 5 starts. - _Requirements: 4.1_ -- [ ] 1.6 Reconciler bound (open question 7) +- [x] 1.6 Reconciler bound (open question 7) -- 489 lines after 1.2; projected 514 during group 2: the bound moves to 550 in 2.4 - Record `wc -l src/openultrasast/plane/reconciler.py` (477 at 17ac7ad) and the lines task 2.4 may add (receiver wiring removed: `-12`, `Delivery` calls: `+8` estimated). If the estimate crosses 500, the bound in `tests/test_plane_reconciler.py:803` moves to 550 in task 2.4 with the reason in the assertion message. - Evidence: the numbers in `record.json`. - _Requirements: 2.1_ -- [ ] 1.7 gvisor tarball reachability (open question 3) **manual** +- [x] 1.7 gvisor tarball reachability (open question 3) **manual** -- documented as planned in `docs/deployment.md` B.1; the live check waits for k3s - `docs/deployment.md` (B.1, planned): the k3s node must fetch `gs://gvisor/releases/nightly/2026-09-02//gvisor.tar.zstd` (`substrate/manifests/ate-install/sandboxconfig-gvisor.yaml`) or a mirrored copy named in a copied SandboxConfig; the check is `kubectl -n ate-system logs` of the worker pod's atelet fetch line, run by hand on the day k3s exists. Labelled planned until then. - _Requirements: 7.2_ -- [ ] 1.8 Kind proof and record 01 +- [x] 1.8 Kind proof and record 01 - `ousast plane doctor --profile kind` passes every line; `ousast plane doctor --profile k3s` fails only on the unreachable context (the expected line); `test_plane_literals` 0 offenders; the probe of 1.5; the bound of 1.6. - `benchmarks/measurements/-k8s-01-profile/record.json`. - _Requirements: 1.1, 1.2, 1.3, 7.1_ - [ ] 2. Store delivery, receiver-less egress (design section 3) -- [ ] 2.1 Presigned URLs and the store run directory +- [x] 2.1 Presigned URLs and the store run directory - `src/openultrasast/plane/delivery.py`: `Delivery(store, run)` mints presigned PUT for `runs///output.tar` and GET for `runs///` (boto3 `generate_presigned_url` on `S3Client.sdk`, expiry `OUSAST_TASK_TIMEOUT + 600`); `delivered(task)` HEADs the tar; `collect(task)` @@ -78,7 +78,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi - `tests/test_plane_delivery.py`: against a local S3-shaped fake (PUT/GET/HEAD by key); URLs never appear in logs; expiry; member rejection; declared-only re-put; state seeding. - _Requirements: 3.1, 2.1_ -- [ ] 2.2 Runner: `delivery` in the start body +- [x] 2.2 Runner: `delivery` in the start body - `src/openultrasast/plane/runner.py`: `StartGate.offer` validates an optional `delivery {put, inputs}` (https, hostname, no echo); `_deliver_output` PUTs the tar to `delivery.put` (same retry and `DeliveryError` semantics); `fetch_inputs` GETs from `delivery.inputs[name]`; `OUSAST_ARTIFACT_URL` and `OUSAST_ARTIFACT_DIAL` @@ -86,14 +86,14 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi - `tests/test_plane_runner.py`: accepted body with `delivery`; 400 on an `http` or IP URL; PUT to a local server; inputs fetched from presigned-shaped URLs; the marker and retry-only boot unchanged. - _Requirements: 3.1_ -- [ ] 2.3 Egress without the receiver +- [x] 2.3 Egress without the receiver - `src/openultrasast/plane/egress.py`: `policy_for(task, workspaces, model, store_host)` emits one `tlsPassthrough` rule on 443 over the sorted union of store host, Git hosts, Model hosts; no `http` rule; an IP store host raises naming `S3_ENDPOINT`. - `tests/test_plane_egress.py`: for every committed template under `plane/tasks/` the rendered policy is the expected set; the IP refusal. - _Requirements: 3.2_ -- [ ] 2.4 Reconciler on `Delivery` +- [x] 2.4 Reconciler on `Delivery` - `src/openultrasast/plane/reconciler.py`: `run()` opens the store first (no fallback), builds `Delivery`, passes `delivery` to `router.starter`/`start_task` (body extended in `router.py`), `_await` on `Delivery.delivered`, outcome from the collected `summary.json`; the receiver thread is kept behind @@ -101,7 +101,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi - `tests/test_plane_reconciler.py`: the ax fake run completes through the delivery fake; a rerun from an empty results root resumes from the store's `state.json`. - _Requirements: 2.1, 3.1_ -- [ ] 2.5 Kind proof and record 02 +- [x] 2.5 Kind proof and record 02 - `ops/ax/smoke-run.sh` with `OUSAST_DELIVERY=store`: `facts.json` and `summary.json` arrive through the store; the actor's policy read back has one TLS rule naming the store host; `ousast plane status ax-e2e-smoke` prints the table. Instrument: the tar's byte size from the store equals the extracted files' total. diff --git a/benchmarks/measurements/2026-09-08-module-audit.json b/benchmarks/measurements/2026-09-08-module-audit.json index 06f5a39..2e9181f 100644 --- a/benchmarks/measurements/2026-09-08-module-audit.json +++ b/benchmarks/measurements/2026-09-08-module-audit.json @@ -1,7 +1,7 @@ { - "modules": 155, + "modules": 157, "counts": { - "load_bearing": 150, + "load_bearing": 152, "standalone_capability": 5 }, "orphaned": [], @@ -855,6 +855,8 @@ "learn.examples", "learn.experiments", "learn.program", + "plane.delivery", + "plane.doctor", "plane.egress", "plane.engine_alerts", "plane.generate", @@ -885,6 +887,14 @@ "plane.tasks.verify" ] }, + { + "module": "plane.delivery", + "classification": "load_bearing", + "reason": "reachable from gate, map_gate, pair_gate, cli", + "importers": [ + "plane.reconciler" + ] + }, { "module": "plane.doctor", "classification": "load_bearing", @@ -932,6 +942,7 @@ "reason": "ai-service-plane Req 1.1-1.4: ax-shaped Task/Workspace/Model and the Run kind, validated before anything runs", "importers": [ "learn.cli", + "plane.delivery", "plane.egress", "plane.generate", "plane.harvest", @@ -954,11 +965,25 @@ "learn.examples", "learn.experiments", "learn.program", + "plane.delivery", + "plane.doctor", "plane.generate", + "plane.reconciler", "plane.tasks.loop", "plane.tasks.remember" ] }, + { + "module": "plane.profile", + "classification": "load_bearing", + "reason": "python -m openultrasast.plane.profile", + "importers": [ + "cli", + "plane.doctor", + "plane.generate", + "plane.reconciler" + ] + }, { "module": "plane.reconciler", "classification": "load_bearing", diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-k3s.txt b/benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-k3s.txt new file mode 100644 index 0000000..a413efd --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-k3s.txt @@ -0,0 +1,5 @@ +[ok] profile k3s: exec=remote kube_context=ousast-k3s registry=ghcr.io/norandom images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=ghcr-pull +[FAIL] cluster ousast-k3s: context ousast-k3s unreachable (error: context "ousast-k3s" does not exist); docs/deployment.md lists the prerequisites +[FAIL] runner image in ghcr.io/norandom: images file /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json: No such file or directory; docs/deployment.md lists the prerequisites +[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified) +exit=1 seconds=1.1 diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-kind.txt b/benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-kind.txt new file mode 100644 index 0000000..10846e1 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-kind.txt @@ -0,0 +1,7 @@ +[ok] profile kind: exec=local kube_context=kind-ousast registry=localhost:5001 images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none) +[ok] cluster kind-ousast: reachable +[ok] agent substrate (ate-system): 9 pods ready +[ok] ax controller (ax-system): 4 pods ready +[ok] runner image in localhost:5001: ousast-runner@sha256:95edc7ed5621... present in localhost:5001 +[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified) +exit=0 seconds=1.4 diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/literal-reinserted-fail.txt b/benchmarks/measurements/2026-10-02-k8s-01-profile/literal-reinserted-fail.txt new file mode 100644 index 0000000..470e66f --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/literal-reinserted-fail.txt @@ -0,0 +1,19 @@ +F.. [100%] +=================================== FAILURES =================================== +________ test_no_host_literal_outside_the_profile_and_the_kind_bring_up ________ + + def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None: + kind = load_profile("kind", environ={}) + found = offenders(_files(), kind.registry) +> assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found) +E AssertionError: host-only literals (Req 1.3): +E src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration +E assert ['src/openult...emonstration'] == [] +E +E Left contains one more item: 'src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration' +E Use -v to get more diff + +tests/test_plane_literals.py:81: AssertionError +=========================== short test summary info ============================ +FAILED tests/test_plane_literals.py::test_no_host_literal_outside_the_profile_and_the_kind_bring_up +1 failed, 2 passed in 0.26s diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/literal-restored-pass.txt b/benchmarks/measurements/2026-10-02-k8s-01-profile/literal-restored-pass.txt new file mode 100644 index 0000000..73decde --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/literal-restored-pass.txt @@ -0,0 +1,2 @@ +... [100%] +3 passed in 0.23s diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-big.yaml b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-big.yaml new file mode 100644 index 0000000..3693ca4 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-big.yaml @@ -0,0 +1,16 @@ +apiVersion: ax.io/v1alpha1 +kind: Workspace +metadata: {name: ousast-probe-big-ws, atespace: default} +spec: + files: + - {path: "hello.py", content: "def hello():\n return greet()\n"} +--- +apiVersion: ax.io/v1alpha1 +kind: Task +metadata: {name: ousast-probe-big, atespace: default} +spec: + image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + command: ["repo-facts"] + env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}] + resources: {requests: {cpu: "250m", memory: "2560Mi"}, limits: {cpu: "1", memory: "3Gi"}} + workspaces: [{name: ousast-probe-big-ws, path: "/workspace"}] diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-small.yaml b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-small.yaml new file mode 100644 index 0000000..3b86c99 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-small.yaml @@ -0,0 +1,16 @@ +apiVersion: ax.io/v1alpha1 +kind: Workspace +metadata: {name: ousast-probe-small-ws, atespace: default} +spec: + files: + - {path: "hello.py", content: "def hello():\n return greet()\n"} +--- +apiVersion: ax.io/v1alpha1 +kind: Task +metadata: {name: ousast-probe-small, atespace: default} +spec: + image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + command: ["repo-facts"] + env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}] + resources: {requests: {cpu: "250m", memory: "256Mi"}, limits: {cpu: "1", memory: "512Mi"}} + workspaces: [{name: ousast-probe-small-ws, path: "/workspace"}] diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/probe-pool.yaml b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/probe-pool.yaml new file mode 100644 index 0000000..157f6cb --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/probe-pool.yaml @@ -0,0 +1,26 @@ +# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by +# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU, +# POOL_MEMORY (ops/k8s/profiles/.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a +# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every +# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of +# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl). +apiVersion: ate.dev/v1alpha1 +kind: WorkerPool +metadata: + name: probe-pool + namespace: ax-system + labels: + workload: ousast +spec: + replicas: 1 + workerImage: localhost:5001/ateom-gvisor-715889664656de67e44382a8d6ab981d@sha256:220f1cf13ea485b73a2e4270fa544901a18ae826d8f22bd54d42761944a41c5c + template: + nodeSelector: + ate.dev/substrate-version: "4f9c293" + resources: + limits: + cpu: "2" + memory: 3Gi + requests: + cpu: 250m + memory: 3Gi diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-before.json b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-before.json new file mode 100644 index 0000000..79d9783 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-before.json @@ -0,0 +1,78 @@ +{ + "workers": [ + { + "ip": "10.244.0.22", + "labels": { + "workload": "ousast" + }, + "metadata": { + "createTime": "2026-09-29T16:32:08.594663703Z", + "name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6", + "uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e", + "updateTime": "2026-10-02T05:24:59.069304031Z", + "version": "1758" + }, + "nodeName": "ousast-control-plane", + "sandboxClass": "gvisor", + "status": { + "capacity": { + "actors": 1000, + "resources": { + "limits": [ + { + "name": "cpu", + "quantity": "1" + }, + { + "name": "memory", + "quantity": "1536Mi" + } + ] + } + }, + "state": "WORKER_STATE_ACTIVE" + }, + "workerNamespace": "ax-system", + "workerPod": "ousast-pool-6656b55f5b-9wbrw", + "workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6", + "workerPool": "ousast-pool" + }, + { + "ip": "10.244.0.28", + "labels": { + "workload": "ousast" + }, + "metadata": { + "createTime": "2026-10-02T18:29:30.486026311Z", + "name": "1407a421-7dcf-4ad7-b999-70a4624bf890", + "uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800", + "updateTime": "2026-10-02T18:29:30.551668391Z", + "version": "2" + }, + "nodeName": "ousast-control-plane", + "sandboxClass": "gvisor", + "status": { + "capacity": { + "actors": 1000, + "resources": { + "limits": [ + { + "name": "cpu", + "quantity": "2" + }, + { + "name": "memory", + "quantity": "3Gi" + } + ] + } + }, + "state": "WORKER_STATE_ACTIVE" + }, + "workerNamespace": "ax-system", + "workerPod": "probe-pool-5dffd74bdc-dqlww", + "workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890", + "workerPool": "probe-pool" + } + ] +} diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-big.json b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-big.json new file mode 100644 index 0000000..cd90d21 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-big.json @@ -0,0 +1,81 @@ +{ + "workers": [ + { + "ip": "10.244.0.22", + "labels": { + "workload": "ousast" + }, + "metadata": { + "createTime": "2026-09-29T16:32:08.594663703Z", + "name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6", + "uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e", + "updateTime": "2026-10-02T18:29:45.281958347Z", + "version": "1761" + }, + "nodeName": "ousast-control-plane", + "sandboxClass": "gvisor", + "status": { + "allocated": { + "actors": 1 + }, + "capacity": { + "actors": 1000, + "resources": { + "limits": [ + { + "name": "cpu", + "quantity": "1" + }, + { + "name": "memory", + "quantity": "1536Mi" + } + ] + } + }, + "state": "WORKER_STATE_ACTIVE" + }, + "workerNamespace": "ax-system", + "workerPod": "ousast-pool-6656b55f5b-9wbrw", + "workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6", + "workerPool": "ousast-pool" + }, + { + "ip": "10.244.0.28", + "labels": { + "workload": "ousast" + }, + "metadata": { + "createTime": "2026-10-02T18:29:30.486026311Z", + "name": "1407a421-7dcf-4ad7-b999-70a4624bf890", + "uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800", + "updateTime": "2026-10-02T18:29:30.551668391Z", + "version": "2" + }, + "nodeName": "ousast-control-plane", + "sandboxClass": "gvisor", + "status": { + "capacity": { + "actors": 1000, + "resources": { + "limits": [ + { + "name": "cpu", + "quantity": "2" + }, + { + "name": "memory", + "quantity": "3Gi" + } + ] + } + }, + "state": "WORKER_STATE_ACTIVE" + }, + "workerNamespace": "ax-system", + "workerPod": "probe-pool-5dffd74bdc-dqlww", + "workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890", + "workerPool": "probe-pool" + } + ] +} diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-small.json b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-small.json new file mode 100644 index 0000000..566d053 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-small.json @@ -0,0 +1,81 @@ +{ + "workers": [ + { + "ip": "10.244.0.22", + "labels": { + "workload": "ousast" + }, + "metadata": { + "createTime": "2026-09-29T16:32:08.594663703Z", + "name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6", + "uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e", + "updateTime": "2026-10-02T18:29:53.001457323Z", + "version": "1763" + }, + "nodeName": "ousast-control-plane", + "sandboxClass": "gvisor", + "status": { + "allocated": { + "actors": 1 + }, + "capacity": { + "actors": 1000, + "resources": { + "limits": [ + { + "name": "cpu", + "quantity": "1" + }, + { + "name": "memory", + "quantity": "1536Mi" + } + ] + } + }, + "state": "WORKER_STATE_ACTIVE" + }, + "workerNamespace": "ax-system", + "workerPod": "ousast-pool-6656b55f5b-9wbrw", + "workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6", + "workerPool": "ousast-pool" + }, + { + "ip": "10.244.0.28", + "labels": { + "workload": "ousast" + }, + "metadata": { + "createTime": "2026-10-02T18:29:30.486026311Z", + "name": "1407a421-7dcf-4ad7-b999-70a4624bf890", + "uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800", + "updateTime": "2026-10-02T18:30:04.825058843Z", + "version": "4" + }, + "nodeName": "ousast-control-plane", + "sandboxClass": "gvisor", + "status": { + "capacity": { + "actors": 1000, + "resources": { + "limits": [ + { + "name": "cpu", + "quantity": "2" + }, + { + "name": "memory", + "quantity": "3Gi" + } + ] + } + }, + "state": "WORKER_STATE_ACTIVE" + }, + "workerNamespace": "ax-system", + "workerPod": "probe-pool-5dffd74bdc-dqlww", + "workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890", + "workerPool": "probe-pool" + } + ] +} diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/record.json b/benchmarks/measurements/2026-10-02-k8s-01-profile/record.json new file mode 100644 index 0000000..b289da2 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/record.json @@ -0,0 +1,132 @@ +{ + "record": "k8s-01-profile", + "spec": "plane-on-kubernetes", + "tasks": ["1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8"], + "date": "2026-10-02", + "host": "the maintainer's laptop: 7.9 GiB, single-node kind cluster, RustFS store reachable over https", + "commits": { + "1.1": "8eac1b2", + "1.2": "77a03ba", + "1.3": "36f5927", + "1.4": "8099b29", + "1.5": "86ec237", + "1.6": "2afa20e", + "1.7": "2cca03c", + "1.8": "the commit that adds this line (git log: plane(1.8))" + }, + "images": { + "runner_before": "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884", + "runner_after_up_sh": "localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb", + "pin_file": "ops/k8s/profiles/kind-images.json (written by ops/ax/up.sh; the templates re-pinned to it, 128 image lines)" + }, + "instrument_checks": { + "code_under_test": "PYTHONPATH=/src first; `python -c 'import openultrasast; print(openultrasast.__file__)'` printed the worktree path before every gate", + "store_reached": { + "how": "scratch store_probe.py: open_store('s3://sast-memory') through the worktree code (verify_bucket runs), then read the probe object", + "open_and_verify_seconds": 0.62, + "probe_key": "_probe/select.jsonl", + "probe_bytes": 24, + "probe_version_id_present": true + }, + "smoke_delivery_sizes_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291, "task.yaml": 1148}, + "registry_resolution": "doctor HEADs /v2/ousast-runner/manifests/ on the kind registry: HTTP 2xx for the new digest" + }, + "doctor": { + "kind": { + "command": "ousast plane doctor --profile kind", + "exit": 0, + "seconds": 1.4, + "lines": [ + "[ok] profile kind: exec=local kube_context=... registry=... images=... router_url=(tunnel) ax_server=(tunnel) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)", + "[ok] cluster : reachable", + "[ok] agent substrate (ate-system): 9 pods ready", + "[ok] ax controller (ax-system): 4 pods ready", + "[ok] runner image in : ousast-runner@sha256:95edc7ed5621... present", + "[ok] memory store s3://sast-memory: opens (an S3 bucket is verified)" + ], + "file": "doctor-kind.txt" + }, + "k3s": { + "command": "ousast plane doctor --profile k3s", + "exit": 1, + "seconds": 1.1, + "failing_lines": [ + "cluster ousast-k3s: context does not exist (expected: the cluster does not exist yet)", + "runner image in ghcr.io/norandom: images file ops/k8s/profiles/images.json missing (expected: the release asset is published by task 4.1)" + ], + "passing_lines": ["profile k3s: every address reported", "memory store s3://sast-memory: opens"], + "note": "two expected failures, not one: the images file is the release asset and does not exist before group 4; the namespaces are not checked behind an unreachable context, so no third line fails for the same cause", + "file": "doctor-k3s.txt" + } + }, + "literal_scan": { + "test": "tests/test_plane_literals.py", + "scope": "src/, plane/, ops/ (minus ops/ax/up.sh, ops/k8s/profiles/), docs/", + "patterns": ["kind context", "kind registry", "kind gateway 172.19.", "loopback with port", "port-forward"], + "exemptions": [ + "router.py for loopback-with-port and port-forward", + "lines tagged ax-tunnel or loopback (two: the runner's own health URL, the Joern session's query URL)", + "digest-pinned image: lines under plane/tasks/ whose registry equals the kind profile's (a third test asserts every one does)" + ], + "offenders_at_1_3": 0, + "offenders_before_1_3": 23, + "reinsertion_demo": { + "inserted": "a kind-context literal appended to src/openultrasast/plane/workspaces.py", + "pytest_exit_with_literal": 1, + "named": "src/openultrasast/plane/workspaces.py:37: [kind context]", + "restored": "byte for byte (cmp)", + "pytest_exit_restored": 0, + "files": ["literal-reinserted-fail.txt", "literal-restored-pass.txt"] + } + }, + "up_sh_rerun": { + "command": "OUSAST_PYTHON=
ops/ax/up.sh", + "exit": 0, + "seconds": 38, + "kept": ["cluster", "substrate", "egress gateway", "ax control plane", "worker pool ousast-pool (2/2 ready)", "ax CLI (newer than the checkout)"], + "done": ["receiver Service re-applied", "runner image rebuilt and pushed (new digest above)", "kind-images.json written", "smoke Task applied and resumed"], + "first_attempt_failure": "`profile --print images` answered `no image named ''`; fixed in profile._lookup (an images path with no sub-key is the file path) and covered by test_plane_profile", + "disk_gb_free_before_after": [1.7, 1.5], + "file": "up-sh-rerun.txt" + }, + "smoke_run_1_4": { + "command": "ops/ax/smoke-run.sh (receiver delivery: the path group 2 replaces)", + "exit": 0, + "seconds": 24, + "status": "done", + "delivered": "facts.json 451 B, summary.json 238 B, units.jsonl 291 B; summary.image = the new digest, units 2/2", + "file": "smoke-run-1.4.txt" + }, + "placement_probe_1_5": { + "script": "ops/ax/probe-placement.sh", + "pools": {"ousast-pool": "scaled to 1 worker, 1 CPU / 1536Mi", "probe-pool": "1 worker, 2 CPU / 3Gi, the already-built ateom-gvisor image (no ko build)"}, + "big_actor": {"requests_memory": "2560Mi", "limits_memory": "3Gi", "landed_on": "ousast-pool (1536Mi worker): status.allocated.actors=1; probe-pool 0"}, + "small_actor": {"requests_memory": "256Mi", "limits_memory": "512Mi", "landed_on": "ousast-pool: allocated.actors=1; probe-pool 0"}, + "placement": "any", + "consequence": "design.md section 4: group 5 takes the fallback (an engine atespace with engine-pool as its only pool if Substrate scopes pools by atespace; else a plain Kubernetes Job on k3s and alerts-engine on kind)", + "resume_latency_seconds": "first resume of each new template succeeded at once (golden snapshot built within the resume deadline); whole probe 46 s", + "cleanup": "probe tasks deleted, probe-pool deleted, ousast-pool back to 2/2 (verified by kubectl get workerpool)", + "files": ["probe/probe.log", "probe/workers-before.json", "probe/workers-ousast-probe-big.json", "probe/workers-ousast-probe-small.json", "probe/ousast-probe-big.yaml", "probe/ousast-probe-small.yaml", "probe/probe-pool.yaml"] + }, + "reconciler_bound_1_6": { + "lines_at_spec_commit_6bd44ca": 477, + "lines_after_1_2": 489, + "added_by_1_2": "the profile parameter of run() and its docstring (+12)", + "estimate_2_4": "Delivery wiring: open the store, build Delivery, pass the delivery body to the starter, delivered() and collect() in the await/execute path, state mirroring, the OUSAST_DELIVERY=receiver switch: about +25 while the receiver is kept (group 2 keeps both paths), about -20 when 2.6 removes the receiver wiring", + "projected_during_group_2": 514, + "projected_after_2_6": 494, + "decision": "the estimate crosses 500 while both paths coexist: task 2.4 moves the bound in tests/test_plane_reconciler.py to 550 with the reason in the assertion message, and 2.6 records the final count" + }, + "gvisor_tarball_1_7": { + "status": "planned, manual", + "where": "docs/deployment.md B.1", + "url": "gs://gvisor/releases/nightly/2026-09-02//gvisor.tar.zstd (sha256-pinned in substrate/manifests/ate-install/sandboxconfig-gvisor.yaml)", + "check": "the atelet's fetch line in `kubectl -n ate-system logs` of a worker pod on the k3s node; labelled planned until read there" + }, + "gates": { + "1.1": "pytest 2002 passed + module-audit manifest regenerated (test_module_audit 6 passed), ruff 0, format 0, mypy 0", + "1.2": "pytest 2006 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0", + "1.3": "pytest 2009 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0; mkdocs --strict 0", + "1.4": "see the commit message" + } +} diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/smoke-run-1.4.txt b/benchmarks/measurements/2026-10-02-k8s-01-profile/smoke-run-1.4.txt new file mode 100644 index 0000000..320649b --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/smoke-run-1.4.txt @@ -0,0 +1,9 @@ +run finished: done +memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none +run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke) + +TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD +facts - 0 n/a n/a n/a n/a [done] +---------------------------------------------------------------------------------------------------- +total - n/a n/a n/a n/a n/a +smoke exit=0 seconds=24 diff --git a/benchmarks/measurements/2026-10-02-k8s-01-profile/up-sh-rerun.txt b/benchmarks/measurements/2026-10-02-k8s-01-profile/up-sh-rerun.txt new file mode 100644 index 0000000..656c4f4 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-01-profile/up-sh-rerun.txt @@ -0,0 +1,58 @@ + +== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (20:35:35) +cluster exists; kept + +== agent substrate (ate-system) (20:35:35) +substrate ready; kept +NAME READY STATUS RESTARTS AGE +ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (12h ago) 3d +ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (12h ago) 3d +ate-controller-c95586684-g7nvn 1/1 Running 0 3d2h +atelet-4f9c293-9zpfg 1/1 Running 0 3d2h +atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d +atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (12h ago) 3d2h +postgres-0 2/2 Running 3 (3d2h ago) 3d2h +rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d2h ago) 3d2h +rustfs-bucket-init-nvfr6 0/1 Completed 0 3d2h + +== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (20:35:35) +gateway present; kept + +== ax control plane (ax-system) (20:35:35) +ax ready; kept +NAME READY STATUS RESTARTS AGE +ax-redis-86df786c87-pz5tp 1/1 Running 0 3d2h +ax-server-b479bdc76-hwkqb 1/1 Running 0 3d2h +ousast-pool-6656b55f5b-5brkn 1/1 Running 0 5m23s +ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d2h + +== artifact receiver Service (ousast-receiver.ax-system -> this host) (20:35:35) +service/ousast-receiver unchanged +endpointslice.discovery.k8s.io/ousast-receiver-host configured + +== worker pool (gVisor): pools.default of the profile (20:35:36) +pool ousast-pool exists (2/2 ready); kept +pod/ousast-pool-6656b55f5b-5brkn condition met +pod/ousast-pool-6656b55f5b-9wbrw condition met + +== runner image localhost:5001/ousast-runner (20:35:37) +sha256:f004a5ef8f71b1cb8a5880c509389c0569181af7b19170b574958cc3ef75cb1c +localhost:5001/ousast-runner:dev +runner image localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json +re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3) + +== ax CLI from the deployed checkout (a release CLI skews from the server) (20:36:05) +ax CLI newer than the checkout; kept + +== smoke task (20:36:05) +task.ax.io/ousast-smoke created +workspace.ax.io/ousast-smoke-ws unchanged +task.ax.io/ousast-smoke resumed +NAME ATESPACE PHASE ACTOR WORKER-IP AGE +ousast-smoke default Running ousast-smoke 10.244.0.29 5s + +== footprint (20:36:11) + total used free shared buff/cache available +Mem: 7877 4797 225 63 3464 3079 +kind-registry 53.77MiB / 7.693GiB 0.00% +up.sh exit 0 diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/egress-policy-read-back.json b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/egress-policy-read-back.json new file mode 100644 index 0000000..fcd7066 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/egress-policy-read-back.json @@ -0,0 +1,24 @@ +{ + "metadata": { + "atespace": "default", + "createTime": "2026-10-02T19:39:19.428045847Z", + "name": "default", + "uid": "741e52b7-aaf1-4a91-b4bf-fc3b20ba4a36", + "updateTime": "2026-10-02T19:39:19.428045847Z", + "version": "1" + }, + "rules": [ + { + "tlsPassthrough": { + "hostnames": [ + "files.because-security.com" + ], + "ports": { + "numbers": [ + 443 + ] + } + } + } + ] +} diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json new file mode 100644 index 0000000..78a32df --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json @@ -0,0 +1,83 @@ +{ + "record": "k8s-02-store-delivery", + "spec": "plane-on-kubernetes", + "tasks": ["2.1", "2.2", "2.3", "2.4", "2.5", "2.6"], + "date": "2026-10-02", + "commits": { + "2.1": "35d8032", + "2.2": "9506a18", + "2.3": "c49eae4", + "2.4": "605b7af", + "2.5": "the commit that adds this file (git log: plane(2.5))", + "receiver_removed": "pending: task 2.6 (paused 2026-10-03 by the maintainer's reprioritisation; the receiver path still exists behind OUSAST_DELIVERY=receiver)" + }, + "images": { + "runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019", + "built_by": "ops/ax/up.sh rerun (file up-sh-rebuild.txt): every step kept but the runner build; the image carries the 2.2 runner", + "disk_gb_free_after_build": 1.3 + }, + "instrument_checks": { + "code_under_test": "the worktree's src first on PYTHONPATH; `ousast` run as `python -m openultrasast.cli` from the worktree (ops/ax/smoke-run.sh with OUSAST_PYTHON)", + "store_read_back": { + "how": "scratch store_tar_check.py opens the store through the worktree code (verify_bucket runs), lists runs/ax-e2e-smoke/, downloads output.tar, compares its members with the files the reconciler extracted (file store-check.json)", + "tar_bytes": 10240, + "tar_members_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291}, + "extracted_files_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291}, + "members_equal_extracted": true, + "member_bytes_total": 980, + "note": "a tar is 512-byte blocks: 10240 B of tar carry 980 B of files; done.json records the tar's byte size" + }, + "no_receiver_in_the_task": "the rendered task.yaml is 996 B against 1148 B on the receiver path: no OUSAST_ARTIFACT_URL/DIAL in the Task's env; the start body carries the URLs" + }, + "smoke_runs": { + "store_1": { + "command": "ops/ax/smoke-run.sh (store delivery, the default after 2.4; OUSAST_DELIVERY unset)", + "exit": 0, + "seconds_submit_to_done": 26, + "tar_put_at": "2026-10-02T19:37:24+00:00 (the object's LastModified)", + "collected_at": "2026-10-02T19:37:26+00:00 (done.json finished)", + "put_to_collect_seconds": 2, + "poll_interval_seconds": 3, + "file": "smoke-run-store-1.txt" + }, + "seeded_rerun": { + "command": "the same, with the local results root deleted and the store's runs/ax-e2e-smoke/ intact", + "exit": 0, + "seconds": 2, + "what_happened": "state.json seeded from the store: every task done, nothing submitted to ax (plane-on-kubernetes 2.1, design section 2: a rerun from another machine resumes from the store)", + "file": "smoke-run-seeded-rerun.txt" + }, + "store_2": { + "command": "the same, after deleting runs/ax-e2e-smoke/ from the store (5 objects) so the task executes again, with the egress policy polled", + "exit": 0, + "seconds_submit_to_done": 24, + "tar_put_at": "2026-10-02T19:39:38+00:00", + "collected_at": "2026-10-02T19:39:40+00:00", + "put_to_collect_seconds": 2, + "file": "smoke-run-store-2.txt" + } + }, + "objects_delivered_to_the_store": { + "runs/ax-e2e-smoke/facts/output.tar": 10240, + "runs/ax-e2e-smoke/facts/facts.json": 451, + "runs/ax-e2e-smoke/facts/summary.json": 238, + "runs/ax-e2e-smoke/facts/done.json": 149, + "runs/ax-e2e-smoke/state.json": 350, + "note": "units.jsonl is extracted locally but not a declared output of the smoke Run, so it is not re-put" + }, + "egress_policy_read_back": { + "how": "kubectl ate get egress-policy ax-e2e-smoke-facts -a default -o json, polled while the actor lived (file egress-policy-read-back.json)", + "rules": [{"tlsPassthrough": {"hostnames": ["files.because-security.com"], "ports": {"numbers": [443]}}}], + "reading": "one TLS rule naming the store host, no http rule; the model-free smoke task names no Git host and no Model host", + "also": "Egress.apply reads every policy back and raises on a mismatch, so each completed Run is itself the equality check" + }, + "status_table": "ousast plane status ax-e2e-smoke printed the attribution table (facts: done, calls 0) in every run; the table is in the smoke-run-*.txt files", + "reconciler_lines": {"after_2_4": 549, "bound": 550, "note": "both paths coexist until 2.6; the receiver wiring goes then"}, + "gates": { + "2.1": "pytest 2020 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0", + "2.2": "pytest 2024 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0", + "2.3": "pytest 2027 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0", + "2.4": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0", + "2.5": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0 (first attempt 2026-10-02: 11 failed + 5 errors, all the FileStore 1 GiB free-space refusal at 0.97 GiB free; rerun after the disk was freed)" + } +} diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-seeded-rerun.txt b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-seeded-rerun.txt new file mode 100644 index 0000000..d5f3429 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-seeded-rerun.txt @@ -0,0 +1,9 @@ +run finished: done +memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none +run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke) + +TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD +facts - n/a n/a n/a n/a n/a [done] +---------------------------------------------------------------------------------------------------- +total - n/a n/a n/a n/a n/a +smoke exit=0 seconds=2 diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-1.txt b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-1.txt new file mode 100644 index 0000000..492952d --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-1.txt @@ -0,0 +1,9 @@ +run finished: done +memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none +run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke) + +TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD +facts - 0 n/a n/a n/a n/a [done] +---------------------------------------------------------------------------------------------------- +total - n/a n/a n/a n/a n/a +smoke exit=0 seconds=26 diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-2.txt b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-2.txt new file mode 100644 index 0000000..320649b --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-2.txt @@ -0,0 +1,9 @@ +run finished: done +memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none +run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke) + +TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD +facts - 0 n/a n/a n/a n/a [done] +---------------------------------------------------------------------------------------------------- +total - n/a n/a n/a n/a n/a +smoke exit=0 seconds=24 diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/store-check.json b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/store-check.json new file mode 100644 index 0000000..ef4a4ba --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/store-check.json @@ -0,0 +1,36 @@ +{ + "store": "s3://sast-memory", + "objects_under_run": { + "runs/ax-e2e-smoke/facts/done.json": 149, + "runs/ax-e2e-smoke/facts/facts.json": 451, + "runs/ax-e2e-smoke/facts/output.tar": 10240, + "runs/ax-e2e-smoke/facts/summary.json": 238, + "runs/ax-e2e-smoke/state.json": 350 + }, + "tar_bytes": 10240, + "tar_members": { + "facts.json": 451, + "summary.json": 238, + "units.jsonl": 291 + }, + "extracted_files": { + "facts.json": 451, + "units.jsonl": 291, + "summary.json": 238 + }, + "members_equal_extracted": true, + "member_bytes_total": 980, + "extracted_bytes_total": 980, + "done_json": { + "bytes": 10240, + "files": 3, + "finished": "2026-10-02T19:39:40+00:00", + "published": [ + "facts.json", + "summary.json" + ], + "status": "done", + "task": "facts" + }, + "tar_put_at_last_modified": "2026-10-02T19:39:38+00:00" +} diff --git a/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/up-sh-rebuild.txt b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/up-sh-rebuild.txt new file mode 100644 index 0000000..e231696 --- /dev/null +++ b/benchmarks/measurements/2026-10-02-k8s-02-store-delivery/up-sh-rebuild.txt @@ -0,0 +1,58 @@ + +== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (21:36:18) +cluster exists; kept + +== agent substrate (ate-system) (21:36:18) +substrate ready; kept +NAME READY STATUS RESTARTS AGE +ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (13h ago) 3d1h +ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (13h ago) 3d1h +ate-controller-c95586684-g7nvn 1/1 Running 0 3d3h +atelet-4f9c293-9zpfg 1/1 Running 0 3d3h +atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d1h +atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (13h ago) 3d3h +postgres-0 2/2 Running 3 (3d3h ago) 3d3h +rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d3h ago) 3d3h +rustfs-bucket-init-nvfr6 0/1 Completed 0 3d3h + +== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (21:36:18) +gateway present; kept + +== ax control plane (ax-system) (21:36:19) +ax ready; kept +NAME READY STATUS RESTARTS AGE +ax-redis-86df786c87-pz5tp 1/1 Running 0 3d3h +ax-server-b479bdc76-hwkqb 1/1 Running 0 3d3h +ousast-pool-6656b55f5b-5brkn 1/1 Running 0 66m +ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d3h + +== artifact receiver Service (ousast-receiver.ax-system -> this host) (21:36:19) +service/ousast-receiver unchanged +endpointslice.discovery.k8s.io/ousast-receiver-host configured + +== worker pool (gVisor): pools.default of the profile (21:36:20) +pool ousast-pool exists (2/2 ready); kept +pod/ousast-pool-6656b55f5b-5brkn condition met +pod/ousast-pool-6656b55f5b-9wbrw condition met + +== runner image localhost:5001/ousast-runner (21:36:21) +sha256:0501baf180b29f7aa7bbb8888bebabfc5d62429b7bdaa7f5a62b5b188f8ad5f6 +localhost:5001/ousast-runner:dev +runner image localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json +re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3) + +== ax CLI from the deployed checkout (a release CLI skews from the server) (21:36:32) +ax CLI newer than the checkout; kept + +== smoke task (21:36:32) +task.ax.io/ousast-smoke created +workspace.ax.io/ousast-smoke-ws unchanged +task.ax.io/ousast-smoke resumed +NAME ATESPACE PHASE ACTOR WORKER-IP AGE +ousast-smoke default Running ousast-smoke 10.244.0.29 4s + +== footprint (21:36:38) + total used free shared buff/cache available +Mem: 7877 5202 382 63 2903 2674 +kind-registry 32.45MiB / 7.693GiB 0.00% +up.sh exit 0 diff --git a/docs/deployment.md b/docs/deployment.md index 2ac3de6..2e1d645 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -31,11 +31,11 @@ flowchart LR sub["Agent Substrate (ate-system): atenet-router, atenet-egress"] act["Task actors in gVisor workers (WorkerPool ousast-pool)"] svc["Service ousast-receiver + EndpointSlice to the host"] - reg["Registry localhost:5001: runner image by digest"] + reg["Registry named by the profile: runner image by digest"] end mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")] rec -- "ax apply, resume, get, delete" --> ax - rec -- "EgressPolicy per actor; start request via port-forward" --> sub + rec -- "EgressPolicy per actor; start request through the kube-context tunnel" --> sub sub --> act act -- "HTTP by Service name through the gateway" --> svc svc --> rcv @@ -50,8 +50,8 @@ flowchart LR | How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` | | ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` | | Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` | -| Runner image | the kind-local registry `localhost:5001`, pinned by digest (`~/.cache/ousast/ax-src/runner-image`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` | -| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through a `kubectl port-forward` to `svc/atenet-router` | `src/openultrasast/plane/router.py` | +| Runner image | the registry the profile names (`registry` in `ops/k8s/profiles/kind.toml`: the kind-local one), pinned by digest in the profile's images file (`ops/k8s/profiles/kind-images.json`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` | +| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through the tunnel `router.py` opens to `svc/atenet-router` in the profile's kube context | `src/openultrasast/plane/router.py` | | Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://[/]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) | | Run state and artifacts | `$OUSAST_RESULTS/plane//` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) | @@ -80,7 +80,7 @@ files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`). The script is idempotent: it skips each step that is already done. In order, it: - creates the kind cluster (`KIND_CLUSTER_NAME`, default `ousast`) with its registry - (`KO_DOCKER_REPO`, default `localhost:5001`); + (`KO_DOCKER_REPO`, default: the `registry` of `ops/k8s/profiles/kind.toml`); - installs Agent Substrate; - applies the egress gateway if the install left it out (the agentgateway variant, one prebuilt image); - deploys the ax control plane; @@ -95,17 +95,20 @@ files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`). 2. **Check it.** ```bash - uv run ousast plane doctor + uv run ousast plane doctor --profile kind ``` - It runs four checks: + It checks the profile you pass (`--profile`, default `kind`; the profiles are in `ops/k8s/profiles/`): - - the kind context is reachable; + - its configured addresses, printed on the first line; + - the kube context is reachable; - every pod in `ate-system` is ready; - every pod in `ax-system` is ready; - - the registry at `localhost:5001` holds `ousast-runner`. + - the runner image of the profile's images file is in its registry (a local registry over its v2 + API, a remote one through `docker manifest inspect`); + - the memory store opens (an S3 bucket is verified). - Each failing line names `ops/ax/up.sh` as the fix. `doctor` is informational only. + Each failing line names the fix (`ops/ax/up.sh` for the kind profile). `doctor` is informational: `ousast plane run` does not call it. 3. **Run the end-to-end smoke Run.** @@ -165,8 +168,8 @@ The target cluster would do four things: - keep its memory in your S3 bucket. The provider key already travels only in the start request through `atenet-router`. That works the same -through a port-forward to any cluster. Everything else below is either a step you take or a code change -that is not made yet. +through the kube-context tunnel to any cluster. Everything else below is either a step you take or a code +change that is not made yet. ### B.1 Cluster with Agent Substrate, its egress gateway and ax @@ -186,6 +189,14 @@ that is not made yet. - TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts. - Deploy ax (`make deploy AX_IMAGE_REPO=` in the ax checkout) **after** B.2. - Apply a WorkerPool (B.6). Substrate installs none by itself ("no free workers"). +- **gVisor on the nodes (planned, checked by hand on the day k3s exists).** Substrate does not use a Kubernetes + `RuntimeClass`: the `ateom-gvisor` worker runs `runsc` itself, from the gVisor release tarball named in + `manifests/ate-install/sandboxconfig-gvisor.yaml` of the Substrate checkout + (`gs://gvisor/releases/nightly/2026-09-02//gvisor.tar.zstd`, sha256-pinned per architecture), which + the atelet fetches on the worker node. The node must reach that URL, or a copied `SandboxConfig` must name a + mirrored copy with the same sha256 and the ActorTemplates must name that config. The check is the atelet's + fetch line in `kubectl -n ate-system logs` of the worker pod; it is labelled planned until that log has been + read on a k3s node (plane-on-kubernetes task 1.7, design open question 3). ### B.2 ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` @@ -218,31 +229,23 @@ long Runs need a janitor that removes directories no live actor owns. - Pass `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`. They re-pin the Tasks they generate. - The committed templates under `plane/tasks/` and the generated `plane/tasks/validation-46.yaml` carry - `localhost:5001/ousast-runner@sha256:...`. They need the same re-pin before they run elsewhere. + the kind profile's registry and digest. They need the same re-pin before they run elsewhere. Regenerate them, or replace the reference. - Workers must be able to pull from the registry. Credentials, if any, are the cluster's concern. ax's default runner image on `gcr.io` needs credentials and is not used here. -- **Not yet configurable:** `ousast plane doctor` checks the registry at `localhost:5001` by name - (`src/openultrasast/plane/doctor.py:45`). So its fourth check fails on another cluster, even when the - image is in place. `doctor` is informational; `plane run` does not depend on it. +- `ousast plane doctor --profile ` looks up the runner reference of the profile's images file in + the profile's `registry` (implemented 2026-10-02, plane-on-kubernetes task 1.2). `doctor` is + informational; `plane run` does not depend on it. -### B.4 One configurable kube context +### B.4 One configurable kube context (implemented 2026-10-02) -The reconciler, the egress client and the router port-forward all use one kube context. Its name is -`kind-$KIND_CLUSTER_NAME` (`kind-ousast` by default). +The reconciler, the egress client and the router tunnel take the kube context from the profile. -**Not yet configurable**: the prefix `kind-` is hard-coded in these places: - -- `src/openultrasast/plane/doctor.py:39`; -- `src/openultrasast/plane/egress.py:115` and `:148`; -- `src/openultrasast/plane/router.py:145`; -- `ops/ax/up.sh:24` builds the same name. - -The planned change is one variable read in those three modules, for example `OUSAST_KUBE_CONTEXT`. - -Until then, there is one workaround to drive another cluster with the unchanged code. Rename its -kubeconfig context to that form: `kubectl config rename-context kind-` and -`KIND_CLUSTER_NAME=`. This is a workaround, not a supported configuration. +- The setting is `kube_context` in `ops/k8s/profiles/.toml`; `OUSAST_KUBE_CONTEXT` overrides it + (plane-on-kubernetes task 1.2). +- `doctor.py`, `egress.py` and `router.py` are handed the context. They build none themselves. +- `tests/test_plane_literals.py` fails on any module, manifest or page that names this host's cluster, + registry or addresses. The egress client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names). It must be able to use that context. @@ -333,10 +336,10 @@ Running the reconciler in-cluster is itself planned. Today it is a host process. | Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented | | S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) | | Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented | -| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates still name `localhost:5001` | +| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates name the kind profile's registry (`tests/test_plane_literals.py` checks it) | | Deployment to a cluster other than this host's kind cluster | **not done** | -| One configurable kube context | **planned**; `kind-` hard-coded in `doctor.py`, `egress.py`, `router.py` | -| `doctor` registry check against a configurable registry | **planned**; `localhost:5001` hard-coded in `doctor.py:45` | +| One configurable kube context | implemented (2026-10-02): `kube_context` in the profile, `OUSAST_KUBE_CONTEXT` overrides | +| `doctor` registry check against a configurable registry | implemented (2026-10-02): the profile's `registry` and images file | | Receiver as an in-cluster Deployment | **planned** (code change) | | Presigned-URL delivery to the S3 store | **follow-on**, not started | | Reconciler running in-cluster with Secrets injected into its environment | **planned** | @@ -373,7 +376,7 @@ flowchart LR ctl["ate-api-server, ate-controller, atelet"] rt["atenet-router"] eg["atenet-egress gateway"] - wp["WorkerPool: gVisor workers, runsc RuntimeClass, HPA (planned)"] + wp["WorkerPool: gVisor workers, runsc in the worker, HPA (planned)"] ewp["Engine WorkerPool: Joern as a task (planned)"] end end @@ -410,7 +413,7 @@ profile only. It does not appear in production. - It is a Deployment `ax-server` with a Redis beside it, in the namespace `ax-system` (`ax/deploy/ax-server.yaml` in the ax checkout). - Its controller runs with `--template=default-template --template-atespace=ax-system`. -- The `ax` CLI reaches it through a `kubectl port-forward` to `svc/ax-server` +- The `ax` CLI reaches it through a tunnel to `svc/ax-server` over the kube context (`ax/internal/tunnel/tunnel.go`). - So `ousast plane run` needs a kube context or an ingress, not a public endpoint. @@ -440,14 +443,16 @@ profile only. It does not appear in production. - k3s runs containerd with no Docker socket. So the host-Docker path of `ousast plane alerts-engine` does not exist there. The engine must become a task (below). -- Substrate's gVisor workers need `runsc` and a `RuntimeClass` on the nodes they run on. +- Substrate's gVisor workers need no Kubernetes `RuntimeClass`: the worker fetches a pinned gVisor + tarball and runs `runsc` inside its own pod. The node checklist is pod security, tarball + reachability and local-path storage (B.1). - k3s ships Traefik and local-path storage. Traefik is an ingress the CLI can use instead of a kubeconfig. - GitHub Actions builds the images from tagged commits. They are published to GitHub Container Registry and digest-pinned in the manifests. - A private package needs an `imagePullSecret` on the worker ServiceAccount. - Substrate's and ax's own images are built with `ko` from their checkouts. They are pushed to the same registry. -- The kind profile keeps `localhost:5001` and `ops/ax/up.sh`. +- The kind profile keeps its local registry (`ops/k8s/profiles/kind.toml`) and `ops/ax/up.sh`. ### One image, and what concurrency equals @@ -486,7 +491,7 @@ Substrate ships two scaling aids: | Component | Today (the kind profile) | Production (the k3s profile, planned) | | --- | --- | --- | -| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against `kind-ousast`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step | +| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against the profile's `kube_context`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step | | Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction | | Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path | diff --git a/ops/ax/README.md b/ops/ax/README.md index 6e20d0a..44bfad3 100644 --- a/ops/ax/README.md +++ b/ops/ax/README.md @@ -7,12 +7,15 @@ single-node kind cluster. ax is the only executor; `ousast plane run` submits, r ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway, # receiver Service, runner image (digest pinned), ax CLI, smoke Task - ousast plane doctor # kind, Substrate, ax controller, runner image + ousast plane doctor # the profile's cluster (ops/k8s/profiles/kind.toml), Substrate, ax, runner image, memory store ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table ops/ax/down.sh # deletes the cluster and its registry Tools live in `~/go/bin` (kind, ax, ko, kubectl-ate) and `~/.local/bin` (kubectl); sources and rendered files -in `~/.cache/ousast/ax-src/` (the runner digest pin is `runner-image` there). +in `~/.cache/ousast/ax-src/`. The cluster's context, registry and pool sizes come from the kind profile +(`ops/k8s/profiles/kind.toml`); `up.sh` writes the runner's digest pin to that profile's images file +(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them. +`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`). ## How a Run executes @@ -192,16 +195,16 @@ assume this laptop and must change first (checked 2026-10-02): | Assumption | Where | Needed in a real cluster | |---|---|---| -| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind gateway `172.19.0.1:18090` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it | -| Registry `localhost:5001`, rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), `doctor.py:45` (hard-coded check), the runner digest pin, the `image` of every template under `plane/tasks/` | a registry the workers can pull from; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks | -| kubectl context `kind-$KIND_CLUSTER_NAME` | `doctor.py:39`, `egress.py:115` and `:148`, `router.py:145`, `up.sh:24` | one configurable context (for example `OUSAST_KUBE_CONTEXT`); not yet configurable | +| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind network's gateway address on `OUSAST_ARTIFACT_PORT` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it | +| The kind-local registry (`registry` in `ops/k8s/profiles/kind.toml`), rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), the profile's images file, the `image` of every template under `plane/tasks/` | the `k3s` profile's registry; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks | +| kubectl context | `kube_context` in the profile, read by `doctor.py`, `egress.py`, `router.py` (plane-on-kubernetes 1.2); `up.sh` reads the same profile | the `k3s` profile's context; nothing else changes | | ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` in ax's `deploy/ax-server.yaml` points at the ax authors' GCS bucket | ax deploy manifest | your own bucket, set before deploying ax | | Egress gateway applied by hand (agentgateway variant, no Rust build) | this README | the Substrate-installed gateway; per-task EgressPolicies work unchanged | | Worker pool of 2 x 1 CPU / 1.5 GiB for the 7 GB host | `workerpool.yaml.tmpl` | sized to the cluster; `--workers` to match | | Plane memory store and results under `~/ousast-results/` on the host | `reconciler.py` (`OUSAST_RESULTS`), `memory.py` (`OUSAST_MEMORY`) | a persistent volume or bucket shared by the reconciler; the `s3://` store already works against any reachable S3-compatible server | The provider key already travels only in the start request through `atenet-router`, which works the same through -a port-forward to any cluster. +the kube-context tunnel to any cluster. ## What a larger ax deployment needs diff --git a/ops/ax/probe-placement.sh b/ops/ax/probe-placement.sh new file mode 100755 index 0000000..660a7ef --- /dev/null +++ b/ops/ax/probe-placement.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# Placement probe (plane-on-kubernetes task 1.5, design open question 2): does Agent Substrate place an actor on a +# worker that fits its memory request, or on any free worker of the sandbox class? +# +# Applies a second WorkerPool `probe-pool` (1 replica, 2 CPU / 3Gi) from ops/ax/workerpool.yaml.tmpl with the worker +# image the existing pool already built (no ko build), scales the default pool to one worker so the 7 GB host holds +# both, then runs one model-free repo-facts Task asking 2560Mi (limit 3Gi) and one asking 256Mi, recording after +# each resume which worker holds the actor (`kubectl ate get workers -o json`). Everything it created is removed +# on exit and the default pool is scaled back. Usage: ops/ax/probe-placement.sh OUT_DIR (the JSON evidence lands there). +set -euo pipefail +export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH" +OUT="${1:?usage: probe-placement.sh OUT_DIR}" +mkdir -p "$OUT" +REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}" +profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; } +CTX="$(profile kube_context)" +RUNNER_IMAGE="$(profile images.runner)" +DEFAULT_REPLICAS="$(profile pools.default.replicas)" +K() { kubectl --context "$CTX" -n ax-system "$@"; } +log() { printf '%s %s\n' "$(date +%H:%M:%S)" "$*" | tee -a "$OUT/probe.log"; } + +cleanup() { + rc=$? + log "cleanup (exit $rc): delete the probe tasks and probe-pool, scale ousast-pool back to $DEFAULT_REPLICAS" + for t in ousast-probe-big ousast-probe-small; do ax delete task "$t" >/dev/null 2>&1 || true; done + K delete workerpool probe-pool --ignore-not-found >/dev/null 2>&1 || true + K patch workerpool ousast-pool --type merge -p "{\"spec\":{\"replicas\":$DEFAULT_REPLICAS}}" >/dev/null 2>&1 || true + echo "probe exit $rc" +} +trap cleanup EXIT + +workers() { kubectl ate --context "$CTX" get workers -o json > "$OUT/$1.json"; kubectl ate --context "$CTX" get workers | tee -a "$OUT/probe.log"; } + +export POOL_NAME=probe-pool POOL_REPLICAS=1 POOL_CPU=2 POOL_MEMORY=3Gi +WORKER_IMAGE="$(K get workerpool ousast-pool -o jsonpath='{.spec.workerImage}')" # already built and pulled by the node +SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')" +export WORKER_IMAGE SUBSTRATE_VERSION +log "default pool to 1 replica (room for a 3Gi worker on this host)" +K patch workerpool ousast-pool --type merge -p '{"spec":{"replicas":1}}' +for i in $(seq 1 60); do [ "$(K get pods -l ate.dev/worker-pool=ousast-pool --no-headers 2>/dev/null | wc -l)" = 1 ] && break; sleep 2; done +log "apply probe-pool: $POOL_REPLICAS x $POOL_CPU CPU / $POOL_MEMORY, image $WORKER_IMAGE" +envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" | tee "$OUT/probe-pool.yaml" | K apply -f - +K wait --for=condition=Ready pod -l ate.dev/worker-pool=probe-pool --timeout=600s +workers workers-before + +task() { # name, request memory, limit memory + cat > "$OUT/$1.yaml" </dev/null 2>&1 || true + log "apply $1 (requests $2, limits $3)" + ax apply -f "$OUT/$1.yaml" + for i in 1 2 3 4 5 6 7 8 9; do + if ax resume task "$1" 2>&1 | tee -a "$OUT/probe.log"; then break; fi # the first resume may wait for the golden snapshot + sleep 20 + done + sleep 15 + ax get task "$1" | tee "$OUT/$1.status.yaml" | grep -E "phase|message" || true + workers "workers-$1" + ax delete task "$1" >/dev/null 2>&1 || true + for i in $(seq 1 30); do kubectl ate --context "$CTX" get workers | grep -q " 1/1000 " || break; sleep 2; done +} +task ousast-probe-big 2560Mi 3Gi +task ousast-probe-small 256Mi 512Mi +log "done; evidence under $OUT" diff --git a/ops/ax/smoke-run.sh b/ops/ax/smoke-run.sh index bb49312..09c7c80 100755 --- a/ops/ax/smoke-run.sh +++ b/ops/ax/smoke-run.sh @@ -1,14 +1,18 @@ #!/usr/bin/env bash -# Render the end-to-end smoke Run with the current runner digest and execute it through the reconciler on ax. +# Render the end-to-end smoke Run with the profile's runner digest and execute it through the reconciler on ax. +# The profile (OUSAST_PLANE_PROFILE, default kind; ops/k8s/profiles/) names the images file up.sh wrote. set -euo pipefail export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH" SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}" -RUNNER_IMAGE="$(cat "$SRC/runner-image")"; export RUNNER_IMAGE +REPO="$(cd "$(dirname "$0")/../.." && pwd)" +PY="${OUSAST_PYTHON:-$REPO/.venv/bin/python}" +RUNNER_IMAGE="$(PYTHONPATH="$REPO/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print images.runner)" +export RUNNER_IMAGE # Actors reach the receiver by its Service name through the egress gateway (ops/ax/receiver-service.yaml.tmpl, # applied by up.sh, endpoint this host at OUSAST_ARTIFACT_PORT): the runner dials the Service's ClusterIP on port # 80 with that name as Host; the reconciler looks the ClusterIP up unless OUSAST_ARTIFACT_DIAL is set. export OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}" envsubst < "$(dirname "$0")/smoke-run.yaml.tmpl" > "$SRC/smoke-run.yaml" -REPO="$(cd "$(dirname "$0")/../.." && pwd)" -"$REPO/.venv/bin/ousast" plane run "$SRC/smoke-run.yaml" "$@" -"$REPO/.venv/bin/ousast" plane status ax-e2e-smoke +ousast() { PYTHONPATH="$REPO/src" "$PY" -m openultrasast.cli "$@"; } # this checkout's code, whatever python runs it +ousast plane run "$SRC/smoke-run.yaml" "$@" +ousast plane status ax-e2e-smoke diff --git a/ops/ax/up.sh b/ops/ax/up.sh index a5a85c6..2d2af49 100755 --- a/ops/ax/up.sh +++ b/ops/ax/up.sh @@ -1,13 +1,21 @@ #!/usr/bin/env bash # Bring up ax on this host: kind cluster + local registry, Agent Substrate, the ax control plane, one smoke -# Task. Idempotent enough to rerun; `down.sh` removes everything. Requires docker, go, kubectl, kind, ko, ax. +# Task. Idempotent: every step is skipped when already done; `down.sh` removes everything. Requires docker, go, +# kubectl, kind, ko, ax. The cluster's context, the registry and the pool sizes come from the kind profile +# (ops/k8s/profiles/kind.toml; OUSAST_PLANE_PROFILE selects another file with exec = "local"). This script may +# name the kind cluster because it creates it; the code reads the profile (tests/test_plane_literals.py). set -euo pipefail trap 'rc=$?; echo "up.sh FAILED at line $LINENO (exit $rc)"' ERR trap 'echo "up.sh exit $?"' EXIT export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH" export GOTOOLCHAIN=auto -export KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-ousast}" -export KO_DOCKER_REPO="${KO_DOCKER_REPO:-localhost:5001}" +REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}" +profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; } +CTX="$(profile kube_context)" # kind names its context kind- +export KIND_CLUSTER_NAME="${CTX#kind-}" +export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}" +IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}" for t in docker go kubectl kind ko ax; do @@ -19,9 +27,8 @@ clone substrate https://github.com/agent-substrate/substrate.git clone ax https://github.com/google/ax.git step() { printf '\n== %s (%s)\n' "$1" "$(date +%H:%M:%S)"; } -step "kind cluster '$KIND_CLUSTER_NAME' + registry $KO_DOCKER_REPO" +step "kind cluster '$KIND_CLUSTER_NAME' (context $CTX) + registry $KO_DOCKER_REPO" # Each step is skipped when already done: the substrate helper DELETES an existing cluster before creating one. -CTX="kind-$KIND_CLUSTER_NAME" ready() { kubectl --context "$CTX" -n "$1" get pods --no-headers 2>/dev/null | awk '$2 ~ /^[0-9]+\/[0-9]+$/ {n++; split($2,a,"/"); if (a[1]!=a[2] && $3!="Completed") bad++} END {exit !(n>0 && bad==0)}'; } if kind get clusters 2>/dev/null | grep -qx "$KIND_CLUSTER_NAME"; then echo "cluster exists; kept" @@ -56,25 +63,36 @@ KIND_GATEWAY="$(docker network inspect kind --format '{{range .IPAM.Config}}{{if export KIND_GATEWAY OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}" # the reconciler's fixed receiver port envsubst < "$(dirname "$0")/receiver-service.yaml.tmpl" | kubectl --context "$CTX" apply -f - -step "worker pool (gVisor)" +step "worker pool (gVisor): pools.default of the profile" +# `ko apply` builds the worker image from the Substrate checkout (several GB of Go caches): only when the pool +# is absent. A size change in the profile is applied with `kubectl patch` or by deleting the pool first. +export POOL_NAME=ousast-pool POOL_REPLICAS="$(profile pools.default.replicas)" POOL_CPU="$(profile pools.default.cpu)" POOL_MEMORY="$(profile pools.default.memory)" +export WORKER_IMAGE="ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor" SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')" export SUBSTRATE_VERSION envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" > "$SRC/workerpool.yaml" -(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml") -kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l ate.dev/worker-pool=ousast-pool --timeout=600s \ +if kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" >/dev/null 2>&1; then + echo "pool $POOL_NAME exists ($(kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" -o jsonpath='{.status.readyReplicas}/{.spec.replicas}') ready); kept" +else + (cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml") +fi +kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l "ate.dev/worker-pool=$POOL_NAME" --timeout=600s \ || kubectl --context "$CTX" -n ax-system get pods step "runner image $KO_DOCKER_REPO/ousast-runner" -REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)" docker build -q -f "$REPO_ROOT/plane/Dockerfile.runner" -t "$KO_DOCKER_REPO/ousast-runner:dev" "$REPO_ROOT" docker push -q "$KO_DOCKER_REPO/ousast-runner:dev" RUNNER_IMAGE="$(docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$KO_DOCKER_REPO/ousast-runner:dev" | grep "^$KO_DOCKER_REPO/ousast-runner@" | head -1)" [ -n "$RUNNER_IMAGE" ] || { echo "no digest for the runner image" >&2; exit 2; } -export RUNNER_IMAGE; echo "$RUNNER_IMAGE" > "$SRC/runner-image" # the reconciler reads this pin -echo "runner image $RUNNER_IMAGE" +export RUNNER_IMAGE +printf '{\n "runner": "%s"\n}\n' "$RUNNER_IMAGE" > "$IMAGES_FILE" # the profile's images file: commit it with the templates +echo "runner image $RUNNER_IMAGE -> $IMAGES_FILE" +echo "re-pin the templates with: ousast plane workspaces ... --runner-image $IMAGES_FILE (or ousast plane repin, task 4.3)" step "ax CLI from the deployed checkout (a release CLI skews from the server)" -(cd "$SRC/ax" && go install ./cmd/ax) +if [ -x "$HOME/go/bin/ax" ] && [ "$HOME/go/bin/ax" -nt "$SRC/ax/.git/HEAD" ]; then echo "ax CLI newer than the checkout; kept"; else + (cd "$SRC/ax" && go install ./cmd/ax) +fi step "smoke task" ax delete task ousast-smoke >/dev/null 2>&1 || true diff --git a/ops/ax/workerpool.yaml.tmpl b/ops/ax/workerpool.yaml.tmpl index f71cdcf..346029d 100644 --- a/ops/ax/workerpool.yaml.tmpl +++ b/ops/ax/workerpool.yaml.tmpl @@ -1,23 +1,26 @@ -# Warm gVisor workers for ax tasks on this host (Agent Substrate WorkerPool). ax's per-task ActorTemplates -# set no workerSelector, so every pool of the matching sandbox class is eligible. Two workers of 1 CPU / -# 1.5 GiB fit the 7 GB host beside the control planes; an actor occupies a whole worker. +# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by +# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU, +# POOL_MEMORY (ops/k8s/profiles/.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a +# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every +# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of +# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl). apiVersion: ate.dev/v1alpha1 kind: WorkerPool metadata: - name: ousast-pool + name: ${POOL_NAME} namespace: ax-system labels: workload: ousast spec: - replicas: 2 - workerImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor + replicas: ${POOL_REPLICAS} + workerImage: ${WORKER_IMAGE} template: nodeSelector: ate.dev/substrate-version: "${SUBSTRATE_VERSION}" resources: limits: - cpu: "1" - memory: 1536Mi + cpu: "${POOL_CPU}" + memory: ${POOL_MEMORY} requests: cpu: 250m - memory: 1536Mi + memory: ${POOL_MEMORY} diff --git a/ops/k8s/profiles/k3s.toml b/ops/k8s/profiles/k3s.toml new file mode 100644 index 0000000..d1fa987 --- /dev/null +++ b/ops/k8s/profiles/k3s.toml @@ -0,0 +1,26 @@ +# The k3s profile: the plane on the maintainer's production cluster (plane-on-kubernetes design section 1; +# requirements "Target Environment"). Documented values, not secrets: the kubeconfig for the context, the GHCR +# pull secret's content and the store's keys live outside this file. Every field is overridable by OUSAST_, +# a pool by OUSAST_POOL__. Nothing has been applied to a k3s cluster yet; docs/deployment.md says +# which items are planned. + +exec = "remote" # the same CLI from a laptop or a CI job, through the cluster's kube context +kube_context = "ousast-k3s" # the maintainer's context name for the k3s cluster +registry = "ghcr.io/norandom" # images built by GitHub Actions from tagged commits (design section 8) +images = "images.json" # the release asset {"runner": "ghcr.io/norandom/ousast-runner@sha256:...", "engine": ...} +router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel) +ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel) +memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from the CI secrets or .env +atespace = "default" +snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch) +image_pull_secret = "ghcr-pull" # only needed while the GHCR packages are private + +[pools.default] # production sizing: four verify-class workers; the HPA (design section 5) scales it +replicas = 4 +cpu = "1" +memory = "1536Mi" + +[pools.engine] # two engine workers sized from the measured 2.1 GiB peak under a 3 GiB limit +replicas = 2 +cpu = "2" +memory = "4Gi" diff --git a/ops/k8s/profiles/kind-images.json b/ops/k8s/profiles/kind-images.json new file mode 100644 index 0000000..b27d47b --- /dev/null +++ b/ops/k8s/profiles/kind-images.json @@ -0,0 +1,3 @@ +{ + "runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" +} diff --git a/ops/k8s/profiles/kind.toml b/ops/k8s/profiles/kind.toml new file mode 100644 index 0000000..e383696 --- /dev/null +++ b/ops/k8s/profiles/kind.toml @@ -0,0 +1,25 @@ +# The kind profile: the plane on this laptop's single-node kind cluster, which ops/ax/up.sh creates and which is +# the development default (plane-on-kubernetes design section 1). Values only; a credential never goes here +# (keys stay in the environment or .env). Every field is overridable by OUSAST_ in the environment, a pool +# by OUSAST_POOL__. This file may name the kind cluster; the code must not (tests/test_plane_literals.py). + +exec = "local" # the reconciler runs on this host against the cluster below +kube_context = "kind-ousast" # kind's context for KIND_CLUSTER_NAME=ousast +registry = "localhost:5001" # the kind-local registry up.sh creates (Substrate rewrites it for the node) +images = "kind-images.json" # digest-pinned image references, written by up.sh after each build +router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel) +ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel) +memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from .env +atespace = "default" +snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch) +image_pull_secret = "" # the kind registry needs none + +[pools.default] # ousast-pool: two gVisor workers fit the 7 GB host beside the control planes +replicas = 2 +cpu = "1" +memory = "1536Mi" + +[pools.engine] # engine-pool (design section 4): one worker for the Joern engine task +replicas = 1 +cpu = "2" +memory = "4Gi" diff --git a/plane/tasks/agree.yaml b/plane/tasks/agree.yaml index 75fa308..75cce55 100644 --- a/plane/tasks/agree.yaml +++ b/plane/tasks/agree.yaml @@ -7,7 +7,7 @@ metadata: name: agree atespace: default spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["agree"] resources: requests: diff --git a/plane/tasks/alerts.yaml b/plane/tasks/alerts.yaml index d17f5fc..248736d 100644 --- a/plane/tasks/alerts.yaml +++ b/plane/tasks/alerts.yaml @@ -7,7 +7,7 @@ metadata: name: alerts atespace: default spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["alerts"] resources: requests: diff --git a/plane/tasks/features.yaml b/plane/tasks/features.yaml index e844fd4..814106f 100644 --- a/plane/tasks/features.yaml +++ b/plane/tasks/features.yaml @@ -7,7 +7,7 @@ metadata: name: features atespace: default spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["features"] resources: requests: diff --git a/plane/tasks/loop.yaml b/plane/tasks/loop.yaml index 926e6c1..3650ce7 100644 --- a/plane/tasks/loop.yaml +++ b/plane/tasks/loop.yaml @@ -7,7 +7,7 @@ metadata: name: loop atespace: default spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["loop"] resources: requests: diff --git a/plane/tasks/remember.yaml b/plane/tasks/remember.yaml index 85f7af9..4dd0b29 100644 --- a/plane/tasks/remember.yaml +++ b/plane/tasks/remember.yaml @@ -7,7 +7,7 @@ metadata: name: remember atespace: default spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["remember"] resources: requests: diff --git a/plane/tasks/repo-facts.yaml b/plane/tasks/repo-facts.yaml index 8ebc7fb..3ec5baa 100644 --- a/plane/tasks/repo-facts.yaml +++ b/plane/tasks/repo-facts.yaml @@ -8,7 +8,7 @@ metadata: name: repo-facts atespace: default spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["repo-facts"] resources: requests: diff --git a/plane/tasks/roles.yaml b/plane/tasks/roles.yaml index 8ba31c2..011edd6 100644 --- a/plane/tasks/roles.yaml +++ b/plane/tasks/roles.yaml @@ -11,7 +11,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["roles"] resources: requests: diff --git a/plane/tasks/validation-46.yaml b/plane/tasks/validation-46.yaml index 7ab64e1..8f17654 100644 --- a/plane/tasks/validation-46.yaml +++ b/plane/tasks/validation-46.yaml @@ -5,9 +5,9 @@ metadata: name: repo-facts-budibase-mongo-template-nosqli atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}' + openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -36,7 +36,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -67,7 +67,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -96,7 +96,7 @@ metadata: name: agree-budibase-mongo-template-nosqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -123,9 +123,9 @@ metadata: name: repo-facts-cct-studio-server-cmdi atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}' + openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -154,7 +154,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -185,7 +185,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -214,7 +214,7 @@ metadata: name: agree-cct-studio-server-cmdi atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -241,9 +241,9 @@ metadata: name: repo-facts-contextforge-prompt-ssti atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}' + openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -272,7 +272,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -303,7 +303,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -332,7 +332,7 @@ metadata: name: agree-contextforge-prompt-ssti atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -359,9 +359,9 @@ metadata: name: repo-facts-froxlor-admin-ip-second-order-sqli atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}' + openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -390,7 +390,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -421,7 +421,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -450,7 +450,7 @@ metadata: name: agree-froxlor-admin-ip-second-order-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -477,9 +477,9 @@ metadata: name: repo-facts-fuxa-tdengine-escape-sqli atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}' + openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -508,7 +508,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -539,7 +539,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -568,7 +568,7 @@ metadata: name: agree-fuxa-tdengine-escape-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -595,9 +595,9 @@ metadata: name: repo-facts-lmdeploy-media-url-ssrf atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}' + openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -626,7 +626,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -657,7 +657,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -686,7 +686,7 @@ metadata: name: agree-lmdeploy-media-url-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -713,9 +713,9 @@ metadata: name: repo-facts-medplum-external-auth-open-redirect atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}' + openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -744,7 +744,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -775,7 +775,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -804,7 +804,7 @@ metadata: name: agree-medplum-external-auth-open-redirect atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -831,9 +831,9 @@ metadata: name: repo-facts-pgadmin-maintenance-sqli atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}' + openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -862,7 +862,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -893,7 +893,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -922,7 +922,7 @@ metadata: name: agree-pgadmin-maintenance-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -949,9 +949,9 @@ metadata: name: repo-facts-qwed-sympy-eval atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}' + openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -980,7 +980,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1011,7 +1011,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1040,7 +1040,7 @@ metadata: name: agree-qwed-sympy-eval atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1067,9 +1067,9 @@ metadata: name: repo-facts-reactpress-findall-column-sqli atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}' + openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -1098,7 +1098,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1129,7 +1129,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1158,7 +1158,7 @@ metadata: name: agree-reactpress-findall-column-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1185,9 +1185,9 @@ metadata: name: repo-facts-unleash-addon-webhook-ssrf atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}' + openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -1216,7 +1216,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1247,7 +1247,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1276,7 +1276,7 @@ metadata: name: agree-unleash-addon-webhook-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1303,9 +1303,9 @@ metadata: name: repo-facts-winml-cli-serve-cors atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}' + openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -1334,7 +1334,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1365,7 +1365,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1394,7 +1394,7 @@ metadata: name: agree-winml-cli-serve-cors atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1421,9 +1421,9 @@ metadata: name: repo-facts-wp-directorist-avatar-ssrf atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}' + openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -1452,7 +1452,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1483,7 +1483,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1512,7 +1512,7 @@ metadata: name: agree-wp-directorist-avatar-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1539,9 +1539,9 @@ metadata: name: repo-facts-wp-gopay-log-filter-sqli atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}' + openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -1570,7 +1570,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1601,7 +1601,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1630,7 +1630,7 @@ metadata: name: agree-wp-gopay-log-filter-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1657,9 +1657,9 @@ metadata: name: repo-facts-wp-groundhogg-confirm-redirect atespace: default annotations: - openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}' + openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}' spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - repo-facts env: @@ -1688,7 +1688,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1719,7 +1719,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1748,7 +1748,7 @@ metadata: name: agree-wp-groundhogg-confirm-redirect atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - agree env: @@ -1777,7 +1777,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1808,7 +1808,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1839,7 +1839,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1870,7 +1870,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1901,7 +1901,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1932,7 +1932,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1963,7 +1963,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -1994,7 +1994,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2025,7 +2025,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2056,7 +2056,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2087,7 +2087,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2118,7 +2118,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2149,7 +2149,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2180,7 +2180,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2211,7 +2211,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - verify env: @@ -2240,7 +2240,7 @@ metadata: name: features-budibase-mongo-template-nosqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2263,7 +2263,7 @@ metadata: name: remember-budibase-mongo-template-nosqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2288,7 +2288,7 @@ metadata: name: features-cct-studio-server-cmdi atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2311,7 +2311,7 @@ metadata: name: remember-cct-studio-server-cmdi atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2336,7 +2336,7 @@ metadata: name: features-contextforge-prompt-ssti atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2359,7 +2359,7 @@ metadata: name: remember-contextforge-prompt-ssti atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2384,7 +2384,7 @@ metadata: name: features-froxlor-admin-ip-second-order-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2407,7 +2407,7 @@ metadata: name: remember-froxlor-admin-ip-second-order-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2432,7 +2432,7 @@ metadata: name: features-fuxa-tdengine-escape-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2455,7 +2455,7 @@ metadata: name: remember-fuxa-tdengine-escape-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2480,7 +2480,7 @@ metadata: name: features-lmdeploy-media-url-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2503,7 +2503,7 @@ metadata: name: remember-lmdeploy-media-url-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2528,7 +2528,7 @@ metadata: name: features-medplum-external-auth-open-redirect atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2551,7 +2551,7 @@ metadata: name: remember-medplum-external-auth-open-redirect atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2576,7 +2576,7 @@ metadata: name: features-pgadmin-maintenance-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2599,7 +2599,7 @@ metadata: name: remember-pgadmin-maintenance-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2624,7 +2624,7 @@ metadata: name: features-qwed-sympy-eval atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2647,7 +2647,7 @@ metadata: name: remember-qwed-sympy-eval atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2672,7 +2672,7 @@ metadata: name: features-reactpress-findall-column-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2695,7 +2695,7 @@ metadata: name: remember-reactpress-findall-column-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2720,7 +2720,7 @@ metadata: name: features-unleash-addon-webhook-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2743,7 +2743,7 @@ metadata: name: remember-unleash-addon-webhook-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2768,7 +2768,7 @@ metadata: name: features-winml-cli-serve-cors atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2791,7 +2791,7 @@ metadata: name: remember-winml-cli-serve-cors atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2816,7 +2816,7 @@ metadata: name: features-wp-directorist-avatar-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2839,7 +2839,7 @@ metadata: name: remember-wp-directorist-avatar-ssrf atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2864,7 +2864,7 @@ metadata: name: features-wp-gopay-log-filter-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2887,7 +2887,7 @@ metadata: name: remember-wp-gopay-log-filter-sqli atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: @@ -2912,7 +2912,7 @@ metadata: name: features-wp-groundhogg-confirm-redirect atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - features env: @@ -2935,7 +2935,7 @@ metadata: name: remember-wp-groundhogg-confirm-redirect atespace: default spec: - image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884 + image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 command: - remember env: diff --git a/plane/tasks/verify.yaml b/plane/tasks/verify.yaml index 2ca6050..89b78eb 100644 --- a/plane/tasks/verify.yaml +++ b/plane/tasks/verify.yaml @@ -10,7 +10,7 @@ metadata: annotations: openultrasast.io/model: deepseek-flash spec: - image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884" + image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019" command: ["verify"] resources: requests: diff --git a/src/openultrasast/cli.py b/src/openultrasast/cli.py index 6722fbe..61a185c 100644 --- a/src/openultrasast/cli.py +++ b/src/openultrasast/cli.py @@ -323,16 +323,21 @@ def _main(argv: list[str] | None) -> int: learn_labels.add_argument("--include-title", action="store_true", help="the title-tier pairs as well (a sensitivity arm)") learn_labels.add_argument("--no-assumed-benign", action="store_true", help="skip mining the assumed-benign history source") add_learn_commands(learn_sub) - plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on this host's kind cluster") + plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on the profile's cluster (ops/k8s/profiles/)") + profile_help = "the PlaneProfile: a name under ops/k8s/profiles/ or a .toml path (default OUSAST_PLANE_PROFILE, else kind)" plane_sub = plane.add_subparsers(dest="plane_command", required=True) plane_run = plane_sub.add_parser("run", help="execute a Run manifest on ax; a rerun skips done tasks") plane_run.add_argument("run_manifest", type=Path) plane_run.add_argument("--workers", type=int, default=1, help="tasks in flight at once (serialize labels never overlap)") plane_run.add_argument("--ax", default="ax", help="path of the ax CLI") + plane_run.add_argument("--profile", help=profile_help) plane_status = plane_sub.add_parser("status", help="per-task status and the token attribution table of a run") plane_status.add_argument("run") plane_status.add_argument("--units", action="store_true", help="per-unit rows from each task's units.jsonl") - plane_sub.add_parser("doctor", help="check kind, Agent Substrate, the ax controller and the runner image") + plane_doctor = plane_sub.add_parser( + "doctor", help="check the profile's cluster, Agent Substrate, the ax controller, the runner image and the memory store" + ) + plane_doctor.add_argument("--profile", help=profile_help) plane_remember = plane_sub.add_parser("remember", help="ingest a run's memory rows into the store (OUSAST_MEMORY)") plane_remember.add_argument("run") plane_remember.add_argument("--plane", type=Path, default=Path("plane"), help="where runs/.yaml names population and split") @@ -370,9 +375,11 @@ def _main(argv: list[str] | None) -> int: ) plane_harvest.add_argument("--ceiling", type=float, default=10.0, help="USD: each Run's task budgets sum to at most this") plane_harvest.add_argument("--runner-image", type=Path, help="re-pin the task templates to the image in this file first (ops/ax/up.sh)") + plane_harvest.add_argument("--profile", help=profile_help) plane_ws = plane_sub.add_parser("workspaces", help="one Workspace manifest per case pin of a population file") plane_ws.add_argument("population", type=Path) plane_ws.add_argument("--out", type=Path, default=Path("plane/workspaces")) + plane_ws.add_argument("--profile", help=profile_help) plane_ws.add_argument("--validation-set", type=Path, help="JSON {case: [[path, function], ...]}: emit that set's increment instead") plane_ws.add_argument("--candidates", type=Path, help="with --validation-set: per-case candidate records (independent-v2-modelsinks)") plane_ws.add_argument("--triage", type=Path, help="with --validation-set: recorded batched-check scans (--vulnerable_a.jsonl)") @@ -1529,10 +1536,22 @@ def _learn_labels(args: argparse.Namespace) -> int: def _plane(args: argparse.Namespace) -> int: from .plane import reconciler # lazy: the plane is a maintainer surface, not the scan path + from .plane.profile import ProfileError, load_profile + try: + profile = load_profile(getattr(args, "profile", None)) + except ProfileError as exc: + print(f"plane: {exc}", file=sys.stderr) + return 2 if args.plane_command == "run": + from .plane.memory import MemoryStoreError + _plane_memory("seed", args.run_manifest) - result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax) + try: + result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax, profile=profile) + except MemoryStoreError as exc: # the store is the state of record: no fallback, the Run is not submitted + print(f"memory store {profile.memory}: {exc}", file=sys.stderr) + return 1 print(f"run finished: {result}") _plane_memory("ingest", args.run_manifest) return 0 if result == "done" else 1 @@ -1547,14 +1566,14 @@ def _plane(args: argparse.Namespace) -> int: if args.plane_command == "alerts-engine": return _plane_alerts_engine(args) if args.plane_command == "harvest": - return _plane_harvest(args) + return _plane_harvest(args, profile.atespace) if args.plane_command == "doctor": - checks = reconciler.doctor() + checks = reconciler.doctor(profile) for name, ok, text in checks: print(f"[{'ok' if ok else 'FAIL'}] {name}: {text}") return 0 if all(ok for _, ok, _ in checks) else 1 if args.validation_set is not None: - return _plane_increment(args) + return _plane_increment(args, profile.atespace) from .plane.workspaces import workspaces written = workspaces(args.population, args.out) @@ -1562,7 +1581,7 @@ def _plane(args: argparse.Namespace) -> int: return 0 -def _plane_harvest(args: argparse.Namespace) -> int: +def _plane_harvest(args: argparse.Namespace, atespace: str) -> int: from .plane.generate import repin_templates from .plane.harvest import write_harvest @@ -1571,7 +1590,7 @@ def _plane_harvest(args: argparse.Namespace) -> int: command = f"ousast plane harvest --labels {args.labels.name} --name {args.name} --ceiling {args.ceiling:g}" files = write_harvest( args.labels, args.plane, name=args.name, command=command, catalog=args.catalog, cache=args.cache, templates=args.templates, - ceiling=args.ceiling, + ceiling=args.ceiling, atespace=atespace, ) # fmt: skip for relative in sorted(k for k in files if k.startswith("runs/")): print(f"{args.plane / relative}: {files[relative].splitlines()[1].lstrip('# ')}") @@ -1652,7 +1671,7 @@ def _plane_memory( return 0 -def _plane_increment(args: argparse.Namespace) -> int: +def _plane_increment(args: argparse.Namespace, atespace: str) -> int: from .plane.generate import LOOP_MANIFEST, Loop, increment if args.candidates is None or args.triage is None: @@ -1678,7 +1697,7 @@ def _plane_increment(args: argparse.Namespace) -> int: command += "".join(f" --qualify-population {p}" for p in loop.populations) written = increment( args.population, args.validation_set, args.candidates, args.triage, plane=args.plane, run_name=args.run, command=command, - repos=args.repos, runner_image=args.runner_image, loop=loop, + repos=args.repos, runner_image=args.runner_image, loop=loop, atespace=atespace, ) # fmt: skip print(f"{len(written)} manifests written under {args.plane} (Run {args.run})") return 0 diff --git a/src/openultrasast/cpg/session.py b/src/openultrasast/cpg/session.py index 2727e9e..f421754 100644 --- a/src/openultrasast/cpg/session.py +++ b/src/openultrasast/cpg/session.py @@ -162,7 +162,9 @@ class EngineSession: return None payload = json.dumps({"query": code}).encode() call = urllib.request.Request( - f"http://127.0.0.1:{self._port}/query-sync", data=payload, headers={"Content-Type": "application/json"} + f"http://127.0.0.1:{self._port}/query-sync", # loopback: the Joern server this process started + data=payload, + headers={"Content-Type": "application/json"}, ) try: with urllib.request.urlopen(call, timeout=timeout) as response: # noqa: S310 -- fixed loopback URL diff --git a/src/openultrasast/plane/delivery.py b/src/openultrasast/plane/delivery.py new file mode 100644 index 0000000..420dcf3 --- /dev/null +++ b/src/openultrasast/plane/delivery.py @@ -0,0 +1,192 @@ +"""Task artifacts through the memory store (plane-on-kubernetes design section 3, Req 3.1). + +A task's runner PUTs its output tar to a presigned URL the reconciler minted, and GETs each declared input from +another; the URLs travel in the start request's ``delivery`` field (a presigned URL is a bearer capability, so it +goes where the credentials go, never into an ActorTemplate's env). The reconciler polls the store for the tar, +extracts it under the run directory with the member checks the old receiver had, re-puts the task's declared +``outputs[]`` as single objects so a consumer's URL names exactly one artifact, and writes ``done.json``. The run's +``state.json`` is mirrored to the store after each change and seeded from it on a rerun elsewhere. + +Store layout under the 30-day ``runs/`` expiry rule (``memory.RUNS_RULE_ID``):: + + runs//state.json the reconciler's state, mirrored + runs///output.tar the runner's PUT + runs/// each declared output, re-put by the reconciler + runs///done.json status, bytes, file count, finished + runs//inputs/ an input no task produces (``put_input``) + +Nothing here logs or prints a URL. +""" + +from __future__ import annotations + +import io +import json +import os +import re +import tarfile +from collections.abc import Iterable, Mapping +from datetime import UTC, datetime, timedelta +from pathlib import Path +from typing import Any +from urllib.parse import urlsplit + +from .manifests import Run +from .memory import RUNS_PREFIX, MemoryStore + +__all__ = ["DONE", "GRACE_SECONDS", "OUTPUT_TAR", "STATE", "Delivery", "DeliveryError", "declared", "expiry", "input_var"] + +OUTPUT_TAR = "output.tar" +DONE = "done.json" +STATE = "state.json" +INPUTS_DIR = "inputs" +GRACE_SECONDS = 600 # a URL outlives the task timeout by this much; a re-start after a re-resume re-mints anyway +_NAME = re.compile(r"[^A-Z0-9]+") + + +class DeliveryError(RuntimeError): + """A delivered tar that cannot be accepted, or an object that is not there; the message never carries a URL.""" + + +def expiry(environ: Mapping[str, str] | None = None) -> timedelta: + """``OUSAST_TASK_TIMEOUT`` (7200 s) plus :data:`GRACE_SECONDS`.""" + env = os.environ if environ is None else environ + return timedelta(seconds=float(env.get("OUSAST_TASK_TIMEOUT") or 7200) + GRACE_SECONDS) + + +def input_var(name: str) -> str: + """The ``OUSAST_INPUT_`` suffix of a Run input name, as the reconciler renders it.""" + return _NAME.sub("_", name.upper()) + + +def _now() -> str: + return datetime.now(UTC).isoformat(timespec="seconds") + + +def declared(run: Run) -> dict[str, list[str]]: + """Per Run task, the artifacts to publish as single objects: its ``outputs[]`` plus every ``/`` + a consumer declares as an input on it (``summary.json`` of a producer is a usual one), in declaration order.""" + wanted: dict[str, list[str]] = {entry.name: list(entry.outputs) for entry in run.tasks} + for entry in run.tasks: + for ref in entry.inputs.values(): + producer, _, artifact = ref.partition("/") + if producer in wanted and artifact and artifact not in wanted[producer]: + wanted[producer].append(artifact) + return wanted + + +class Delivery: + """Presigned URLs, polling, collection and state mirroring for one Run on one store.""" + + def __init__(self, store: MemoryStore, run: str, base: Path, *, expires: timedelta | None = None) -> None: + self.store, self.run, self.base = store, run, base + self.expires = expires or expiry() + + def key(self, *parts: str) -> str: + return f"{RUNS_PREFIX}{self.run}/" + "/".join(parts) + + @property + def host(self) -> str | None: + """The hostname the runner will dial (for the task's egress policy); None for a store without one.""" + return urlsplit(self.store.presign_get(self.key(STATE), timedelta(seconds=60))).hostname + + # --- the start body ------------------------------------------------------------------------------------------- + + def put_url(self, task: str) -> str: + return self.store.presign_put(self.key(task, OUTPUT_TAR), self.expires) + + def input_urls(self, inputs: Mapping[str, str]) -> dict[str, str]: + """``{OUSAST_INPUT_ suffix: presigned GET of runs///}``.""" + out: dict[str, str] = {} + for name, ref in inputs.items(): + producer, _, artifact = ref.partition("/") + if not producer or not artifact: + raise DeliveryError(f"input {name}: {ref!r} is not /") + out[input_var(name)] = self.store.presign_get(self.key(producer, artifact), self.expires) + return out + + def body(self, task: str, inputs: Mapping[str, str]) -> dict[str, Any]: + """The start request's ``delivery`` field for ``task``.""" + return {"put": self.put_url(task), "inputs": self.input_urls(inputs)} + + # --- completion ---------------------------------------------------------------------------------------------- + + def reset(self, task: str) -> None: + """Before an attempt: remove the previous attempt's tar and ``done.json`` so a rerun (an unfinished task + resumed with a larger budget, an interrupted attempt) never completes on a stale object.""" + for name in (OUTPUT_TAR, DONE): + self.store._delete(self.key(task, name)) + + def delivered(self, task: str) -> bool: + """Whether the runner's tar is in the store (an exact-key listing, the store's HEAD).""" + key = self.key(task, OUTPUT_TAR) + return key in self.store._keys(key) + + def collect(self, task: str, outputs: Iterable[str]) -> tuple[dict[str, Any] | None, int]: + """Download and extract the tar under ``//``, re-put the declared ``outputs``, write + ``done.json``; returns (the extracted ``summary.json`` or None, the tar's byte size).""" + got = self.store._get(self.key(task, OUTPUT_TAR)) + if got is None: + raise DeliveryError(f"task {task}: output.tar is not in the store") + data = got[0] + dest = (self.base / task).resolve() + try: + with tarfile.open(fileobj=io.BytesIO(data), mode="r:") as tar: + members = tar.getmembers() + for m in members: + if m.issym() or m.islnk() or not (dest / m.name).resolve().is_relative_to(dest): + raise DeliveryError(f"task {task}: rejected member {m.name}") + dest.mkdir(parents=True, exist_ok=True) + tar.extractall(dest, members) # every member checked above + except tarfile.TarError as exc: + raise DeliveryError(f"task {task}: bad tar: {exc}") from None + published = self.publish(task, outputs) + summary = _read_json(dest / "summary.json") + done = { + "task": task, + "status": summary.get("status") if isinstance(summary, dict) else None, + "bytes": len(data), + "files": sum(1 for m in members if m.isfile()), + "published": published, + "finished": _now(), + } + self.store._put(self.key(task, DONE), json.dumps(done, sort_keys=True).encode("utf-8")) + return (summary if isinstance(summary, dict) else None), len(data) + + def publish(self, task: str, outputs: Iterable[str]) -> list[str]: + """Re-put the task's declared outputs that exist under ``//`` as single objects; the names put.""" + put: list[str] = [] + root = (self.base / task).resolve() + for artifact in outputs: + path = (root / artifact).resolve() + if path.is_relative_to(root) and path.is_file(): + self.store._put(self.key(task, artifact), path.read_bytes()) + put.append(artifact) + return put + + def put_input(self, name: str, path: Path) -> str: + """Store a file no task produces as ``runs//inputs/``; returns its presigned GET URL.""" + self.store._put(self.key(INPUTS_DIR, name), Path(path).read_bytes()) + return self.store.presign_get(self.key(INPUTS_DIR, name), self.expires) + + # --- state --------------------------------------------------------------------------------------------------- + + def save_state(self, data: Mapping[str, Any]) -> None: + self.store._put(self.key(STATE), (json.dumps(data, indent=2) + "\n").encode("utf-8")) + + def load_state(self) -> dict[str, Any] | None: + got = self.store._get(self.key(STATE)) + if got is None: + return None + try: + loaded = json.loads(got[0].decode("utf-8")) + except ValueError: + return None + return loaded if isinstance(loaded, dict) else None + + +def _read_json(path: Path) -> Any: + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None diff --git a/src/openultrasast/plane/doctor.py b/src/openultrasast/plane/doctor.py index 7e067ea..f9cc345 100644 --- a/src/openultrasast/plane/doctor.py +++ b/src/openultrasast/plane/doctor.py @@ -1,15 +1,35 @@ -"""``ousast plane doctor``: four checks of this host's ax before a Run is submitted (ai-service-plane Req 3.5). +"""``ousast plane doctor``: checks of the configured cluster before a Run is submitted (ai-service-plane Req 3.5, +plane-on-kubernetes Req 1.2). + +Everything is read from the :class:`~openultrasast.plane.profile.PlaneProfile`: the kube context is checked for +reachability and, when reachable, the two namespaces for ready pods; the ``images`` file's runner reference is +resolved in its registry (a local registry over its v2 API, a remote one through ``docker manifest inspect``); the +memory store is opened, which verifies an S3 bucket; and the first line reports every configured address. Nothing +here names a cluster, a registry or an address of its own: those are the profile's. A sibling of ``reconciler.py`` (tasks.md task 0 allows either), which re-exports :func:`doctor`. """ from __future__ import annotations -import os +import ipaddress import re import subprocess +import urllib.error +import urllib.request + +from .profile import PlaneProfile, ProfileError, load_profile UP = "ops/ax/up.sh brings it up" +RUNBOOK = "docs/deployment.md lists the prerequisites" +_MANIFEST_TYPES = ", ".join( + ( + "application/vnd.oci.image.index.v1+json", + "application/vnd.oci.image.manifest.v1+json", + "application/vnd.docker.distribution.manifest.list.v2+json", + "application/vnd.docker.distribution.manifest.v2+json", + ) +) # --- checks ---------------------------------------------------------------------------------------------- @@ -34,18 +54,68 @@ def _pods_ready(namespace: str, context: str) -> tuple[bool, str]: return not bad, f"not ready: {', '.join(bad)}" if bad else f"{len(rows)} pods ready" -def doctor() -> list[tuple[str, bool, str]]: - """Four checks of the host's ax: kind, Agent Substrate, the ax controller, the runner image (Req 3.5).""" - context = "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast") +def _is_local(registry: str) -> bool: + host = registry.rsplit(":", 1)[0] if re.search(r":\d+$", registry) else registry + if host == "localhost": + return True + try: + return ipaddress.ip_address(host.strip("[]")).is_loopback or ipaddress.ip_address(host.strip("[]")).is_private + except ValueError: + return False + + +def _registry_has(image: str) -> tuple[bool, str]: + """Resolve a digest-pinned reference ``/@sha256:``: a local registry answers its v2 + API for the manifest by digest (plain HTTP, as kind's registry speaks); a remote one is asked through + ``docker manifest inspect``, which carries the operator's registry login.""" + name, _, digest = image.partition("@") + registry, _, repository = name.partition("/") + short = f"{repository}@{digest[:19]}..." + if _is_local(registry): + url = f"http://{registry}/v2/{repository}/manifests/{digest}" + request = urllib.request.Request(url, method="HEAD", headers={"Accept": _MANIFEST_TYPES}) + try: + with urllib.request.urlopen(request, timeout=20) as response: # noqa: S310 - the profile's registry + found = 200 <= int(response.status) < 300 + except urllib.error.HTTPError as exc: + return False, f"{short} missing from {registry} (HTTP {exc.code})" + except (urllib.error.URLError, OSError) as exc: + return False, f"{registry} unreachable ({str(exc)[:80]})" + return found, f"{short} present in {registry}" if found else f"{short} missing from {registry}" + ok, out = _sh("docker", "manifest", "inspect", image) + return ok, f"{short} present in {registry}" if ok else f"{short} not resolved in {registry} ({out[:120]})" + + +def doctor(profile: PlaneProfile | None = None) -> list[tuple[str, bool, str]]: + """The checks of the profile's cluster: ``(name, ok, text)`` per line, the configured addresses first.""" + profile = profile or load_profile() + hint = UP if profile.name == "kind" else RUNBOOK + context = profile.kube_context + checks = [(f"profile {profile.name}", True, " ".join(f"{k}={v}" for k, v in profile.addresses().items()))] ok, out = _sh("kubectl", "--context", context, "cluster-info") - checks = [("kind cluster", ok, "reachable" if ok else f"context {context} unreachable ({out[:80]}); {UP}")] - for label, ns in (("agent substrate (ate-system)", "ate-system"), ("ax controller (ax-system)", "ax-system")): - ok, out = _pods_ready(ns, context) - checks.append((label, ok, out if ok else f"{out}; {UP}")) - ok, out = _sh("curl", "-s", "localhost:5001/v2/_catalog") - present = ok and "ousast-runner" in out - text = "ousast-runner present" if present else f"missing from localhost:5001 ({out[:80]}); {UP} and loads it" - checks.append(("runner image in kind registry", present, text)) + checks.append((f"cluster {context}", ok, "reachable" if ok else f"context {context} unreachable ({out[:80]}); {hint}")) + if ok: # behind an unreachable context the namespaces cannot be told apart from it + for label, ns in (("agent substrate (ate-system)", "ate-system"), ("ax controller (ax-system)", "ax-system")): + ok, out = _pods_ready(ns, context) + checks.append((label, ok, out if ok else f"{out}; {hint}")) + try: + images = profile.load_images() + runner = images["runner"] + except ProfileError as exc: + checks.append((f"runner image in {profile.registry}", False, f"{exc}; {hint}")) + except KeyError: + checks.append((f"runner image in {profile.registry}", False, f"{profile.images} names no runner image; {hint}")) + else: + ok, out = _registry_has(runner) + checks.append((f"runner image in {profile.registry}", ok, out if ok else f"{out}; {hint} and loads it")) + from .memory import MemoryStoreError, open_store # lazy: boto3 only when the profile names an S3 store + + try: + store = open_store(profile.memory) + except (MemoryStoreError, OSError, ValueError) as exc: + checks.append((f"memory store {profile.memory}", False, str(exc)[:300])) + else: + checks.append((f"memory store {profile.memory}", True, f"{store.describe()} opens (an S3 bucket is verified)")) return checks diff --git a/src/openultrasast/plane/egress.py b/src/openultrasast/plane/egress.py index 6bc35de..57a487f 100644 --- a/src/openultrasast/plane/egress.py +++ b/src/openultrasast/plane/egress.py @@ -84,17 +84,32 @@ def git_hosts(workspaces: Iterable[Workspace]) -> tuple[list[str], list[str]]: return sorted(tls), sorted(plain) -def policy_for(task: Task, workspaces: Iterable[Workspace], model: Model | None, receiver_host: str | None) -> dict[str, Any]: - """The task's EgressPolicy body: the receiver over HTTP/80, Git and Model hosts as TLS passthrough on 443.""" +def policy_for( + task: Task, + workspaces: Iterable[Workspace], + model: Model | None, + store_host: str | None, + receiver_host: str | None = None, +) -> dict[str, Any]: + """The task's EgressPolicy body (plane-on-kubernetes design section 3): one ``tlsPassthrough`` rule on 443 over + the sorted union of the store's hostname (``store_host``: where the runner PUTs its tar and GETs its inputs), the + bound Workspaces' https Git hosts and the bound Model's ``openultrasast.io/egress-hosts``. No ``http`` rule, unless + ``receiver_host`` names the old receiver (the path task 2.6 removes). An http Git repo cannot be reached under + this policy and is refused; a store given as an address is refused naming ``S3_ENDPOINT`` (the gateway allows + hostnames only).""" tls, plain = git_hosts(workspaces) - tls = sorted(set(tls) | set(model.egress_hosts if model else ())) - if receiver_host and not _is_ip(receiver_host): # an address cannot be allowed; only the direct test mode has one - plain = sorted({*plain, receiver_host.lower()}) - rules: list[dict[str, Any]] = [] if plain: - rules.append({"http": {"hostnames": plain, "ports": {"numbers": [80]}}}) - if tls: - rules.append({"tlsPassthrough": {"hostnames": tls, "ports": {"numbers": [443]}}}) + raise ValueError(f"Task/{task.metadata.name}: http Git repos ({', '.join(plain)}) cannot be reached: the policy grants TLS only") + hosts = set(tls) | set(model.egress_hosts if model else ()) + if store_host: + if _is_ip(store_host): + raise ValueError(f"Task/{task.metadata.name}: the store endpoint is an address ({store_host}); S3_ENDPOINT must be a hostname") + hosts.add(store_host.lower()) + rules: list[dict[str, Any]] = [] + if receiver_host and not _is_ip(receiver_host): # the receiver path, kept for group 2 only (task 2.6 deletes it) + rules.append({"http": {"hostnames": [receiver_host.lower()], "ports": {"numbers": [80]}}}) + if hosts: + rules.append({"tlsPassthrough": {"hostnames": sorted(hosts), "ports": {"numbers": [443]}}}) return {"rules": rules} @@ -110,9 +125,11 @@ def rules_of(policy: Mapping[str, Any]) -> list[dict[str, Any]]: return sorted(out, key=lambda r: json.dumps(r, sort_keys=True)) -def receiver_cluster_ip(context: str | None = None) -> str | None: - """The ClusterIP of the ``ousast-receiver`` Service (``kubectl get svc``); None when kubectl cannot say.""" - context = context or "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast") +def receiver_cluster_ip(context: str) -> str | None: + """The ClusterIP of the ``ousast-receiver`` Service (``kubectl get svc`` in the profile's ``context``); None when + kubectl cannot say.""" + if not context: + raise RuntimeError("receiver_cluster_ip needs the profile's kube context") argv = ["kubectl", "--context", context, "-n", "ax-system", "get", "svc", "ousast-receiver", "-o", "jsonpath={.spec.clusterIP}"] try: proc = subprocess.run(argv, capture_output=True, text=True, check=False, timeout=20) @@ -122,8 +139,9 @@ def receiver_cluster_ip(context: str | None = None) -> str | None: return address if proc.returncode == 0 and _is_ip(address) else None -def receiver_address(port: int) -> tuple[str, str | None]: - """(``OUSAST_ARTIFACT_URL``, ``OUSAST_ARTIFACT_DIAL``) for a receiver bound on ``port``. +def receiver_address(port: int, context: str | None) -> tuple[str, str | None]: + """(``OUSAST_ARTIFACT_URL``, ``OUSAST_ARTIFACT_DIAL``) for a receiver bound on ``port``; ``context`` is the + profile's kube context the Service's ClusterIP is looked up in. ``OUSAST_ARTIFACT_HOST`` set: the old direct URL ``http://:/`` and no dial (tests, a host the actor reaches without the gateway). Otherwise the receiver Service's name, dialled at ``OUSAST_ARTIFACT_DIAL`` or the @@ -135,7 +153,9 @@ def receiver_address(port: int) -> tuple[str, str | None]: return f"http://{host}:{port}/", None dial = os.environ.get("OUSAST_ARTIFACT_DIAL") if not dial: - address = receiver_cluster_ip() + if not context: + raise RuntimeError("receiver_address needs the profile's kube context to look the receiver Service up") + address = receiver_cluster_ip(context) dial = f"{address}:80" if address else None return f"http://{RECEIVER_HOST}/", dial @@ -143,9 +163,11 @@ def receiver_address(port: int) -> tuple[str, str | None]: class Egress: """``kubectl ate`` for one actor's egress policy; the executable is injectable so tests use a fake.""" - def __init__(self, executable: str | None = None, context: str | None = None) -> None: + def __init__(self, executable: str | None, context: str) -> None: + if not context: + raise ValueError("Egress needs the profile's kube context; it never builds one") self.executable = executable or os.environ.get("OUSAST_KUBECTL_ATE") or "kubectl-ate" - self.context = context or "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast") + self.context = context def _run(self, *args: str, stdin: str | None = None) -> subprocess.CompletedProcess[str]: argv = [self.executable, "--context", self.context, *args] @@ -214,12 +236,12 @@ class Receiver(ThreadingHTTPServer): daemon_threads = True - def __init__(self, run_name: str, base: Path, port: int) -> None: + def __init__(self, run_name: str, base: Path, port: int, context: str | None = None) -> None: super().__init__(("0.0.0.0", port), _Handler) self.run_name, self.base, self.running, self.delivered = run_name, base, set[str](), set[str]() self.inputs: dict[str, frozenset[str]] = {} self.lock = threading.Lock() - self.url, self.dial = receiver_address(self.server_address[1]) + self.url, self.dial = receiver_address(self.server_address[1], context) def _task(self, headers: Any) -> str | None: """The ``X-Ousast-Task`` of a request naming this run and a task that is running now, else None.""" diff --git a/src/openultrasast/plane/generate.py b/src/openultrasast/plane/generate.py index e354d5e..47c6e0e 100644 --- a/src/openultrasast/plane/generate.py +++ b/src/openultrasast/plane/generate.py @@ -45,8 +45,9 @@ The recorded triage cost is derived per file: a batched-check record's ``usd`` p hunts, and its ``usage`` counts only the hunts (``verify_sinks._Recording`` slices the client's usage from the first hunt on), so the triage share is ``usd`` minus that usage priced at the reference model's list price. -``--runner-image FILE`` (``~/.cache/ousast/ax-src/runner-image``, written by ``ops/ax/up.sh``) re-pins the -templates' ``image:`` to the digest in that file before generating, so a rebuilt image is one command. +``--runner-image FILE`` (the profile's images file, ``ops/k8s/profiles/kind-images.json``, written by ``ops/ax/up.sh``; +the older one-line pin file still reads) re-pins the templates' ``image:`` to the runner digest in that file before +generating, so a rebuilt image is one command. """ from __future__ import annotations @@ -82,7 +83,7 @@ TRIAGE_BASIS = ( RANGES_BASIS = "git diff -U0 , old side: benchmarks/independent/evaluate.hunks(case, 'old')" HEADROOM = 1.5 CALLS_PER_HUNT = MAX_STEPS + 2 # every tool step, the answer, one empty-content retry -ATESPACE = "default" +ATESPACE = "default" # the default when no profile is given; ``increment`` takes ``profile.atespace`` CASE_PATH = "/workspace/case" INPUTS_PATH = "/workspace/inputs" FIXED_PATH = "/workspace/fixed" @@ -236,10 +237,10 @@ def _case_record(item: CaseInputs) -> dict[str, Any]: return record -def _workspaces(item: CaseInputs) -> tuple[dict[str, Any], dict[str, Any]]: +def _workspaces(item: CaseInputs, atespace: str = ATESPACE) -> tuple[dict[str, Any], dict[str, Any]]: case = item.case git = [{"name": "repo", "repo": case["repo"], "dir": "repo", "depth": 1}] - pinned = _doc("Workspace", _ax_name(case["id"], "vulnerable"), {"git": git}, ATESPACE) + pinned = _doc("Workspace", _ax_name(case["id"], "vulnerable"), {"git": git}, atespace) pinned["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"repo={case['vulnerable']}"} files = { "candidates.json": _json({"id": case["id"], "family": case["family"], "candidates": [list(c) for c in item.kept]}), @@ -249,7 +250,7 @@ def _workspaces(item: CaseInputs) -> tuple[dict[str, Any], dict[str, Any]]: {"candidates": [list(c) for c in item.candidates], "kept": [list(c) for c in item.kept], "triage": dict(item.triage)} ), } - inputs = _doc("Workspace", _ax_name(case["id"], "inputs"), {"files": [{"path": k, "content": v} for k, v in files.items()]}, ATESPACE) + inputs = _doc("Workspace", _ax_name(case["id"], "inputs"), {"files": [{"path": k, "content": v} for k, v in files.items()]}, atespace) return pinned, inputs @@ -339,11 +340,11 @@ def _remember(item: CaseInputs, templates: Mapping[str, Task], population: str, return task, entry -def _alerts(item: CaseInputs, templates: Mapping[str, Task]) -> tuple[dict, dict, dict]: +def _alerts(item: CaseInputs, templates: Mapping[str, Task], atespace: str = ATESPACE) -> tuple[dict, dict, dict]: """(fixed-pin Workspace, Task, Run entry) of the case's `alerts`: quick-mode rules on both pins.""" case = item.case git = [{"name": "repo", "repo": case["repo"], "dir": "repo", "depth": 1}] - fixed = _doc("Workspace", _ax_name(case["id"], "fixed"), {"git": git}, ATESPACE) + fixed = _doc("Workspace", _ax_name(case["id"], "fixed"), {"git": git}, atespace) fixed["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"repo={case['fixed']}"} bindings = [(_ax_name(case["id"], "vulnerable"), CASE_PATH), (fixed["metadata"]["name"], FIXED_PATH)] bindings.append((_ax_name(case["id"], "inputs"), INPUTS_PATH)) @@ -358,10 +359,12 @@ def _alerts(item: CaseInputs, templates: Mapping[str, Task]) -> tuple[dict, dict return fixed, task, entry -def _loop(cases: Sequence[CaseInputs], templates: Mapping[str, Task], loop: Loop) -> tuple[dict, list[dict], list[dict]]: +def _loop( + cases: Sequence[CaseInputs], templates: Mapping[str, Task], loop: Loop, atespace: str = ATESPACE +) -> tuple[dict, list[dict], list[dict]]: """(project Workspace, Tasks, Run entries) of the loop's singletons: snapshot -> measure -> propose -> improve.""" git = [{"name": "repo", "repo": loop.repo, "dir": "repo", "depth": 1}] - project = _doc("Workspace", _ax_name("openultrasast", loop.commit[:12]), {"git": git}, ATESPACE) + project = _doc("Workspace", _ax_name("openultrasast", loop.commit[:12]), {"git": git}, atespace) project["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"repo={loop.commit}"} bound = [(project["metadata"]["name"], PROJECT_PATH)] env = { @@ -399,6 +402,7 @@ def render( population: str = "", split: str = "", loop: Loop | None = None, + atespace: str = ATESPACE, ) -> dict[str, str]: """Relative path under the plane root -> file text, for every generated manifest. ``population`` and ``split`` (the population file's and the set's stems) become the Run's annotations, which the memory rows carry. Each @@ -411,7 +415,7 @@ def render( tiebreak: list[dict[str, Any]] = [] for item in cases: case_id = item.case["id"] - pinned, inputs = _workspaces(item) + pinned, inputs = _workspaces(item, atespace) for doc in (pinned, inputs): out[f"workspaces/{doc['metadata']['name']}.yaml"] = _dump(header, [doc]) repo = [(pinned["metadata"]["name"], CASE_PATH), (inputs["metadata"]["name"], INPUTS_PATH)] @@ -451,7 +455,7 @@ def render( entries.extend(tiebreak) # appended: the first Run's entries stay byte-identical, so a rerun keeps their state for item in cases: # appended after the tie-break for the same reason if loop is not None: - fixed, alerts_task, alerts_entry = _alerts(item, templates) + fixed, alerts_task, alerts_entry = _alerts(item, templates, atespace) out[f"workspaces/{fixed['metadata']['name']}.yaml"] = _dump(header, [fixed]) tasks.append(alerts_task) entries.append(alerts_entry) @@ -462,7 +466,7 @@ def render( tasks.append(remember_task) entries.append(remember_entry) if loop is not None: - project, loop_tasks, loop_entries = _loop(cases, templates, loop) + project, loop_tasks, loop_entries = _loop(cases, templates, loop, atespace) out[f"workspaces/{project['metadata']['name']}.yaml"] = _dump(header, [project]) tasks.extend(loop_tasks) entries.extend(loop_entries) @@ -482,11 +486,18 @@ _IMAGE_LINE = re.compile(r'^(\s*image:\s*)"?[^"\s]+"?\s*$', re.MULTILINE) def repin_templates(plane: Path, image_file: Path) -> str: - """Set every template's ``image:`` to the digest-pinned reference in ``image_file`` (as ``ops/ax/up.sh`` - writes it); comments and layout are kept. Returns the reference.""" - image = image_file.read_text(encoding="utf-8").strip() - if not re.fullmatch(r"\S+@sha256:[0-9a-f]{64}", image): - raise ValueError(f"{image_file}: expected a digest-pinned image reference, got {image!r}") + """Set every template's ``image:`` to the runner reference of ``image_file``: a profile images file + (``{"runner": "/@sha256:..."}``, as ``ops/ax/up.sh`` writes it) or the older one-line pin; a + reference without a digest is refused. Comments and layout are kept. Returns the reference. (Task 4.3 gives + the engine template ``images["engine"]``; until then only the runner key is read.)""" + from .profile import ProfileError, load_images + + try: + image = load_images(image_file)["runner"] + except ProfileError as exc: + raise ValueError(str(exc)) from None + except KeyError: + raise ValueError(f"{image_file}: no runner image in the images file") from None for name in TEMPLATES: path = plane / "tasks" / f"{name}.yaml" text = path.read_text(encoding="utf-8") @@ -509,16 +520,24 @@ def increment( repos: Path | None = None, runner_image: Path | None = None, loop: Loop | None = None, + atespace: str | None = None, ) -> list[Path]: """Write the generated manifests under ``plane`` from the templates in ``plane/tasks`` (re-pinned first when - ``runner_image`` is given), with the improvement loop when ``loop`` is given; returns the paths.""" + ``runner_image`` is given), with the improvement loop when ``loop`` is given; returns the paths. Workspaces go + to ``atespace`` (default: the profile's).""" + if atespace is None: + from .profile import load_profile + + atespace = load_profile().atespace if loop is not None and repos is None: raise ValueError("--loop needs --repos: the alerts task marks fixed-pin alerts by the fix's new-side ranges") if runner_image is not None: repin_templates(plane, runner_image) loaded = load_manifests([plane / "tasks" / f"{name}.yaml" for name in TEMPLATES]) cases = read_cases(validation_set, candidates_dir, triage_dir, population, repos, fixed_side=loop is not None) - files = render(cases, loaded.tasks, run_name, command, population=population.stem, split=validation_set.stem, loop=loop) + files = render( + cases, loaded.tasks, run_name, command, population=population.stem, split=validation_set.stem, loop=loop, atespace=atespace + ) written: list[Path] = [] for relative, text in files.items(): path = plane / relative diff --git a/src/openultrasast/plane/harvest.py b/src/openultrasast/plane/harvest.py index a59a3c7..70354b6 100644 --- a/src/openultrasast/plane/harvest.py +++ b/src/openultrasast/plane/harvest.py @@ -210,17 +210,19 @@ def _reader(cache: Path, source: str, ref: str, pin: str) -> Any: # --- manifests ------------------------------------------------------------------------------------------------------- -def _git_workspace(unit: Unit) -> dict[str, Any]: - doc = _doc("Workspace", _ax_name(unit.name, "pin"), {"git": [{"name": REPO, "repo": unit.repo, "dir": REPO, "depth": 1}]}, ATESPACE) +def _git_workspace(unit: Unit, atespace: str = ATESPACE) -> dict[str, Any]: + doc = _doc("Workspace", _ax_name(unit.name, "pin"), {"git": [{"name": REPO, "repo": unit.repo, "dir": REPO, "depth": 1}]}, atespace) doc["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"{REPO}={unit.pin}"} return doc -def _files_workspace(name: str, files: Mapping[str, str]) -> dict[str, Any]: - return _doc("Workspace", name, {"files": [{"path": k, "content": v} for k, v in sorted(files.items())]}, ATESPACE) +def _files_workspace(name: str, files: Mapping[str, str], atespace: str = ATESPACE) -> dict[str, Any]: + return _doc("Workspace", name, {"files": [{"path": k, "content": v} for k, v in sorted(files.items())]}, atespace) -def _verify_docs(unit: Unit, templates: Mapping[str, Task], usd_per_hunt: float) -> tuple[list[dict], list[dict], list[dict]]: +def _verify_docs( + unit: Unit, templates: Mapping[str, Task], usd_per_hunt: float, atespace: str = ATESPACE +) -> tuple[list[dict], list[dict], list[dict]]: """(Workspaces, Tasks, Run entries) of one verify unit.""" asked = list(unit.verify or unit.candidates) candidates = {"family": unit.family, "candidates": [list(c) for c in asked]} @@ -232,15 +234,15 @@ def _verify_docs(unit: Unit, templates: Mapping[str, Task], usd_per_hunt: float) entries: list[dict] = [] if unit.files: files = {f"{REPO}/{path}": text for path, text in unit.files} | {"inputs/candidates.json": _json(candidates)} - workspace = _files_workspace(_ax_name(unit.name, "side"), files) + workspace = _files_workspace(_ax_name(unit.name, "side"), files, atespace) workspaces.append(workspace) bindings = [(workspace["metadata"]["name"], CASE_PATH)] inputs = f"{CASE_PATH}/inputs/candidates.json" else: - pinned = _git_workspace(unit) + pinned = _git_workspace(unit, atespace) functions = [{"path": p, "function": fn, "line": ln} for p, fn, ln in asked] files_doc = _files_workspace( - _ax_name(unit.name, "inputs"), {"candidates.json": _json(candidates), "functions.json": _json(functions)} + _ax_name(unit.name, "inputs"), {"candidates.json": _json(candidates), "functions.json": _json(functions)}, atespace ) workspaces += [pinned, files_doc] bindings = [(pinned["metadata"]["name"], CASE_PATH), (files_doc["metadata"]["name"], INPUTS_PATH)] @@ -309,7 +311,7 @@ def _roles_groups(units: Sequence[Unit]) -> list[list[Unit]]: def render_harvest( - units: Sequence[Unit], templates: Mapping[str, Task], name: str, command: str, *, ceiling: float = 10.0 + units: Sequence[Unit], templates: Mapping[str, Task], name: str, command: str, *, ceiling: float = 10.0, atespace: str = ATESPACE ) -> dict[str, str]: """Relative path under the plane root -> file text: both Runs, their Workspaces and Tasks, and ``units.json``. A population file whose size is unknown is estimated at :data:`UNKNOWN_SIZE`.""" @@ -323,7 +325,7 @@ def render_harvest( tasks: list[dict] = [] entries: list[dict] = [] for unit in verify_units: - w, t, e = _verify_docs(unit, templates, USD_PER_HUNT[_kind(unit)] * scale) + w, t, e = _verify_docs(unit, templates, USD_PER_HUNT[_kind(unit)] * scale, atespace) workspaces += w tasks += t entries += e @@ -333,7 +335,7 @@ def render_harvest( for group in _roles_groups([u for u in units if u.files and _inline(u, roles=True) <= MAX_INLINE_BYTES]): files = {f"{REPO}/{u.name}/{p}": t for u in group for p, t in u.files if p in u.paths} listing = sorted(f.removeprefix(f"{REPO}/") for f in files) - workspace = _files_workspace(_ax_name(name, "roles", group[0].name), files | {"inputs/files.json": _json(listing)}) + workspace = _files_workspace(_ax_name(name, "roles", group[0].name), files | {"inputs/files.json": _json(listing)}, atespace) workspaces.append(workspace) estimates.append(_roles_estimate((len(text), len(text.splitlines())) for text in files.values())) bound = [(workspace["metadata"]["name"], CASE_PATH)] @@ -350,8 +352,8 @@ def render_harvest( same = [u for u in units if not u.files and (u.repo, u.pin) == (unit.repo, unit.pin)] paths = sorted({p for u in same for p in u.paths}) known = {p: (chars, lines) for u in same for p, chars, lines in u.sizes} - pinned = _git_workspace(unit) - files_doc = _files_workspace(_ax_name(unit.name, "roles-in"), {"files.json": _json(paths)}) + pinned = _git_workspace(unit, atespace) + files_doc = _files_workspace(_ax_name(unit.name, "roles-in"), {"files.json": _json(paths)}, atespace) workspaces += [w for w in (pinned, files_doc) if w["metadata"]["name"] not in emitted] estimates.append(_roles_estimate(known.get(p, UNKNOWN_SIZE) for p in paths)) bound = [(pinned["metadata"]["name"], CASE_PATH), (files_doc["metadata"]["name"], INPUTS_PATH)] @@ -380,7 +382,16 @@ def _run_files(header: str, run: str, workspaces: list[dict], tasks: list[dict], def write_harvest( - labels: Path, plane: Path, *, name: str, command: str, catalog: Path, cache: Path, templates: Path, ceiling: float = 10.0 + labels: Path, + plane: Path, + *, + name: str, + command: str, + catalog: Path, + cache: Path, + templates: Path, + ceiling: float = 10.0, + atespace: str = ATESPACE, ) -> dict[str, str]: """Build the units, render, write under ``plane`` (with the Model manifests copied beside); returns the rendered files.""" @@ -388,7 +399,7 @@ def write_harvest( units = pair_units(labels, catalog) + population_units(labels, cache) loaded = load_manifests([templates / "tasks" / f"{n}.yaml" for n in ("repo-facts", "verify", "agree", "roles")]).tasks - files = render_harvest(units, loaded, name, command, ceiling=ceiling) + files = render_harvest(units, loaded, name, command, ceiling=ceiling, atespace=atespace) for path in sorted((templates / "models").glob("*.yaml")): files[f"models/{path.name}"] = path.read_text(encoding="utf-8") for relative, text in files.items(): diff --git a/src/openultrasast/plane/profile.py b/src/openultrasast/plane/profile.py new file mode 100644 index 0000000..d7a752c --- /dev/null +++ b/src/openultrasast/plane/profile.py @@ -0,0 +1,266 @@ +"""The plane's configuration surface: one ``PlaneProfile`` per cluster (plane-on-kubernetes design section 1). + +A profile is a TOML file under ``ops/k8s/profiles/`` (``kind.toml`` for the laptop's kind cluster, ``k3s.toml`` for +the production cluster) holding every value that differs between clusters: the execution mode, the kube context, +the registry and the digest-pinned images file, the router and ax-server addresses (empty: tunnel through the +Kubernetes API), the memory store, the atespace, ax's snapshot bucket, the worker pools and the image pull secret. +:func:`load_profile` reads the file ``OUSAST_PLANE_PROFILE`` names (a profile name or a path; default ``kind``) and +then applies the environment: every scalar field is overridable by ``OUSAST_`` (``OUSAST_KUBE_CONTEXT``, +``OUSAST_REGISTRY``, ``OUSAST_MEMORY``, ``OUSAST_ROUTER_URL``, ...), a pool by ``OUSAST_POOL__``. + +A profile holds no secret: a key whose name ends in ``key``, ``secret``, ``token`` or ``password`` is refused, and so +is any unknown key, an ``exec`` outside ``local``/``remote``, and an images file whose references are not pinned by +digest (Agent Substrate rejects tags). Provider and store credentials stay in the environment or ``.env``. + +``python -m openultrasast.plane.profile --print FIELD`` prints one value for shell scripts (``ops/ax/up.sh``): +``registry``, ``kube_context``, ``images.runner``, ``pools.default.replicas``, ... +""" + +from __future__ import annotations + +import json +import os +import re +import sys +import tomllib +from collections.abc import Mapping +from dataclasses import dataclass, field, fields +from pathlib import Path +from typing import Any + +__all__ = ["DEFAULT_PROFILE", "PROFILES_DIR", "PlaneProfile", "Pool", "ProfileError", "load_images", "load_profile", "profiles_dir"] + +DEFAULT_PROFILE = "kind" +PROFILES_DIR = Path("ops/k8s/profiles") +EXEC_MODES = ("local", "remote") +_DIGEST = re.compile(r"\S+@sha256:[0-9a-f]{64}") +_SECRET_KEY = re.compile(r"(key|secret|token|password)$", re.IGNORECASE) +_SECRET_VALUE = re.compile(r"(sk-[A-Za-z0-9]{8,}|AKIA[0-9A-Z]{12,}|[A-Za-z0-9+/]{40,}={0,2}$)") +_ENV_PREFIX = "OUSAST_" + + +class ProfileError(ValueError): + """A profile that cannot be used; the message names the file and the field.""" + + +@dataclass(frozen=True) +class Pool: + """One WorkerPool's size: how many warm workers, and each worker's CPU and memory limit.""" + + replicas: int + cpu: str + memory: str + + +@dataclass(frozen=True) +class PlaneProfile: + """The values of one cluster. ``images`` is resolved against the profile file's directory.""" + + name: str + path: Path + exec: str = "local" + kube_context: str = "" + registry: str = "" + images: Path = Path("images.json") + router_url: str = "" + ax_server: str = "" + memory: str = "" + atespace: str = "default" + snapshots_bucket: str = "" + image_pull_secret: str = "" + pools: Mapping[str, Pool] = field(default_factory=dict) + + def load_images(self) -> dict[str, str]: + """The digest-pinned image references of ``images`` (``{"runner": ..., "engine": ...}``).""" + return load_images(self.images) + + def addresses(self) -> dict[str, str]: + """Every configured address, for ``ousast plane doctor`` (Req 1.2); an empty router or ax-server address + means the Kubernetes API tunnel.""" + return { + "exec": self.exec, + "kube_context": self.kube_context, + "registry": self.registry, + "images": str(self.images), + "router_url": self.router_url or "(tunnel through the kube context)", + "ax_server": self.ax_server or "(tunnel through the kube context)", + "memory": self.memory, + "atespace": self.atespace, + "snapshots_bucket": self.snapshots_bucket, + "image_pull_secret": self.image_pull_secret or "(none)", + } + + +_SCALARS = {f.name: str(f.type) for f in fields(PlaneProfile) if f.name not in ("name", "path", "pools")} +_POOL_FIELDS = {f.name: str(f.type) for f in fields(Pool)} + + +def profiles_dir() -> Path: + """``ops/k8s/profiles`` of this checkout (``OUSAST_PLANE_PROFILES`` overrides; an installed package falls back + to the working directory).""" + override = os.environ.get("OUSAST_PLANE_PROFILES") + if override: + return Path(override) + checkout = Path(__file__).resolve().parents[3] / PROFILES_DIR + return checkout if checkout.is_dir() else PROFILES_DIR + + +def load_images(path: Path) -> dict[str, str]: + """``{"runner": "/@sha256:<64 hex>", ...}`` from a JSON images file; every value must be + digest-pinned. A one-line file holding a single reference (``ops/ax/up.sh`` before this profile) reads as + ``{"runner": }``.""" + try: + text = path.read_text(encoding="utf-8").strip() + except OSError as exc: + raise ProfileError(f"images file {path}: {exc.strerror or exc}") from None + try: + loaded: Any = json.loads(text) if text.startswith("{") else {"runner": text} + except ValueError as exc: + raise ProfileError(f"images file {path}: not JSON ({exc})") from None + if not isinstance(loaded, dict) or not loaded or not all(isinstance(k, str) and isinstance(v, str) for k, v in loaded.items()): + raise ProfileError(f"images file {path}: expected an object of image names to references") + for name, reference in loaded.items(): + if not _DIGEST.fullmatch(reference): + raise ProfileError(f"images file {path}: {name} is not digest-pinned ({reference!r}); Agent Substrate rejects tags") + return dict(loaded) + + +def _reject_secret(where: str, key: str, value: object, declared: bool = False) -> None: + """A key that is not a declared field and is named like a credential, or any value shaped like one, is refused. + ``image_pull_secret`` is declared: its value is the *name* of a Kubernetes Secret, never its content.""" + if not declared and _SECRET_KEY.search(key): + raise ProfileError(f"{where}: {key} looks like a credential; profiles hold no secret (use the environment or .env)") + if isinstance(value, str) and _SECRET_VALUE.search(value): + raise ProfileError(f"{where}: the value of {key} looks like a credential; profiles hold no secret") + + +def _coerce(where: str, key: str, value: object, kind: str) -> Any: + if kind == "int": + if isinstance(value, bool) or not isinstance(value, int): + raise ProfileError(f"{where}: {key} must be an integer, got {value!r}") + return value + if kind == "Path": + if not isinstance(value, str) or not value: + raise ProfileError(f"{where}: {key} must be a file name") + return Path(value) + if not isinstance(value, str): + raise ProfileError(f"{where}: {key} must be a string, got {value!r}") + return value + + +def _pools(where: str, raw: object) -> dict[str, Pool]: + if not isinstance(raw, dict): + raise ProfileError(f"{where}: pools must be a table of pool names") + out: dict[str, Pool] = {} + for name, spec in raw.items(): + if not isinstance(spec, dict): + raise ProfileError(f"{where}: pools.{name} must be a table") + unknown = sorted(set(spec) - set(_POOL_FIELDS)) + if unknown: + raise ProfileError(f"{where}: pools.{name} has unknown keys: {', '.join(unknown)}") + missing = sorted(set(_POOL_FIELDS) - set(spec)) + if missing: + raise ProfileError(f"{where}: pools.{name} lacks {', '.join(missing)}") + for key, value in spec.items(): + _reject_secret(where, f"pools.{name}.{key}", value, declared=True) + out[str(name)] = Pool(**{k: _coerce(where, f"pools.{name}.{k}", spec[k], _POOL_FIELDS[k]) for k in _POOL_FIELDS}) + return out + + +def _env_overrides(environ: Mapping[str, str], where: str, values: dict[str, Any], pools: dict[str, Pool]) -> None: + for name, kind in _SCALARS.items(): + raw = environ.get(_ENV_PREFIX + name.upper()) + if raw is not None and raw != "": + value: Any = raw + if kind == "int": + try: + value = int(raw) + except ValueError: + raise ProfileError(f"{where}: {_ENV_PREFIX}{name.upper()} must be an integer, got {raw!r}") from None + values[name] = _coerce(where, _ENV_PREFIX + name.upper(), value, kind) + pattern = re.compile(rf"^{_ENV_PREFIX}POOL_([A-Z0-9]+)_(REPLICAS|CPU|MEMORY)$") + for variable, raw in environ.items(): + found = pattern.match(variable) + if not found or raw == "": + continue + pool_name, key = found.group(1).lower(), found.group(2).lower() + current = pools.get(pool_name) + if current is None: + raise ProfileError(f"{where}: {variable} names a pool the profile does not declare ({pool_name})") + value = int(raw) if key == "replicas" and raw.isdigit() else raw + pools[pool_name] = Pool(**{**vars(current), key: _coerce(where, variable, value, _POOL_FIELDS[key])}) + + +def load_profile(name: str | None = None, environ: Mapping[str, str] | None = None) -> PlaneProfile: + """The profile ``name`` (a name under :func:`profiles_dir` or a path to a ``.toml``; default + ``OUSAST_PLANE_PROFILE``, else ``kind``), with the environment's overrides applied; :class:`ProfileError` + names what is wrong.""" + env = os.environ if environ is None else environ + chosen = name or env.get("OUSAST_PLANE_PROFILE") or DEFAULT_PROFILE + path = Path(chosen) if chosen.endswith(".toml") or "/" in chosen else profiles_dir() / f"{chosen}.toml" + where = str(path) + try: + raw = tomllib.loads(path.read_text(encoding="utf-8")) + except OSError as exc: + raise ProfileError(f"profile {where}: {exc.strerror or exc}") from None + except tomllib.TOMLDecodeError as exc: + raise ProfileError(f"profile {where}: {exc}") from None + for key, value in raw.items(): # a credential-shaped key is named as such before it is merely unknown + _reject_secret(where, key, value, declared=key in _SCALARS) + unknown = sorted(set(raw) - set(_SCALARS) - {"pools"}) + if unknown: + raise ProfileError(f"{where}: unknown keys: {', '.join(unknown)}") + values: dict[str, Any] = {} + for key, value in raw.items(): + if key != "pools": + values[key] = _coerce(where, key, value, _SCALARS[key]) + pools = _pools(where, raw.get("pools", {})) + _env_overrides(env, where, values, pools) + if values.get("exec", "local") not in EXEC_MODES: + raise ProfileError(f"{where}: exec must be one of {', '.join(EXEC_MODES)}, got {values.get('exec')!r}") + images = Path(values.get("images", "images.json")) + if not images.is_absolute(): + images = path.resolve().parent / images + values["images"] = images + if images.is_file(): + load_images(images) # a committed file must be pinned; a file the release has not published yet is doctor's finding + return PlaneProfile(name=path.stem, path=path, pools=pools, **values) + + +def _lookup(profile: PlaneProfile, dotted: str) -> str: + head, _, rest = dotted.partition(".") + if head == "images" and rest: + images = profile.load_images() + if rest not in images: + raise ProfileError(f"{profile.images}: no image named {rest!r} (have {', '.join(sorted(images))})") + return images[rest] + if head == "pools": + pool_name, _, key = rest.partition(".") + if pool_name not in profile.pools or key not in _POOL_FIELDS: + raise ProfileError(f"{profile.path}: no such pool field {dotted!r}") + return str(getattr(profile.pools[pool_name], key)) + if head not in _SCALARS or rest: + raise ProfileError(f"{profile.path}: no such field {dotted!r}") + return str(getattr(profile, head)) + + +def main(argv: list[str] | None = None) -> int: + """``python -m openultrasast.plane.profile [--profile NAME] --print FIELD [FIELD...]``: one value per line.""" + import argparse + + parser = argparse.ArgumentParser(prog="openultrasast.plane.profile", description=main.__doc__) + parser.add_argument("--profile", help="profile name or path (default OUSAST_PLANE_PROFILE, else kind)") + parser.add_argument("--print", dest="fields", nargs="+", required=True, metavar="FIELD") + args = parser.parse_args(argv) + try: + profile = load_profile(args.profile) + for dotted in args.fields: + print(_lookup(profile, dotted)) + except ProfileError as exc: + print(f"profile: {exc}", file=sys.stderr) + return 2 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/openultrasast/plane/reconciler.py b/src/openultrasast/plane/reconciler.py index de7ad92..279d893 100644 --- a/src/openultrasast/plane/reconciler.py +++ b/src/openultrasast/plane/reconciler.py @@ -1,23 +1,26 @@ """ax-backed reconciler of a ``Run``: submission, resume, start, completion by delivery (ai-service-plane Req 3, 4, 7). -ax executes every task on this host's kind cluster; nothing here runs a task locally or knows a prompt, a score or -a model call. Per ready task: one rendered YAML file (the Task with its env extended, its Workspaces, the bound -Model), ``ax apply -f``, ``ax resume task`` (retried on DeadlineExceeded/Unavailable -- a new image's first resume -waits for Agent Substrate's golden snapshot -- for up to ``OUSAST_RESUME_TIMEOUT`` s, 900), and once ax says +ax executes every task on the profile's cluster (``profile.py``: kube context, registry, store); nothing here runs a +task locally or knows a prompt, a score or a model call. Per ready task: one rendered YAML file (the Task with its +env extended, its Workspaces, the bound Model), ``ax apply -f``, ``ax resume task`` (retried on +DeadlineExceeded/Unavailable -- a new image's first resume waits for Agent Substrate's golden snapshot -- for up to +``OUSAST_RESUME_TIMEOUT`` s, 900), and once ax says ``Running`` one start request through Substrate's router (``router.py``, Req 4.5) carrying the bound Model's -credential from this process's environment, never from a file (Req 4.6). Completion is the runner's artifact -delivery (ax has no Completed phase); ``Failed``, the task vanishing, a refused start or ``OUSAST_TASK_TIMEOUT`` s -(7200) without delivery fail it; ``ax delete task`` follows. +credential from this process's environment, never from a file (Req 4.6), and the presigned store URLs of +``delivery.py`` (plane-on-kubernetes design section 3). Completion is the runner's output tar appearing in the +memory store (ax has no Completed phase), which the reconciler polls, downloads, extracts and re-publishes per +declared output; ``Failed``, the task vanishing, a refused start or ``OUSAST_TASK_TIMEOUT`` s (7200) without +delivery fail it; ``ax delete task`` follows. ``OUSAST_DELIVERY=receiver`` keeps the host receiver of the kind +profile for group 2 only (task 2.6 removes it). ax rejects unknown fields, so rendered documents carry only ax's fields; ``openultrasast.io/model`` reaches the task as ``OUSAST_MODEL`` plus ``OUSAST_MODEL_PARAMS`` (Req 7.1), each Workspace's pin annotation as ``OUSAST_GIT_PINS``. Inputs other tasks produced are never rendered: ``OUSAST_INPUTS`` maps each input's name to -``/``, which the runner fetches after the start from the receiver's ``GET /inputs/...`` to -``OUSAST_INPUT_`` under ``/workspace/.ousast-in/`` (the receiver serves a running task its declared -inputs only). The receiver takes the runner's tar (``X-Ousast-Run``, ``X-Ousast-Task``) on -``OUSAST_ARTIFACT_PORT`` (18090), reached by its Service name through the egress gateway; between ``ax apply`` and -resume the task's egress policy is written (``egress.py``). -State: ``~/ousast-results/plane//`` (``OUSAST_RESULTS``), polled every ``OUSAST_POLL_SECONDS`` (3). +``/``, fetched after the start by the presigned GET of the start body to ``OUSAST_INPUT_`` +under ``/workspace/.ousast-in/``. Between ``ax apply`` and resume the task's egress policy is written +(``egress.py``). State: ``~/ousast-results/plane//`` (``OUSAST_RESULTS``), polled every ``OUSAST_POLL_SECONDS`` +(3), mirrored to ``runs//state.json`` on the store after every change and seeded from there by a rerun with no +local state (design section 2); the store is opened first and required, there is no fallback. """ from __future__ import annotations @@ -41,9 +44,12 @@ from urllib.parse import urlsplit import yaml +from .delivery import Delivery, DeliveryError, declared from .doctor import doctor from .egress import RECEIVER_PORT, Egress, Receiver, policy_for from .manifests import AX_API_VERSION, Manifests, Model, Run, RunTask, Task, Workspace, load_manifests +from .memory import open_store +from .profile import PlaneProfile, load_profile from .router import Router, credentials, open_router MODEL_ANNOTATION = "openultrasast.io/model" @@ -136,15 +142,16 @@ def _model_doc(model: Model) -> dict[str, Any]: return _doc("Model", model.metadata.name, spec, model.metadata.atespace) -def render_task(run: Run, entry: RunTask, manifests: Manifests, artifact_url: str, dial: str | None = None) -> list[dict[str, Any]]: +def render_task(run: Run, entry: RunTask, manifests: Manifests, artifact_url: str = "", dial: str | None = None) -> list[dict[str, Any]]: """The documents ``ax apply`` receives for one Run task: its Model, its Workspaces, the Task last. Task-produced - inputs are never rendered: ax puts every bound Workspace into one env value, which Substrate caps at 32768 - characters, so the runner fetches them from the receiver (``OUSAST_INPUTS``) after the start.""" + inputs are never rendered (Substrate caps one env value at 32768 characters): the runner fetches them + (``OUSAST_INPUTS``) after the start by presigned URL, or from the receiver when ``artifact_url`` is given.""" task: Task = manifests.tasks[entry.task] ax_name = _ax_name(run.metadata.name, entry.name, limit=TASK_NAME_LIMIT) env = {e.name: e.value for e in task.env} env.update(OUSAST_RUN=run.metadata.name, OUSAST_TASK=entry.name, OUSAST_OUTPUT_DIR=f"{OUTPUT_ROOT}/{entry.name}") - env.update(OUSAST_ARTIFACT_URL=artifact_url, **({"OUSAST_ARTIFACT_DIAL": dial} if dial else {})) + if artifact_url: # receiver mode only (task 2.6 removes it): the store path carries no URL in the env + env.update(OUSAST_ARTIFACT_URL=artifact_url, **({"OUSAST_ARTIFACT_DIAL": dial} if dial else {})) budget = _fields(entry.budget) if entry.budget else {} env.update({f"OUSAST_BUDGET_{k.upper()}": str(v) for k, v in budget.items()}) inputs = {re.sub(r"[^A-Z0-9]+", "_", name.upper()): ref for name, ref in entry.inputs.items()} @@ -224,12 +231,25 @@ class _State: path: Path data: dict[str, Any] lock: threading.Lock = field(default_factory=threading.Lock) + mirror: Callable[[Mapping[str, Any]], None] | None = None # the store copy; a failure is reported, never fatal def set(self, task: str, **fields: object) -> None: with self.lock: self.data["tasks"].setdefault(task, {"status": "pending", "started": None, "finished": None, "usd": None, "model": None}) self.data["tasks"][task].update(fields) - _write_json(self.path, self.data) + self._flush() + + def flush(self) -> None: + with self.lock: + self._flush() + + def _flush(self) -> None: + _write_json(self.path, self.data) + if self.mirror is not None: + try: + self.mirror(self.data) + except Exception as exc: # noqa: BLE001 - the local file is written; the mirror is reported, the run goes on + print(f"state mirror to the store failed: {exc}", file=sys.stderr) def status(self, task: str) -> str: return str(self.data["tasks"].get(task, {}).get("status", "pending")) @@ -313,25 +333,46 @@ def _await(ax: Ax, name: str, delivered: Callable[[], bool], start: Callable[[], def _execute( - ax: Ax, egress: Egress, run: Run, entry: RunTask, manifests: Manifests, base: Path, receiver: Receiver, router: Router + ax: Ax, + egress: Egress, + run: Run, + entry: RunTask, + manifests: Manifests, + base: Path, + delivery: Delivery, + router: Router, + receiver: Receiver | None = None, ) -> tuple[str, str]: ax_name, task = _ax_name(run.metadata.name, entry.name, limit=TASK_NAME_LIMIT), manifests.tasks[entry.task] atespace, manifest = task.metadata.atespace or "default", base / entry.name / "task.yaml" manifest.parent.mkdir(parents=True, exist_ok=True) - docs = render_task(run, entry, manifests, receiver.url, receiver.dial) + docs = render_task(run, entry, manifests, receiver.url if receiver else "", receiver.dial if receiver else None) manifest.write_text(yaml.safe_dump_all(docs, sort_keys=False), encoding="utf-8") model = manifests.models.get(task.metadata.annotations.get(MODEL_ANNOTATION) or "") - policy = policy_for(task, [manifests.workspaces[b.name] for b in task.workspaces], model, urlsplit(receiver.url).hostname) + workspaces = [manifests.workspaces[b.name] for b in task.workspaces] + body = None if receiver else delivery.body(entry.name, entry.inputs) # presigned URLs ride in the start body + store_host, receiver_host = (None, urlsplit(receiver.url).hostname) if receiver else (delivery.host, None) + if receiver is None: + delivery.reset(entry.name) # a previous attempt's tar must not complete this one + + def delivered() -> bool: + return entry.name in receiver.delivered if receiver is not None else delivery.delivered(entry.name) + try: # Req 4.6: the bound Model's credential is read here and only ever sent in the start request - start = router.starter(atespace, ax_name, run.metadata.name, credentials(model)) + policy = policy_for(task, workspaces, model, store_host, receiver_host) + start = router.starter(atespace, ax_name, run.metadata.name, credentials(model), body) ax.apply(manifest) # creates the Substrate actor; its egress is denied until the policy exists egress.apply(ax_name, atespace, policy) egress.settle() - except RuntimeError as exc: # StartError included: a Model-bound task whose secret is unset + except (RuntimeError, ValueError) as exc: # StartError (a Model's secret unset) and a policy that cannot be granted ax.delete(ax_name) return "failed", str(exc) try: - failure = _await(ax, ax_name, lambda: entry.name in receiver.delivered, start) + failure = _await(ax, ax_name, delivered, start) + if failure is None and receiver is None: + delivery.collect(entry.name, declared(run)[entry.name]) # download, extract, re-publish what is declared + except DeliveryError as exc: + failure = str(exc) finally: ax.delete(ax_name) leftover = egress.delete(ax_name, atespace) # the policy goes with its actor; a survivor is reported @@ -340,20 +381,44 @@ def _execute( return ("failed", failure) if failure else outcome_of(_read_json(base / entry.name / "summary.json")) -def run(run_manifest: Path, *, workers: int = 1, ax: str = "ax", results_root: Path | None = None, kubectl_ate: str | None = None) -> str: - """Execute a Run on ax and return its status: ``done``, ``unfinished`` or ``failed`` (Req 3.1-3.3, 2.2).""" +def run( + run_manifest: Path, + *, + workers: int = 1, + ax: str = "ax", + results_root: Path | None = None, + kubectl_ate: str | None = None, + profile: PlaneProfile | None = None, +) -> str: + """Execute a Run on ax and return its status: ``done``, ``unfinished`` or ``failed`` (Req 3.1-3.3, 2.2). Every + address comes from ``profile`` (default: the one ``OUSAST_PLANE_PROFILE`` names).""" + profile = profile or load_profile() run_spec, manifests = load_run(Path(run_manifest)) base = run_dir(run_spec.metadata.name, results_root) base.mkdir(parents=True, exist_ok=True) + store = open_store(profile.memory) # first, before the lock and before ax sees anything: no fallback + delivery = Delivery(store, run_spec.metadata.name, base) + if not (base / "state.json").exists(): + seeded = delivery.load_state() # a rerun from another machine or an empty results root + if seeded is not None: + _write_json(base / "state.json", seeded) lock = _acquire_lock(base) - state = _State(base / "state.json", _read_json(base / "state.json") or {"run": run_spec.metadata.name, "started": _now(), "tasks": {}}) - port = os.environ.get("OUSAST_ARTIFACT_PORT") or (0 if os.environ.get("OUSAST_ARTIFACT_HOST") else RECEIVER_PORT) - receiver = Receiver(run_spec.metadata.name, base, int(port)) # the receiver Service targets RECEIVER_PORT - threading.Thread(target=receiver.serve_forever, daemon=True).start() - cli, egress, finished = Ax(ax), Egress(kubectl_ate), queue.Queue[tuple[str, str, str]]() + state = _State( + base / "state.json", + _read_json(base / "state.json") or {"run": run_spec.metadata.name, "started": _now(), "tasks": {}}, + mirror=delivery.save_state, + ) + receiver: Receiver | None = None + if os.environ.get("OUSAST_DELIVERY") == "receiver": # the kind profile's host receiver, group 2 only (2.6 removes it) + port = os.environ.get("OUSAST_ARTIFACT_PORT") or (0 if os.environ.get("OUSAST_ARTIFACT_HOST") else RECEIVER_PORT) + receiver = Receiver(run_spec.metadata.name, base, int(port), profile.kube_context) # the Service targets RECEIVER_PORT + threading.Thread(target=receiver.serve_forever, daemon=True).start() + cli, egress, finished = Ax(ax), Egress(kubectl_ate, profile.kube_context), queue.Queue[tuple[str, str, str]]() pending = [t for t in run_spec.tasks if state.status(t.name) != "done"] for entry in run_spec.tasks: state.set(entry.name, **({"status": "pending"} if entry in pending else {})) + if entry not in pending: + delivery.publish(entry.name, declared(run_spec)[entry.name]) # a seeded or earlier task's artifacts reach its consumers running: dict[str, RunTask] = {} halted = False @@ -361,10 +426,14 @@ def run(run_manifest: Path, *, workers: int = 1, ax: str = "ax", results_root: P if state.status(entry.name) in ("running", "suspended"): cli.delete(_ax_name(run_spec.metadata.name, entry.name, limit=TASK_NAME_LIMIT)) # an interrupted attempt; ax must not keep it state.set(entry.name, status="running", started=_now(), finished=None) - finished.put((entry.name, *_execute(cli, egress, run_spec, entry, manifests, base, receiver, router))) + try: + outcome = _execute(cli, egress, run_spec, entry, manifests, base, delivery, router, receiver) + except Exception as exc: # noqa: BLE001 - a worker that dies silently would hang the run on the queue + outcome = ("failed", f"{type(exc).__name__}: {exc}") + finished.put((entry.name, *outcome)) try: - with open_router() as router: # one port-forward to Substrate's router for the whole run + with open_router(profile.kube_context, url=profile.router_url or None) as router: # one tunnel per run (ax-tunnel) while pending or running: blocked = {t.serialize for t in running.values() if t.serialize} for entry in list(pending): @@ -375,29 +444,32 @@ def run(run_manifest: Path, *, workers: int = 1, ax: str = "ax", results_root: P pending.remove(entry) running[entry.name] = entry blocked.add(entry.serialize or "") - with receiver.lock: # the receiver serves a running task its declared inputs, nothing else - receiver.running.add(entry.name) - receiver.inputs[entry.name] = frozenset(entry.inputs.values()) + if receiver is not None: + with receiver.lock: # the receiver serves a running task its declared inputs, nothing else + receiver.running.add(entry.name) + receiver.inputs[entry.name] = frozenset(entry.inputs.values()) threading.Thread(target=worker, args=(entry,), daemon=True).start() if not running: break # nothing can start: a producer stopped short name, status, reason = finished.get() del running[name] - with receiver.lock: - receiver.running.discard(name) - receiver.inputs.pop(name, None) + if receiver is not None: + with receiver.lock: + receiver.running.discard(name) + receiver.inputs.pop(name, None) summary = _read_json(base / name / "summary.json") or {} state.set(name, status=status, finished=_now(), usd=summary.get("usd"), model=summary.get("model"), reason=reason or None) halted = halted or status == "failed" if status == "unfinished": pending = [t for t in pending if name not in t.producers] finally: - receiver.shutdown() + if receiver is not None: + receiver.shutdown() lock.unlink(missing_ok=True) statuses = [state.status(t.name) for t in run_spec.tasks] result = "failed" if "failed" in statuses else "done" if all(s == "done" for s in statuses) else "unfinished" state.data.update(status=result, finished=_now()) - _write_json(state.path, state.data) + state.flush() return result diff --git a/src/openultrasast/plane/router.py b/src/openultrasast/plane/router.py index 80c6503..84cc010 100644 --- a/src/openultrasast/plane/router.py +++ b/src/openultrasast/plane/router.py @@ -28,6 +28,7 @@ import time import urllib.error import urllib.request from collections.abc import Callable, Iterable, Iterator, Mapping +from typing import Any from .manifests import Model @@ -77,7 +78,9 @@ class Router: def __init__(self, url: str) -> None: self.url = url.rstrip("/") - def starter(self, atespace: str, task: str, run: str, credentials: Mapping[str, str]) -> Callable[[], str | None]: + def starter( + self, atespace: str, task: str, run: str, credentials: Mapping[str, str], delivery: Mapping[str, Any] | None = None + ) -> Callable[[], str | None]: """:meth:`start_task` bound to one task, returning the refusal text instead of raising (None once accepted). The reconciler calls it again after a re-resume, since a resumed actor may have rebooted and wait for a @@ -90,20 +93,30 @@ class Router: nonlocal calls calls += 1 try: - self.start_task(atespace, task, run, credentials) + self.start_task(atespace, task, run, credentials, delivery) except StartError as exc: return None if calls > 1 and ALREADY_STARTED in str(exc) else str(exc) return None return start - def start_task(self, atespace: str, task: str, run: str, credentials: Mapping[str, str]) -> None: - """Start ``/``'s command once; :class:`StartError` on 409, another 4xx, or the timeout.""" - body = json.dumps({"run": run, "task": task, "credentials": dict(credentials)}).encode("utf-8") + def start_task( + self, atespace: str, task: str, run: str, credentials: Mapping[str, str], delivery: Mapping[str, Any] | None = None + ) -> None: + """Start ``/``'s command once; :class:`StartError` on 409, another 4xx, or the timeout. + ``delivery`` (``{put, inputs}`` presigned URLs, ``delivery.py``) rides in the body like the credentials and is + redacted like them.""" + payload: dict[str, Any] = {"run": run, "task": task, "credentials": dict(credentials)} + if delivery: + payload["delivery"] = dict(delivery) + body = json.dumps(payload).encode("utf-8") headers = {"Content-Type": "application/json", TARGET_HEADER: f"{atespace}/{task}"} limit = float(os.environ.get("OUSAST_START_TIMEOUT") or 300) deadline, backoff, last = time.monotonic() + limit, 0.5, "no attempt made" secrets = list(credentials.values()) + if delivery: + secrets.append(str(delivery.get("put") or "")) + secrets.extend(str(u) for u in (delivery.get("inputs") or {}).values()) while True: request = urllib.request.Request(self.url + START_PATH, data=body, method="POST", headers=headers) try: @@ -131,18 +144,20 @@ def _drain(stream: Iterable[str], lines: queue.Queue[str | None]) -> None: @contextlib.contextmanager -def open_router(context: str | None = None, *, kubectl: str = "kubectl", ready_timeout: float = 30.0) -> Iterator[Router]: +def open_router(context: str, *, url: str | None = None, kubectl: str = "kubectl", ready_timeout: float = 30.0) -> Iterator[Router]: """A :class:`Router` for the duration of the block. - ``OUSAST_ROUTER_URL`` set: that URL, nothing started. Otherwise ``kubectl port-forward svc/atenet-router :80`` - in ``ate-system`` of ``context`` (default ``kind-$KIND_CLUSTER_NAME``, ``kind-ousast``), the local port read - from kubectl's ``Forwarding from 127.0.0.1:NNNN`` line; on exit that child process is terminated. + ``url`` (the profile's ``router_url``) or ``OUSAST_ROUTER_URL`` set: that URL, nothing started. Otherwise + ``kubectl port-forward svc/atenet-router :80`` in ``ate-system`` of the profile's ``context`` (ax-tunnel: the + Kubernetes API is the one authenticated door), the local port read from kubectl's ``Forwarding from + 127.0.0.1:NNNN`` line; on exit that child process is terminated. No context is ever built here. """ - override = os.environ.get("OUSAST_ROUTER_URL") + override = url or os.environ.get("OUSAST_ROUTER_URL") if override: yield Router(override) return - context = context or "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast") + if not context: + raise StartError("open_router needs the profile's kube context (or router_url)") argv = [kubectl, "--context", context, "-n", NAMESPACE, "port-forward", SERVICE, ":80"] proc = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) try: diff --git a/src/openultrasast/plane/runner.py b/src/openultrasast/plane/runner.py index 22555f1..631dba8 100644 --- a/src/openultrasast/plane/runner.py +++ b/src/openultrasast/plane/runner.py @@ -10,18 +10,23 @@ not (ax docs/runner.md). The command never starts by itself (Requirements 4.5, 4.6). Agent Substrate boots every new ActorTemplate once as a "golden" actor to snapshot it, with this image and the same ``AX_TASK_YAML``/``AX_WORKSPACES_YAML``/env, and inside the sandbox nothing tells that boot from the real one. So after the workspaces are ready the runner waits for -``POST /ousast/v1/start`` with ``{"run", "task", "credentials"}``, which the reconciler sends through Substrate's +``POST /ousast/v1/start`` with ``{"run", "task", "credentials", "delivery"}``, which the reconciler sends through Substrate's router to the task's actor only (``plane/router.py``). ``run`` and ``task`` must match ``OUSAST_RUN`` and the Task's name (409 otherwise); one request per boot is accepted (202), later ones are 409, and a boot whose completion marker records a delivered run refuses every request (409 with the reason). Before the workspaces are ready the answer is 503, which the reconciler retries. The credentials live in this process's memory and go into the command's environment only; they are never logged or written, and their values are redacted from the command's echoed stderr. +``delivery`` (plane-on-kubernetes design section 3) is ``{"put": , "inputs": +{"": }}``: a presigned URL is a bearer capability, so it travels where the credentials do. +Each URL must be ``https`` to a hostname (never an address; ``AX_RUNNER_INSECURE_DELIVERY=1`` admits ``http`` in unit +tests only), the answer never echoes one, and no URL is logged. The command runs as a child process, ``python -m openultrasast.plane.tasks. ``, in its own process group, with the first workspace as its working directory and ``AX_METADATA_URL`` pointing at this server. ax has no artifact channel and never reports that a command finished, so after the child exits the runner reads ``summary.json`` (a crash without one becomes ``failed`` with the exit code and the stderr tail), -posts the whole ``OUSAST_OUTPUT_DIR`` as a tar to ``OUSAST_ARTIFACT_URL`` -- that delivery is the task's +PUTs the whole ``OUSAST_OUTPUT_DIR`` as a tar to ``delivery.put`` (or POSTs it to ``OUSAST_ARTIFACT_URL``, the receiver +path kept until task 2.6 removes it) -- that delivery is the task's completion (Requirement 4.4); a failed one is reported ``failed``, never done -- and writes the completion marker ``/.done.json`` (``exit``, ``status``, ``delivered``). Then it keeps serving until SIGTERM, as ax requires of PID 1. A boot that finds the marker never reruns the command (a resumed actor must not repeat @@ -38,6 +43,7 @@ Environment the runner reads, all optional except the two ax variables: AX_RUNNER_AUTOSTART "1" runs the command without waiting for a start request (unit tests only; required with AX_RUNNER_HTTP=0, where no request could arrive) AX_RUNNER_TASK_PACKAGE package the command's module is looked up in (unit tests only) + AX_RUNNER_INSECURE_DELIVERY "1" admits http delivery URLs to a hostname (unit tests only; the image never sets it) OUSAST_STATE_DIR completion markers, default ``/workspace/.ousast-state`` (the durable volume) OUSAST_TERM_GRACE seconds between SIGTERM and SIGKILL of the command's group, default 10 OUSAST_CASE_CACHE a directory of git checkouts (``benchmarks/independent`` cache layout) @@ -47,8 +53,9 @@ Environment the runner reads, all optional except the two ax variables: ``Host`` (the egress gateway decides on and resolves that name) OUSAST_DELIVERY_BACKOFF seconds before the second delivery attempt, doubled per retry OUSAST_INPUTS JSON ``{"": "/"}``: after the start and before the - command each is fetched from ``/inputs/`` (same - dial, headers and retries as delivery) to ``OUSAST_INPUT_``; a failed + command each is fetched from ``delivery.inputs[NAME]`` (else from + ``/inputs/`` with the receiver's dial and headers) + to ``OUSAST_INPUT_``, with the same retries as delivery; a failed fetch delivers a failed summary naming the input, the command never runs The runner exports ``OUSAST_WORKSPACE_DIR`` (the first bound workspace) when the Task's env leaves it unset, @@ -60,6 +67,7 @@ from __future__ import annotations import contextlib import http.client import io +import ipaddress import json import logging import os @@ -76,7 +84,7 @@ import urllib.parse import urllib.request from collections import deque from collections.abc import Mapping, MutableMapping, Sequence -from dataclasses import dataclass +from dataclasses import dataclass, field from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path from typing import Any @@ -206,14 +214,58 @@ _CREDENTIAL_NAME = re.compile(r"[A-Z][A-Z0-9_]*") _RESERVED_PREFIXES = ("AX_", "OUSAST_") # the runner's own contract; a credential must not redirect it +@dataclass(frozen=True) +class DeliveryTargets: + """Where this task PUTs its output tar and GETs each input (presigned URLs from the start request).""" + + put: str | None = None + inputs: Mapping[str, str] = field(default_factory=dict) + + def __repr__(self) -> str: # a presigned URL is a credential: never print one + return f"DeliveryTargets(put={'yes' if self.put else 'no'}, inputs={sorted(self.inputs)})" + + @dataclass(frozen=True) class Start: - """An accepted start request: the credentials go to the command's environment and nowhere else.""" + """An accepted start request: the credentials go to the command's environment and nowhere else; the delivery + targets go to the artifact PUT and the input GETs and nowhere else.""" credentials: Mapping[str, str] + delivery: DeliveryTargets | None = None def __repr__(self) -> str: # never print a value by accident - return f"Start(credentials={sorted(self.credentials)})" + return f"Start(credentials={sorted(self.credentials)}, delivery={self.delivery!r})" + + +def _check_url(url: object, where: str, insecure: bool) -> str: + """``url`` is a string, ``https`` (``http`` only when ``insecure``) to a hostname that is not an address, without + userinfo; the error names the field and never echoes the value.""" + if not isinstance(url, str): + raise ValueError(f"{where} must be a URL string") + parts = urlsplit(url) + allowed = ("https", "http") if insecure else ("https",) + host = parts.hostname or "" + if parts.scheme not in allowed or not host or parts.username is not None or parts.password is not None: + raise ValueError(f"{where} must be an https URL to a hostname") + try: + ipaddress.ip_address(host.strip("[]")) + except ValueError: + return url + raise ValueError(f"{where} must name a host, not an address") + + +def parse_delivery(raw: object, insecure: bool = False) -> DeliveryTargets | None: + """The start body's ``delivery`` field validated: None when absent; :class:`ValueError` names the bad field.""" + if raw is None: + return None + if not isinstance(raw, dict) or not set(raw) <= {"put", "inputs"}: + raise ValueError("delivery must be an object with put and/or inputs") + put = _check_url(raw["put"], "delivery.put", insecure) if raw.get("put") is not None else None + inputs_raw = raw.get("inputs") or {} + if not isinstance(inputs_raw, dict) or not all(isinstance(k, str) and _CREDENTIAL_NAME.fullmatch(k) for k in inputs_raw): + raise ValueError("delivery.inputs must map ^[A-Z][A-Z0-9_]*$ names to URLs") + inputs = {name: _check_url(url, f"delivery.inputs.{name}", insecure) for name, url in inputs_raw.items()} + return DeliveryTargets(put, inputs) class StartGate: @@ -225,7 +277,8 @@ class StartGate: logged, and no answer echoes it. """ - def __init__(self) -> None: + def __init__(self, insecure: bool = False) -> None: + self._insecure = insecure # AX_RUNNER_INSECURE_DELIVERY=1: http delivery URLs (unit tests only) self._lock = threading.Lock() self._accepted = threading.Event() self._expected: tuple[str, str] | None = None @@ -267,10 +320,14 @@ class StartGate: for k, v in creds.items() ): return 400, "credentials must map ^[A-Z][A-Z0-9_]*$ names (not AX_/OUSAST_) to strings" + try: + delivery = parse_delivery(request.get("delivery"), self._insecure) + except ValueError as exc: + return 400, str(exc) run, task = self._expected if request.get("run") != run or request.get("task") != task: return 409, f"this actor runs task {task!r} of run {run!r}" - self._start = Start(dict(creds)) + self._start = Start(dict(creds), delivery) self._accepted.set() return 202, "started" @@ -653,6 +710,12 @@ def _dialled(url: str, headers: Mapping[str, str], dial: str | None) -> tuple[st return parts._replace(netloc=dial).geturl(), out +def safe_url(url: str) -> str: + """``scheme://host/path`` of a URL: what a log line or an error may carry (a presigned query is a credential).""" + parts = urlsplit(url) + return f"{parts.scheme}://{parts.netloc}{parts.path}" + + def deliver( url: str, payload: bytes, @@ -661,15 +724,17 @@ def deliver( attempts: int = DELIVERY_ATTEMPTS, backoff: float = 1.0, dial: str | None = None, + method: str = "POST", ) -> int: - """POST ``payload`` to ``url`` with ``headers``; retry with doubling backoff; return the 2xx status. + """Send ``payload`` to ``url`` (``POST`` to the receiver, ``PUT`` to a presigned store URL) with ``headers``; retry + with doubling backoff; return the 2xx status. Log lines and errors carry the URL without its query. ``dial`` (``host:port``): connect there instead and send the URL's host as ``Host``, as the actor must -- it cannot resolve the receiver's cluster name, and the egress gateway decides on ``Host`` and resolves it itself. """ last, (target, headers) = "no attempt made", _dialled(url, headers, dial) for attempt in range(1, attempts + 1): - request = urllib.request.Request(target, data=payload, method="POST", headers=headers) + request = urllib.request.Request(target, data=payload, method=method, headers=headers) try: with urllib.request.urlopen(request, timeout=60) as response: # noqa: S310 - the reconciler's URL status = int(response.status) @@ -680,10 +745,10 @@ def deliver( last = f"HTTP {exc.code}" except (urllib.error.URLError, http.client.HTTPException, OSError, ValueError) as exc: last = f"{type(exc).__name__}: {exc}" - log.warning("artifact delivery attempt %d/%d to %s failed: %s", attempt, attempts, url, last) + log.warning("artifact delivery attempt %d/%d to %s failed: %s", attempt, attempts, safe_url(url), last) if attempt < attempts: time.sleep(backoff * (2 ** (attempt - 1))) - raise DeliveryError(f"{last} after {attempts} attempts to {url}") + raise DeliveryError(f"{last} after {attempts} attempts to {safe_url(url)}") def fetch( @@ -711,14 +776,15 @@ def fetch( except (urllib.error.URLError, http.client.HTTPException, OSError, ValueError) as exc: last = f"{type(exc).__name__}: {exc}" partial.unlink(missing_ok=True) - log.warning("input fetch attempt %d/%d from %s failed: %s", attempt, attempts, url, last) + log.warning("input fetch attempt %d/%d from %s failed: %s", attempt, attempts, safe_url(url), last) if attempt < attempts: time.sleep(backoff * (2 ** (attempt - 1))) - raise DeliveryError(f"{last} after {attempts} attempts to {url}") + raise DeliveryError(f"{last} after {attempts} attempts to {safe_url(url)}") -def fetch_inputs(environ: Mapping[str, str]) -> list[Path]: - """Fetch every ``OUSAST_INPUTS`` entry from the receiver to its ``OUSAST_INPUT_`` path. +def fetch_inputs(environ: Mapping[str, str], delivery: DeliveryTargets | None = None) -> list[Path]: + """Fetch every ``OUSAST_INPUTS`` entry to its ``OUSAST_INPUT_`` path: from ``delivery.inputs[NAME]`` (a + presigned GET, no headers) when the start request carries one, else from the receiver. :class:`RunnerError` names the input when one cannot be fetched: a task never runs with an input missing.""" raw = environ.get("OUSAST_INPUTS") @@ -736,11 +802,15 @@ def fetch_inputs(environ: Mapping[str, str]) -> list[Path]: fetched: list[Path] = [] for name, ref in sorted(inputs.items()): dest = environ.get(f"OUSAST_INPUT_{name}") - if not url or not dest or not isinstance(ref, str): - raise RunnerError(f"input {name} ({ref}) not fetched: OUSAST_ARTIFACT_URL or OUSAST_INPUT_{name} is unset") - source = f"{url.rstrip('/')}/inputs/{urllib.parse.quote(ref)}" + presigned = delivery.inputs.get(name) if delivery is not None else None + if not dest or not isinstance(ref, str) or not (presigned or url): + raise RunnerError(f"input {name} ({ref}) not fetched: no delivery URL, OUSAST_ARTIFACT_URL or OUSAST_INPUT_{name}") try: - size = fetch(source, headers, Path(dest), backoff=backoff, dial=environ.get("OUSAST_ARTIFACT_DIAL") or None) + if presigned: + size = fetch(presigned, {}, Path(dest), backoff=backoff) + else: + source = f"{str(url).rstrip('/')}/inputs/{urllib.parse.quote(ref)}" + size = fetch(source, headers, Path(dest), backoff=backoff, dial=environ.get("OUSAST_ARTIFACT_DIAL") or None) except DeliveryError as exc: raise RunnerError(f"input {name} ({ref}) not fetched: {exc}") from None log.info("input %s (%s): %d bytes to %s", name, ref, size, dest) @@ -748,8 +818,13 @@ def fetch_inputs(environ: Mapping[str, str]) -> list[Path]: return fetched -def _deliver_output(task: Task, output_dir: Path, environ: Mapping[str, str]) -> bool: - """POST the output directory; False when there is nowhere to deliver; :class:`DeliveryError` when it fails.""" +def _deliver_output(task: Task, output_dir: Path, environ: Mapping[str, str], delivery: DeliveryTargets | None = None) -> bool: + """PUT the output tar to ``delivery.put`` when the start request carried one, else POST it to the receiver; + False when there is nowhere to deliver; :class:`DeliveryError` when it fails.""" + backoff = float(environ.get("OUSAST_DELIVERY_BACKOFF", "1.0")) + if delivery is not None and delivery.put: + deliver(delivery.put, build_tar(output_dir), {"Content-Type": "application/x-tar"}, backoff=backoff, method="PUT") + return True url = environ.get("OUSAST_ARTIFACT_URL") if not url: return False @@ -758,7 +833,6 @@ def _deliver_output(task: Task, output_dir: Path, environ: Mapping[str, str]) -> run = environ.get("OUSAST_RUN") if run: headers["X-Ousast-Run"] = run - backoff = float(environ.get("OUSAST_DELIVERY_BACKOFF", "1.0")) deliver(url, build_tar(output_dir), headers, backoff=backoff, dial=environ.get("OUSAST_ARTIFACT_DIAL") or None) return True @@ -810,14 +884,16 @@ def _exit_of(marker: Mapping[str, Any]) -> int: return EXIT_BY_STATUS.get(str(marker.get("status")), EXIT_BY_STATUS["failed"]) -def _finish(session: _Session, env: Mapping[str, str], exit_code: int | None, summary: Mapping[str, Any]) -> dict[str, Any]: +def _finish( + session: _Session, env: Mapping[str, str], exit_code: int | None, summary: Mapping[str, Any], delivery: DeliveryTargets | None = None +) -> dict[str, Any]: """Deliver the output directory and write the completion marker; a failed delivery reports ``failed``. The marker keeps the task's own summary so a later boot can put it back and retry the delivery alone. """ marker: dict[str, Any] = {"exit": exit_code, "status": str(summary.get("status")), "delivered": False, "summary": dict(summary)} try: - marker["delivered"] = _deliver_output(session.task, session.output_dir, env) + marker["delivered"] = _deliver_output(session.task, session.output_dir, env, delivery) except DeliveryError as exc: log.error("artifact delivery failed: %s", exc) marker.update(status="failed", reason=f"artifact delivery failed: {exc}") @@ -857,24 +933,24 @@ def _run_once(env: MutableMapping[str, str], ready: threading.Event, stop: threa stored = done.get("summary") kept: dict[str, Any] = stored if isinstance(stored, dict) else read_summary(session.output_dir) or {} write_summary(session.output_dir, kept) - return _exit_of(_finish(session, env, done.get("exit"), kept)) + return _exit_of(_finish(session, env, done.get("exit"), kept, start.delivery)) # a re-minted URL, if any if stop.is_set(): return 0 try: _materialise_once(session, env) - fetch_inputs(env) # after the start as well: the receiver serves a task its inputs only while it runs + fetch_inputs(env, start.delivery) # after the start: the URLs arrive with it (and a receiver serves a running task only) except RunnerError as exc: log.error("runner failed: %s", exc) failed = _failed(f"runner failed: {exc}", read_summary(session.output_dir)) write_summary(session.output_dir, failed) - return _exit_of(_finish(session, env, None, failed)) + return _exit_of(_finish(session, env, None, failed, start.delivery)) log.info("workspaces and inputs ready; running task %s: %s", session.task.metadata.name, list(session.task.command)) grace = float(env.get("OUSAST_TERM_GRACE") or TERM_GRACE) code, summary = run_command(session.task, session.output_dir, env, stop, grace, start.credentials) if code is None: log.info("task %s was stopped before its command finished; a resume runs it again", session.task.metadata.name) return 0 - return _exit_of(_finish(session, env, code, summary)) + return _exit_of(_finish(session, env, code, summary, start.delivery)) except RunnerError as exc: # The failure is reported, never acted on unasked: a start request is answered with it (409 without a # session, else 202 and the failed summary is delivered), so the golden boot delivers nothing either. @@ -887,9 +963,10 @@ def _run_once(env: MutableMapping[str, str], ready: threading.Event, stop: threa write_summary(session.output_dir, summary) gate.arm(env.get("OUSAST_RUN", ""), session.task.metadata.name) ready.set() - if env.get("AX_RUNNER_AUTOSTART") != "1" and gate.wait(stop) is None: + late = gate.autostart() if env.get("AX_RUNNER_AUTOSTART") == "1" else gate.wait(stop) + if late is None: return 0 - return _exit_of(_finish(session, env, None, summary)) + return _exit_of(_finish(session, env, None, summary, late.delivery)) def _materialise_once(session: _Session, env: Mapping[str, str]) -> None: @@ -918,14 +995,14 @@ def main(argv: Sequence[str] | None = None, environ: MutableMapping[str, str] | env = os.environ if environ is None else environ if argv: log.warning("ax-task-runner ignores command-line arguments %r; the Task manifest is the only input", list(argv)) - ready, stop, gate = threading.Event(), threading.Event(), StartGate() + ready, stop, gate = threading.Event(), threading.Event(), StartGate(insecure=env.get("AX_RUNNER_INSECURE_DELIVERY") == "1") previous = _on_sigterm(stop) server: HealthServer | None = None if env.get("AX_RUNNER_HTTP", "1") != "0": metadata = {"task": env.get("AX_TASK_YAML", ""), "workspaces": env.get("AX_WORKSPACES_YAML", "")} server = start_health_server(int(env.get("AX_RUNNER_PORT", str(DEFAULT_PORT))), ready, metadata, gate) env["AX_RUNNER_BOUND_PORT"] = str(server.port) - env["AX_METADATA_URL"] = f"http://127.0.0.1:{server.port}" + env["AX_METADATA_URL"] = f"http://127.0.0.1:{server.port}" # loopback: this runner's own health server log.info("health and metadata server on port %d", server.port) try: if server is None and env.get("AX_RUNNER_AUTOSTART") != "1": diff --git a/tests/conftest.py b/tests/conftest.py index 32e4b46..c5a43f5 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -65,11 +65,12 @@ from pathlib import Path HERE = Path(__file__).resolve().parent argv = sys.argv[1:] +context = "" if argv[:1] == ["--context"]: - argv = argv[2:] + context, argv = argv[1], argv[2:] verb, actor, space = argv[0], argv[2], argv[argv.index("-a") + 1] with (HERE / "ax.log").open("a") as log: - log.write(f"{time.monotonic():.4f} egress {verb} {actor}\n") + log.write(f"{time.monotonic():.4f} egress {verb} {actor} {context}\n") record = HERE / "applied" / f"{actor}.json" alive = record.exists() and not json.loads(record.read_text()).get("gone") store = HERE / "policies" / f"{space}_{actor}.json" @@ -103,3 +104,26 @@ def kubectl_ate(tmp_path, monkeypatch: pytest.MonkeyPatch): monkeypatch.setenv("OUSAST_KUBECTL_ATE", str(path)) monkeypatch.setenv("OUSAST_EGRESS_SETTLE", "0") return path + + +PROFILE_CONTEXT = "kind-test" # the fakes must receive this, never a context a module built (plane-on-kubernetes 1.2) +PROFILE_TOML = ( + 'exec = "local"\nkube_context = "{context}"\nregistry = "registry.test:5000"\nimages = "images.json"\n' + 'memory = "file://{memory}"\natespace = "default"\n[pools.default]\nreplicas = 1\ncpu = "1"\nmemory = "1Gi"\n' +) + + +@pytest.fixture +def plane_profile(tmp_path, monkeypatch: pytest.MonkeyPatch): + """A throwaway PlaneProfile under ``tmp_path`` (context ``kind-test``, a file:// memory store, a digest-pinned + images file), selected through ``OUSAST_PLANE_PROFILE`` so every plane module loads it.""" + import json + + root = tmp_path / "profile" + root.mkdir() + (root / "test.toml").write_text(PROFILE_TOML.format(context=PROFILE_CONTEXT, memory=tmp_path / "memory"), encoding="utf-8") + (root / "images.json").write_text(json.dumps({"runner": "registry.test:5000/ousast-runner@sha256:" + "ab" * 32}), encoding="utf-8") + monkeypatch.setenv("OUSAST_PLANE_PROFILE", str(root / "test.toml")) + for key in ("OUSAST_KUBE_CONTEXT", "OUSAST_MEMORY", "OUSAST_REGISTRY", "OUSAST_IMAGES", "OUSAST_EXEC"): + monkeypatch.delenv(key, raising=False) + return root / "test.toml" diff --git a/tests/plane_fake_store.py b/tests/plane_fake_store.py new file mode 100644 index 0000000..5f81c53 --- /dev/null +++ b/tests/plane_fake_store.py @@ -0,0 +1,109 @@ +"""An S3-shaped memory store for the delivery tests: objects in a dict, presigned URLs served by a local HTTP server +that accepts PUT, GET and HEAD by key (``//?X-Amz-Expires=N&X-Amz-Signature=fake``), the way a +presigned S3 URL looks to the runner. Nothing here is imported by the product.""" + +from __future__ import annotations + +import threading +from collections.abc import Mapping +from datetime import timedelta +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import unquote, urlsplit + +from openultrasast.plane.memory import MemoryStore + + +class HttpObjectStore(MemoryStore): + """``presign_*`` hand out URLs on a local server; ``requests`` records every HTTP call (method, key).""" + + def __init__(self, bucket: str = "fake-bucket", host: str = "localhost") -> None: + self.objects: dict[str, bytes] = {} + self.bucket, self.host = bucket, host + self.requests: list[tuple[str, str]] = [] + self.presigned: list[tuple[str, str, int]] = [] + self.lock = threading.Lock() + self.server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler) + self.server.store = self # type: ignore[attr-defined] + threading.Thread(target=self.server.serve_forever, daemon=True).start() + + def close(self) -> None: + self.server.shutdown() + self.server.server_close() + + def describe(self) -> str: + return f"fake://{self.bucket}" + + def _get(self, key: str) -> tuple[bytes, str | None] | None: + with self.lock: + data = self.objects.get(key) + return None if data is None else (data, "v1") + + def _put(self, key: str, data: bytes, labels: Mapping[str, str] | None = None) -> None: + with self.lock: + self.objects[key] = bytes(data) + + def _delete(self, key: str) -> None: + with self.lock: + self.objects.pop(key, None) + + def _keys(self, prefix: str) -> list[str]: + with self.lock: + return sorted(k for k in self.objects if k.startswith(prefix)) + + def _url(self, method: str, key: str, expires: timedelta) -> str: + self.presigned.append((method, key, int(expires.total_seconds()))) + port = self.server.server_address[1] + return f"http://{self.host}:{port}/{self.bucket}/{key}?X-Amz-Expires={int(expires.total_seconds())}&X-Amz-Signature=fake" + + def presign_put(self, key: str, expires: timedelta = timedelta(hours=1)) -> str: + return self._url("PUT", key, expires) + + def presign_get(self, key: str, expires: timedelta = timedelta(hours=1)) -> str: + return self._url("GET", key, expires) + + +class _Handler(BaseHTTPRequestHandler): + def _key(self) -> str | None: + store: HttpObjectStore = self.server.store # type: ignore[attr-defined] + path = unquote(urlsplit(self.path).path) + prefix = f"/{store.bucket}/" + return path[len(prefix) :] if path.startswith(prefix) else None + + def do_PUT(self) -> None: # noqa: N802 + store: HttpObjectStore = self.server.store # type: ignore[attr-defined] + key = self._key() + body = self.rfile.read(int(self.headers.get("Content-Length") or 0)) + store.requests.append(("PUT", key or self.path)) + if key is None: + return self._answer(404) + store._put(key, body) + self._answer(200) + + def do_GET(self) -> None: # noqa: N802 + store: HttpObjectStore = self.server.store # type: ignore[attr-defined] + key = self._key() + store.requests.append(("GET", key or self.path)) + got = store._get(key) if key else None + if got is None: + return self._answer(404) + self.send_response(200) + self.send_header("Content-Length", str(len(got[0]))) + self.end_headers() + self.wfile.write(got[0]) + + def do_HEAD(self) -> None: # noqa: N802 + store: HttpObjectStore = self.server.store # type: ignore[attr-defined] + key = self._key() + store.requests.append(("HEAD", key or self.path)) + got = store._get(key) if key else None + self.send_response(404 if got is None else 200) + self.send_header("Content-Length", "0" if got is None else str(len(got[0]))) + self.end_headers() + + def _answer(self, code: int) -> None: + self.send_response(code) + self.send_header("Content-Length", "0") + self.end_headers() + + def log_message(self, format: str, *args: object) -> None: # noqa: A002 + pass diff --git a/tests/test_plane_delivery.py b/tests/test_plane_delivery.py new file mode 100644 index 0000000..507c4ba --- /dev/null +++ b/tests/test_plane_delivery.py @@ -0,0 +1,164 @@ +"""plane-on-kubernetes task 2.1 (Req 3.1, 2.1): ``Delivery`` against an S3-shaped fake -- presigned URLs in the start +body, expiry, polling, extraction with the receiver's member checks, declared-only re-put, state seeding, and no URL +in any log line.""" + +from __future__ import annotations + +import io +import json +import logging +import tarfile +import urllib.request +from collections.abc import Iterator +from pathlib import Path + +import pytest +from plane_fake_store import HttpObjectStore + +from openultrasast.plane.delivery import GRACE_SECONDS, Delivery, DeliveryError, declared, expiry, input_var + + +@pytest.fixture +def store() -> Iterator[HttpObjectStore]: + fake = HttpObjectStore() + try: + yield fake + finally: + fake.close() + + +def tar_of(files: dict[str, bytes], *, symlink: str | None = None) -> bytes: + buffer = io.BytesIO() + with tarfile.open(fileobj=buffer, mode="w") as tar: + for name, data in files.items(): + info = tarfile.TarInfo(name) + info.size = len(data) + tar.addfile(info, io.BytesIO(data)) + if symlink: + info = tarfile.TarInfo(symlink) + info.type = tarfile.SYMTYPE + info.linkname = "/etc/passwd" + tar.addfile(info) + return buffer.getvalue() + + +def put_via_url(url: str, data: bytes) -> int: + request = urllib.request.Request(url, data=data, method="PUT", headers={"Content-Type": "application/x-tar"}) + with urllib.request.urlopen(request, timeout=10) as response: # noqa: S310 - the fake's local URL + return int(response.status) + + +def test_the_start_body_names_exactly_the_tar_and_the_declared_inputs(store: HttpObjectStore, tmp_path: Path, monkeypatch) -> None: + monkeypatch.setenv("OUSAST_TASK_TIMEOUT", "100") + delivery = Delivery(store, "run-1", tmp_path) + body = delivery.body("verify", {"facts": "repo-facts/facts.json", "pass a": "va/units.jsonl"}) + assert set(body) == {"put", "inputs"} + assert body["put"].startswith(f"http://localhost:{store.server.server_address[1]}/fake-bucket/runs/run-1/verify/output.tar?") + assert set(body["inputs"]) == {"FACTS", "PASS_A"} and input_var("pass a") == "PASS_A" + assert "/runs/run-1/repo-facts/facts.json?" in body["inputs"]["FACTS"] and "/runs/run-1/va/units.jsonl?" in body["inputs"]["PASS_A"] + assert {m for m, _, _ in store.presigned} == {"PUT", "GET"} + assert {e for _, _, e in store.presigned} == {100 + GRACE_SECONDS}, "expiry = OUSAST_TASK_TIMEOUT + 600" + assert expiry({"OUSAST_TASK_TIMEOUT": "7200"}).total_seconds() == 7800 and expiry({}).total_seconds() == 7800 + assert delivery.host == "localhost", "the hostname the egress policy must allow" + with pytest.raises(DeliveryError, match="not /"): + delivery.input_urls({"x": "no-slash"}) + + +def test_delivered_polls_the_store_and_collect_extracts_republishes_and_marks_done( + store: HttpObjectStore, tmp_path: Path, caplog: pytest.LogCaptureFixture +) -> None: + caplog.set_level(logging.DEBUG) + delivery = Delivery(store, "r", tmp_path / "r") + assert delivery.delivered("facts") is False + summary = {"status": "done", "units_done": 1, "units_total": 1} + tar = tar_of({"facts.json": b'{"a": 1}', "summary.json": json.dumps(summary).encode(), "scratch/notes.txt": b"not declared"}) + assert put_via_url(delivery.put_url("facts"), tar) == 200, "the runner's PUT to the presigned URL" + assert delivery.delivered("facts") is True + got, size = delivery.collect("facts", ["facts.json", "summary.json", "missing.json"]) + assert got == summary and size == len(tar) + assert (tmp_path / "r" / "facts" / "facts.json").read_bytes() == b'{"a": 1}' + assert (tmp_path / "r" / "facts" / "scratch" / "notes.txt").exists(), "extracted locally" + assert store.objects["runs/r/facts/facts.json"] == b'{"a": 1}' and "runs/r/facts/summary.json" in store.objects + assert "runs/r/facts/scratch/notes.txt" not in store.objects and "runs/r/facts/missing.json" not in store.objects, "declared only" + done = json.loads(store.objects["runs/r/facts/done.json"]) + assert ( + done["status"] == "done" + and done["bytes"] == len(tar) + and done["files"] == 3 + and done["published"] == ["facts.json", "summary.json"] + ) + # a consumer's presigned GET names one artifact and fetches it + urls = delivery.input_urls({"facts": "facts/facts.json"}) + with urllib.request.urlopen(urls["FACTS"], timeout=10) as response: # noqa: S310 + assert response.read() == b'{"a": 1}' + assert "X-Amz" not in caplog.text and "fake-bucket" not in caplog.text, "URLs never appear in logs" + delivery.reset("facts") # the next attempt must not complete on this attempt's tar + assert delivery.delivered("facts") is False and "runs/r/facts/done.json" not in store.objects + assert "runs/r/facts/facts.json" in store.objects, "a re-put output stays for its consumers until it is re-published" + + +@pytest.mark.parametrize("bad", ["../escape.json", "/abs.json"]) +def test_a_tar_member_that_escapes_is_rejected_and_nothing_is_written(store: HttpObjectStore, tmp_path: Path, bad: str) -> None: + delivery = Delivery(store, "r", tmp_path / "r") + store._put(delivery.key("t", "output.tar"), tar_of({bad: b"{}", "summary.json": b"{}"})) + with pytest.raises(DeliveryError, match="rejected member"): + delivery.collect("t", ["summary.json"]) + assert not (tmp_path / "r" / "t").exists() and not (tmp_path / "escape.json").exists() + assert "runs/r/t/done.json" not in store.objects and "runs/r/t/summary.json" not in store.objects + + +def test_a_symlink_member_and_a_non_tar_are_rejected(store: HttpObjectStore, tmp_path: Path) -> None: + delivery = Delivery(store, "r", tmp_path / "r") + store._put(delivery.key("t", "output.tar"), tar_of({"summary.json": b"{}"}, symlink="link")) + with pytest.raises(DeliveryError, match="rejected member link"): + delivery.collect("t", ["summary.json"]) + store._put(delivery.key("t", "output.tar"), b"not a tar at all") + with pytest.raises(DeliveryError, match="bad tar"): + delivery.collect("t", ["summary.json"]) + with pytest.raises(DeliveryError, match="not in the store"): + delivery.collect("never", []) + + +def test_state_is_mirrored_and_seeds_a_rerun_elsewhere(store: HttpObjectStore, tmp_path: Path) -> None: + first = Delivery(store, "r", tmp_path / "machine-a") + assert first.load_state() is None + first.save_state({"run": "r", "tasks": {"facts": {"status": "done"}}}) + second = Delivery(store, "r", tmp_path / "machine-b") # another results root, the same store + assert second.load_state() == {"run": "r", "tasks": {"facts": {"status": "done"}}} + store._put(second.key("state.json"), b"not json") + assert second.load_state() is None + + +def test_put_input_stores_a_file_no_task_produces(store: HttpObjectStore, tmp_path: Path) -> None: + delivery = Delivery(store, "r", tmp_path / "r") + source = tmp_path / "candidates.json" + source.write_bytes(b"[1, 2, 3]") + url = delivery.put_input("candidates.json", source) + assert store.objects["runs/r/inputs/candidates.json"] == b"[1, 2, 3]" + with urllib.request.urlopen(url, timeout=10) as response: # noqa: S310 + assert response.read() == b"[1, 2, 3]" + + +def test_declared_artifacts_are_outputs_plus_what_consumers_name(tmp_path: Path) -> None: + """A consumer may declare a producer's summary.json (every task writes one) without the producer listing it.""" + from openultrasast.plane.manifests import load_manifests + + text = ( + "apiVersion: ax.io/v1alpha1\nkind: Task\nmetadata: {name: t}\nspec: {command: [repo-facts]}\n---\n" + "apiVersion: openultrasast.io/v1alpha1\nkind: Run\nmetadata: {name: r}\nspec:\n tasks:\n" + " - {name: facts, task: t, outputs: [facts.json]}\n" + " - {name: verify, task: t, inputs: {facts: facts/facts.json, facts_summary: facts/summary.json}, outputs: [units.jsonl]}\n" + " - {name: agree, task: t, inputs: {pass: verify/units.jsonl, pass_summary: verify/summary.json}}\n" + ) + (tmp_path / "run.yaml").write_text(text, encoding="utf-8") + run = next(iter(load_manifests([tmp_path / "run.yaml"]).runs.values())) + assert declared(run) == {"facts": ["facts.json", "summary.json"], "verify": ["units.jsonl", "summary.json"], "agree": []} + + +def test_publish_reputs_only_declared_files_under_the_task_directory(store: HttpObjectStore, tmp_path: Path) -> None: + delivery = Delivery(store, "r", tmp_path / "r") + (tmp_path / "r" / "seeded").mkdir(parents=True) + (tmp_path / "r" / "seeded" / "facts.json").write_bytes(b"{}") + (tmp_path / "r" / "secret.txt").write_bytes(b"outside") + assert delivery.publish("seeded", ["facts.json", "../secret.txt", "absent.json"]) == ["facts.json"] + assert set(store.objects) == {"runs/r/seeded/facts.json"} diff --git a/tests/test_plane_egress.py b/tests/test_plane_egress.py index e9b5749..c369967 100644 --- a/tests/test_plane_egress.py +++ b/tests/test_plane_egress.py @@ -1,8 +1,10 @@ """ai-service-plane task 0: the per-task egress policy, its kubectl-ate calls, and the runner's dialled delivery. -Agent Substrate's egress gateway denies by default; :func:`egress.policy_for` must grant exactly the receiver (HTTP -by name), the bound Workspaces' Git hosts and the bound Model's API host (TLS passthrough), and nothing a gateway -would reject or that would intercept TLS: no IP address, no wildcard, no ``https`` rule. +Agent Substrate's egress gateway denies by default; :func:`egress.policy_for` must grant exactly the memory store's +host (where the runner PUTs its tar and GETs its inputs), the bound Workspaces' Git hosts and the bound Model's API +host, all as one TLS passthrough rule on 443 (plane-on-kubernetes 2.3), and nothing a gateway would reject or that +would intercept TLS: no IP address, no wildcard, no ``https`` rule, no ``http`` rule (the receiver path, kept for +group 2 only, is the one exception). """ from __future__ import annotations @@ -70,24 +72,59 @@ def assert_tight(policy: dict) -> None: ipaddress.ip_address(host) -def test_model_free_task_gets_only_the_receiver(manifests) -> None: - policy = egress.policy_for(manifests.tasks["facts"], [], None, egress.RECEIVER_HOST) - assert policy == {"rules": [{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}]} +STORE = "files.store.example" + + +def test_model_free_task_gets_only_the_store(manifests) -> None: + policy = egress.policy_for(manifests.tasks["facts"], [], None, STORE) + assert policy == {"rules": [{"tlsPassthrough": {"hostnames": [STORE], "ports": {"numbers": [443]}}}]} assert_tight(policy) + assert egress.policy_for(manifests.tasks["facts"], [], None, None) == {"rules": []}, "no store, no Git, no Model: nothing" + + +def test_the_receiver_path_adds_the_one_http_rule_until_it_is_removed(manifests) -> None: + policy = egress.policy_for(manifests.tasks["facts"], [], None, None, receiver_host=egress.RECEIVER_HOST) + assert policy == {"rules": [{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}]} + assert egress.policy_for(manifests.tasks["facts"], [], None, None, receiver_host="127.0.0.1") == {"rules": []} def test_model_bound_task_with_git_workspaces(manifests) -> None: task = manifests.tasks["verify"] - policy = egress.policy_for(task, [manifests.workspaces["cases"]], manifests.models["deepseek-flash"], egress.RECEIVER_HOST) + policy = egress.policy_for(task, [manifests.workspaces["cases"]], manifests.models["deepseek-flash"], STORE) assert policy["rules"] == [ - {"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}, - {"tlsPassthrough": {"hostnames": ["api.deepseek.com", "github.com"], "ports": {"numbers": [443]}}}, - ], "one github.com for two repos; the ssh and IP-address repos add nothing" + {"tlsPassthrough": {"hostnames": ["api.deepseek.com", STORE, "github.com"], "ports": {"numbers": [443]}}}, + ], "one rule: the store, one github.com for two repos, the Model host; the ssh and IP-address repos add nothing" assert_tight(policy) -def test_an_ip_receiver_is_not_granted(manifests) -> None: - assert egress.policy_for(manifests.tasks["facts"], [], None, "127.0.0.1") == {"rules": []} +def test_an_ip_store_endpoint_is_refused_naming_the_variable(manifests) -> None: + with pytest.raises(ValueError, match="S3_ENDPOINT must be a hostname"): + egress.policy_for(manifests.tasks["facts"], [], None, "127.0.0.1") + + +def test_an_http_git_repo_is_refused(tmp_path: Path) -> None: + path = tmp_path / "m.yaml" + path.write_text(MANIFESTS.replace("https://github.com/example-org/alpha", "http://mirror.example/alpha"), encoding="utf-8") + loaded = load_manifests([path]) + with pytest.raises(ValueError, match="http Git repos .*mirror.example.* cannot be reached"): + egress.policy_for(loaded.tasks["verify"], [loaded.workspaces["cases"]], None, STORE) + + +def test_every_committed_template_renders_the_expected_policy() -> None: + """For each template under plane/tasks/: the store, the Model's hosts when bound, nothing else; one TLS rule.""" + from openultrasast.plane.generate import TEMPLATES + + plane = Path("plane") + loaded = load_manifests([plane / "tasks" / f"{name}.yaml" for name in TEMPLATES] + sorted((plane / "models").glob("*.yaml"))) + assert set(loaded.tasks) == set(TEMPLATES) + for name, task in loaded.tasks.items(): + model = loaded.models.get(task.metadata.annotations.get("openultrasast.io/model") or "") + policy = egress.policy_for(task, [], model, STORE) + expected = sorted({STORE, *(model.egress_hosts if model else ())}) + assert policy == {"rules": [{"tlsPassthrough": {"hostnames": expected, "ports": {"numbers": [443]}}}]}, name + assert_tight(policy) + bound = {n for n, t in loaded.tasks.items() if "openultrasast.io/model" in t.metadata.annotations} + assert bound == {"verify", "roles"}, "the model-bound templates, and only they, get the provider host" def test_deepseek_model_manifest_names_the_endpoint_host() -> None: @@ -121,7 +158,7 @@ def calls(tmp_path: Path) -> list[str]: return [" ".join(line.split()[2:4]) for line in (tmp_path / "ax.log").read_text().splitlines()] -POLICY = {"rules": [{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}]} +POLICY = {"rules": [{"tlsPassthrough": {"hostnames": ["files.store.example"], "ports": {"numbers": [443]}}}]} def test_apply_creates_reads_back_and_delete_confirms_the_cascade(tmp_path: Path, kubectl_ate: Path) -> None: @@ -134,10 +171,11 @@ def test_apply_creates_reads_back_and_delete_confirms_the_cascade(tmp_path: Path actor(tmp_path, "t1", gone=True) # ax delete task: the actor, and with it the policy, is gone assert cli.delete("t1", "default") is None assert calls(tmp_path) == ["get t1", "create t1", "get t1", "get t1", "get t1", "get t1"] + assert {line.split()[-1] for line in (tmp_path / "ax.log").read_text().splitlines()} == {"kind-test"}, "the given context" def test_apply_replaces_an_existing_policy(tmp_path: Path, kubectl_ate: Path) -> None: - cli = egress.Egress(str(kubectl_ate)) + cli = egress.Egress(str(kubectl_ate), context="kind-test") actor(tmp_path, "t2") cli.apply("t2", "default", {"rules": []}) cli.apply("t2", "default", POLICY) @@ -146,7 +184,7 @@ def test_apply_replaces_an_existing_policy(tmp_path: Path, kubectl_ate: Path) -> def test_apply_waits_for_the_actor_then_gives_up(tmp_path: Path, kubectl_ate: Path) -> None: - cli = egress.Egress(str(kubectl_ate)) + cli = egress.Egress(str(kubectl_ate), context="kind-test") with pytest.raises(RuntimeError, match="not found"): cli.apply("missing", "default", POLICY, timeout=0.2) timer = threading.Timer(0.3, actor, args=(tmp_path, "late")) @@ -157,10 +195,13 @@ def test_apply_waits_for_the_actor_then_gives_up(tmp_path: Path, kubectl_ate: Pa def test_receiver_address_modes(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("OUSAST_ARTIFACT_HOST", "127.0.0.1") - assert egress.receiver_address(4242) == ("http://127.0.0.1:4242/", None) + assert egress.receiver_address(4242, None) == ("http://127.0.0.1:4242/", None) monkeypatch.delenv("OUSAST_ARTIFACT_HOST") monkeypatch.setenv("OUSAST_ARTIFACT_DIAL", "172.19.0.1:80") - assert egress.receiver_address(18090) == (f"http://{egress.RECEIVER_HOST}/", "172.19.0.1:80") + assert egress.receiver_address(18090, "kind-test") == (f"http://{egress.RECEIVER_HOST}/", "172.19.0.1:80") + monkeypatch.delenv("OUSAST_ARTIFACT_DIAL") + with pytest.raises(RuntimeError, match="kube context"): + egress.receiver_address(18090, None) # no context to look the Service up in: never a built one # --- the runner dials the gateway address with the receiver's name as Host ---------------------------------------- diff --git a/tests/test_plane_increment.py b/tests/test_plane_increment.py index b667e18..46b39b1 100644 --- a/tests/test_plane_increment.py +++ b/tests/test_plane_increment.py @@ -290,3 +290,42 @@ def test_case_record_carries_sites_in_set_and_cost_and_the_image_is_repinned(tmp (tmp_path / "runner-image").write_text("localhost:5001/ousast-runner:dev\n") with pytest.raises(ValueError, match="digest-pinned"): repin_templates(paths["plane"], tmp_path / "runner-image") + + +def test_repin_reads_the_profiles_images_file(tmp_path: Path) -> None: + """plane-on-kubernetes 1.4: ``ops/ax/up.sh`` writes ``kind-images.json`` ({"runner": ...}); repin reads it, the + older one-line pin still reads, and a file without a runner key or without a digest is refused by name.""" + paths = _fixture(tmp_path) + digest = "localhost:5001/ousast-runner@sha256:" + "cd" * 32 + images = tmp_path / "kind-images.json" + images.write_text(json.dumps({"runner": digest}) + "\n") + assert repin_templates(paths["plane"], images) == digest + assert {t.image for t in load_manifests(sorted((paths["plane"] / "tasks").glob("*.yaml"))).tasks.values()} == {digest} + images.write_text(json.dumps({"engine": digest})) + with pytest.raises(ValueError, match="no runner image"): + repin_templates(paths["plane"], images) + images.write_text(json.dumps({"runner": "localhost:5001/ousast-runner:dev"})) + with pytest.raises(ValueError, match="not digest-pinned"): + repin_templates(paths["plane"], images) + + +def test_workspaces_go_to_the_profiles_atespace(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """plane-on-kubernetes 1.2: ``generate.ATESPACE`` is a default; ``increment`` takes the profile's atespace.""" + paths = _fixture(tmp_path) + (tmp_path / "p.toml").write_text('exec = "local"\nkube_context = "c"\nregistry = "r"\nimages = "i.json"\natespace = "team-a"\n') + monkeypatch.setenv("OUSAST_PLANE_PROFILE", str(tmp_path / "p.toml")) + increment(paths["population"], paths["set"], paths["candidates"], paths["scans"], plane=paths["plane"], run_name="sp", command="t") + manifests = load_manifests(sorted((paths["plane"] / "workspaces").glob("*.yaml"))) + assert manifests.workspaces and {w.metadata.atespace for w in manifests.workspaces.values()} == {"team-a"} + increment( + paths["population"], + paths["set"], + paths["candidates"], + paths["scans"], + plane=paths["plane"], + run_name="sp", + command="t", + atespace="z", + ) + manifests = load_manifests(sorted((paths["plane"] / "workspaces").glob("*.yaml"))) + assert {w.metadata.atespace for w in manifests.workspaces.values()} == {"z"}, "an explicit atespace wins over the profile" diff --git a/tests/test_plane_literals.py b/tests/test_plane_literals.py new file mode 100644 index 0000000..743bb07 --- /dev/null +++ b/tests/test_plane_literals.py @@ -0,0 +1,112 @@ +"""plane-on-kubernetes Req 1.3: no module, manifest or page assumes this host. + +The scan covers ``src/``, ``plane/``, ``ops/`` (minus ``ops/ax/up.sh``, which creates the kind cluster and may name +it, and ``ops/k8s/profiles/``, which *are* the configuration) and ``docs/`` for the kind context, the kind registry, +the kind network's gateway, a loopback address with a port, and ``port-forward``. Allowed: ``port-forward`` and the +loopback address inside ``router.py`` (the one module that opens the tunnel), and any line tagged ``ax-tunnel`` +(the tunnel is how both profiles reach ax and the router) or ``loopback`` (a process talking to itself). A +digest-pinned ``image:`` line of a committed Task template under ``plane/tasks/`` is the kind profile's own value, +written there by ``ousast plane`` repinning from ``ops/k8s/profiles/kind-images.json``; it is accepted only while it +names that profile's registry, so the templates and the profile cannot drift apart. The patterns are built at runtime +so this file does not match them (as ``test_removed_plane_references.py`` does). +""" + +from __future__ import annotations + +import re +from pathlib import Path + +from openultrasast.plane.profile import load_profile + +ROOT = Path(__file__).resolve().parents[1] +SCANNED = ("src", "plane", "ops", "docs") +EXEMPT = (Path("ops/ax/up.sh"), Path("ops/k8s/profiles")) +ROUTER = Path("src/openultrasast/plane/router.py") +PATTERNS = { + "kind context": re.compile("kind-" + "ousast"), + "kind registry": re.compile("localhost:" + "5001"), + "kind gateway": re.compile(r"172\.19\."), + "loopback with port": re.compile(r"127\.0\.0\.1" + ":"), + "port-forward": re.compile("port-" + "forward"), +} +ROUTER_ONLY = {"loopback with port", "port-forward"} +TAGS = re.compile(r"ax-tunnel|loopback") +_IMAGE_LINE = re.compile(r'^\s*image:\s*"?(?P\S+?)@sha256:[0-9a-f]{64}"?\s*(#.*)?$') + + +def _files() -> list[Path]: + out = [] + for top in SCANNED: + for path in sorted((ROOT / top).rglob("*")): + relative = path.relative_to(ROOT) + if not path.is_file() or "__pycache__" in relative.parts or path.suffix in (".png", ".jpg", ".gz", ".zip"): + continue + if any(relative == e or e in relative.parents for e in EXEMPT): + continue + out.append(path) + return out + + +def _pinned_template_image(relative: Path, line: str, registry: str) -> bool: + """A committed template's digest-pinned image in the kind profile's registry: configuration, not a literal.""" + if relative.parts[:2] != ("plane", "tasks"): + return False + found = _IMAGE_LINE.match(line) + return bool(found) and found.group("ref").startswith(registry + "/") + + +def offenders(paths: list[Path], registry: str, root: Path = ROOT) -> list[str]: + found = [] + for path in paths: + relative = path.relative_to(root) + try: + text = path.read_text(encoding="utf-8") + except (UnicodeDecodeError, OSError): + continue + for number, line in enumerate(text.splitlines(), 1): + for label, pattern in PATTERNS.items(): + if not pattern.search(line): + continue + if label in ROUTER_ONLY and relative == ROUTER: + continue + if TAGS.search(line) or _pinned_template_image(relative, line, registry): + continue + found.append(f"{relative}:{number}: [{label}] {line.strip()[:120]}") + return found + + +def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None: + kind = load_profile("kind", environ={}) + found = offenders(_files(), kind.registry) + assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found) + + +def test_the_scan_sees_a_reinserted_literal(tmp_path: Path) -> None: + """The instrument is not vacuous: each pattern is caught, a tag or the router exempts only what it says.""" + kind = load_profile("kind", environ={}) + src = tmp_path / "src" + src.mkdir() + (src / "a.py").write_text(f'ctx = "{"kind-" + "ousast"}"\nreg = "{"localhost:" + "5001"}"\nip = "172.19.0.1"\n', encoding="utf-8") + (src / "b.py").write_text('url = "http://127.0.0.1:8080" # loopback, this process\nargv = ["port-' + 'forward"] # ax-tunnel\n') + (src / "c.py").write_text('argv = ["port-' + 'forward"]\n', encoding="utf-8") + paths = sorted(src.glob("*.py")) + found = [o.split(": ", 1)[1] for o in offenders(paths, kind.registry, root=tmp_path)] + assert found == [ + "[kind context] ctx = " + '"kind-' + 'ousast"', + "[kind registry] reg = " + '"localhost:' + '5001"', + '[kind gateway] ip = "172.19.0.1"', + "[port-forward] argv = " + '["port-' + 'forward"]', + ] + + +def test_the_committed_templates_name_the_kind_profiles_registry() -> None: + """The one exemption is checked, not assumed: every template image is in the kind registry, by digest.""" + kind = load_profile("kind", environ={}) + images = [] + for path in sorted((ROOT / "plane/tasks").glob("*.yaml")): + for line in path.read_text(encoding="utf-8").splitlines(): + found = _IMAGE_LINE.match(line) + if found: + images.append(found.group("ref")) + assert found.group("ref").startswith(kind.registry + "/"), f"{path.name}: {line.strip()}" + assert images, "the templates carry digest-pinned images" diff --git a/tests/test_plane_profile.py b/tests/test_plane_profile.py new file mode 100644 index 0000000..67168ba --- /dev/null +++ b/tests/test_plane_profile.py @@ -0,0 +1,158 @@ +"""plane-on-kubernetes task 1.1: the ``PlaneProfile`` (Req 1.1). Defaults, file-then-environment precedence, every +rejection naming its field, both committed profiles loading, and no committed value that looks like a credential.""" + +from __future__ import annotations + +import json +import re +from pathlib import Path + +import pytest + +from openultrasast.plane import profile as profile_module +from openultrasast.plane.profile import PlaneProfile, Pool, ProfileError, load_images, load_profile, profiles_dir + +ROOT = Path(__file__).resolve().parents[1] +DIGEST = "sha256:" + "ab" * 32 +MINIMAL = 'exec = "local"\nkube_context = "kind-test"\nregistry = "registry.test:5000"\nimages = "images.json"\nmemory = "s3://m"\n' + + +def write_profile(tmp_path: Path, text: str = MINIMAL, name: str = "test") -> Path: + path = tmp_path / f"{name}.toml" + path.write_text(text, encoding="utf-8") + (tmp_path / "images.json").write_text(json.dumps({"runner": f"registry.test:5000/ousast-runner@{DIGEST}"}), encoding="utf-8") + return path + + +def test_defaults_and_the_images_path_resolve_against_the_profile_file(tmp_path: Path) -> None: + loaded = load_profile(str(write_profile(tmp_path)), environ={}) + assert loaded.name == "test" and loaded.exec == "local" and loaded.kube_context == "kind-test" + assert loaded.images == tmp_path / "images.json" and loaded.load_images() == {"runner": f"registry.test:5000/ousast-runner@{DIGEST}"} + assert loaded.atespace == "default" and loaded.router_url == "" and loaded.pools == {} + assert "(tunnel through the kube context)" in loaded.addresses()["router_url"] + + +def test_the_environment_overrides_the_file_and_names_the_profile(tmp_path: Path) -> None: + path = write_profile(tmp_path, MINIMAL + '[pools.default]\nreplicas = 2\ncpu = "1"\nmemory = "1536Mi"\n') + env = { + "OUSAST_PLANE_PROFILE": str(path), + "OUSAST_KUBE_CONTEXT": "other-cluster", + "OUSAST_MEMORY": "s3://elsewhere/prefix", + "OUSAST_ROUTER_URL": "https://router.test", + "OUSAST_POOL_DEFAULT_REPLICAS": "5", + "OUSAST_POOL_DEFAULT_MEMORY": "2Gi", + "OUSAST_EXEC": "", # an empty variable is not an override + } + loaded = load_profile(environ=env) + assert (loaded.kube_context, loaded.memory, loaded.router_url, loaded.exec) == ( + "other-cluster", + "s3://elsewhere/prefix", + "https://router.test", + "local", + ) + assert loaded.pools == {"default": Pool(replicas=5, cpu="1", memory="2Gi")} + assert load_profile(str(path), environ={}).kube_context == "kind-test", "the file is unchanged" + + +@pytest.mark.parametrize( + ("text", "field"), + [ + (MINIMAL + 'colour = "blue"\n', "unknown keys: colour"), + (MINIMAL + 'aws_secret_key = "x"\n', "aws_secret_key looks like a credential"), + (MINIMAL + 'api_token = "x"\n', "api_token looks like a credential"), + (MINIMAL.replace('exec = "local"', 'exec = "cluster"'), "exec must be one of local, remote"), + (MINIMAL + '[pools.default]\nreplicas = "two"\ncpu = "1"\nmemory = "1Gi"\n', "pools.default.replicas must be an integer"), + (MINIMAL + '[pools.default]\nreplicas = 1\ncpu = "1"\n', "pools.default lacks memory"), + (MINIMAL + '[pools.default]\nreplicas = 1\ncpu = "1"\nmemory = "1Gi"\nnodes = 3\n', "pools.default has unknown keys: nodes"), + (MINIMAL.replace('kube_context = "kind-test"', "kube_context = 3"), "kube_context must be a string"), + ], +) +def test_each_rejection_names_the_field(tmp_path: Path, text: str, field: str) -> None: + path = write_profile(tmp_path, text) + with pytest.raises(ProfileError, match=re.escape(field)) as exc: + load_profile(str(path), environ={}) + assert str(path) in str(exc.value) + + +def test_a_credential_shaped_value_is_refused(tmp_path: Path) -> None: + path = write_profile(tmp_path, MINIMAL.replace("s3://m", "sk-" + "a1b2c3d4" * 3)) + with pytest.raises(ProfileError, match="value of memory looks like a credential"): + load_profile(str(path), environ={}) + + +def test_an_images_file_without_digest_pins_is_refused(tmp_path: Path) -> None: + path = write_profile(tmp_path) + (tmp_path / "images.json").write_text(json.dumps({"runner": "registry.test:5000/ousast-runner:dev"}), encoding="utf-8") + with pytest.raises(ProfileError, match="runner is not digest-pinned"): + load_profile(str(path), environ={}) + (tmp_path / "images.json").write_text(f"registry.test:5000/ousast-runner@{DIGEST}\n", encoding="utf-8") + assert load_images(tmp_path / "images.json") == {"runner": f"registry.test:5000/ousast-runner@{DIGEST}"}, "the one-line form" + (tmp_path / "images.json").unlink() + loaded = load_profile(str(path), environ={}) # a file the release has not published yet: doctor's finding, not a load error + with pytest.raises(ProfileError, match="images file"): + loaded.load_images() + + +def test_an_environment_override_of_the_wrong_shape_is_refused(tmp_path: Path) -> None: + path = write_profile(tmp_path, MINIMAL + '[pools.default]\nreplicas = 1\ncpu = "1"\nmemory = "1Gi"\n') + with pytest.raises(ProfileError, match="OUSAST_POOL_DEFAULT_REPLICAS must be an integer"): + load_profile(str(path), environ={"OUSAST_POOL_DEFAULT_REPLICAS": "many"}) + with pytest.raises(ProfileError, match="names a pool the profile does not declare"): + load_profile(str(path), environ={"OUSAST_POOL_ENGINE_REPLICAS": "1"}) + with pytest.raises(ProfileError, match="exec must be one of"): + load_profile(str(path), environ={"OUSAST_EXEC": "both"}) + + +def test_a_missing_profile_names_its_path(tmp_path: Path) -> None: + with pytest.raises(ProfileError, match="no-such.toml"): + load_profile(str(tmp_path / "no-such.toml"), environ={}) + with pytest.raises(ProfileError, match="unknown"): + load_profile("unknown", environ={"OUSAST_PLANE_PROFILES": str(tmp_path)}) + + +@pytest.mark.parametrize("name", ["kind", "k3s"]) +def test_both_committed_profiles_load_with_the_designs_values(name: str) -> None: + loaded = load_profile(name, environ={}) + assert loaded.path == profiles_dir() / f"{name}.toml" and profiles_dir() == ROOT / "ops/k8s/profiles" + assert loaded.exec == {"kind": "local", "k3s": "remote"}[name] + assert loaded.memory == "s3://sast-memory" and loaded.atespace == "default" and loaded.snapshots_bucket == "s3://ax-snapshots" + assert set(loaded.pools) == {"default", "engine"} and loaded.pools["engine"].memory == "4Gi" + assert loaded.router_url == "" and loaded.ax_server == "", "both profiles tunnel through the Kubernetes API" + if name == "kind": + assert loaded.image_pull_secret == "" and loaded.images.name == "kind-images.json" and loaded.images.is_file() + assert set(loaded.load_images()) == {"runner"} and loaded.load_images()["runner"].startswith(loaded.registry + "/") + else: + assert loaded.image_pull_secret == "ghcr-pull" and loaded.registry.startswith("ghcr.io/") and loaded.images.name == "images.json" + assert not loaded.images.is_file(), "the release asset is downloaded, never committed (design section 8)" + + +def test_no_committed_profile_value_matches_a_credential_pattern() -> None: + patterns = ( + re.compile(r"(?i)(api_key|secret_key|access_key|token|password)\s*="), # image_pull_secret names a Secret, holds none + re.compile(r"sk-[A-Za-z0-9]{8,}"), + re.compile(r"AKIA[0-9A-Z]{12,}"), + ) + for path in sorted(profiles_dir().glob("*.toml")): + for number, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1): + code = line.split("#", 1)[0] + for pattern in patterns: + assert not pattern.search(code), f"{path}:{number}: {line.strip()}" + + +def test_print_for_shell_scripts(capsys: pytest.CaptureFixture[str], tmp_path: Path) -> None: + path = write_profile(tmp_path, MINIMAL + '[pools.default]\nreplicas = 3\ncpu = "1"\nmemory = "1Gi"\n') + fields = ["registry", "kube_context", "images.runner", "pools.default.replicas"] + assert profile_module.main(["--profile", str(path), "--print", *fields]) == 0 + assert capsys.readouterr().out.splitlines() == ["registry.test:5000", "kind-test", f"registry.test:5000/ousast-runner@{DIGEST}", "3"] + assert profile_module.main(["--profile", str(path), "--print", "images"]) == 0, "the images file's path itself (up.sh writes it)" + assert capsys.readouterr().out.strip() == str(tmp_path / "images.json") + assert profile_module.main(["--profile", str(path), "--print", "images.engine"]) == 2 + assert "no image named 'engine'" in capsys.readouterr().err + assert profile_module.main(["--profile", str(path), "--print", "nothing"]) == 2 + + +def test_profile_is_frozen_and_typed() -> None: + loaded = load_profile("kind", environ={}) + assert isinstance(loaded, PlaneProfile) + with pytest.raises(AttributeError): + loaded.exec = "remote" # type: ignore[misc] diff --git a/tests/test_plane_reconciler.py b/tests/test_plane_reconciler.py index 9b8be51..9375a8a 100644 --- a/tests/test_plane_reconciler.py +++ b/tests/test_plane_reconciler.py @@ -4,9 +4,10 @@ The fake is a Python script the tests write into ``tmp_path`` that follows ax's internal/controller/reconciler.go, internal/server/server.go): ``apply`` creates the task ``Suspended``; ``resume`` fails with DeadlineExceeded the first time (ax then reports ``Failed``, as while Agent Substrate builds a new image's golden snapshot) and succeeds after; ``get task`` answers ``Running`` forever -- ax has no Completed phase --- and on the scripted poll it POSTs a scripted tar to the ``OUSAST_ARTIFACT_URL`` in the applied Task's env, or -turns ``Failed`` with a condition message, or vanishes. Every outcome comes from a scripted delivery, and a -delivery happens only after the reconciler's start request reached the fake router (Req 4.5). +-- and on the scripted poll it PUTs a scripted tar to the presigned ``delivery.put`` URL the start request carried +(plane-on-kubernetes 2.4; or POSTs it to ``OUSAST_ARTIFACT_URL`` in receiver mode), or turns ``Failed`` with a +condition message, or vanishes. Every outcome comes from a scripted delivery, and a delivery happens only after +the reconciler's start request reached the fake router (Req 4.5). The store is ``plane_fake_store.HttpObjectStore``. """ from __future__ import annotations @@ -28,11 +29,13 @@ from pathlib import Path import pytest import yaml +from plane_fake_store import HttpObjectStore from openultrasast.plane import reconciler from openultrasast.plane.workspaces import workspaces RECONCILER = Path("src/openultrasast/plane/reconciler.py") +PROFILE_CONTEXT = "kind-test" # conftest's plane_profile fixture: the context every fake must be handed PIN = "0123456789abcdef0123456789abcdef01234567" SECRET = "sk-reconciler-test-7a2b-only-in-the-start-request" @@ -145,11 +148,15 @@ elif argv[:2] == ["get", "task"]: started = (HERE / "started" / argv[2]).exists() # the runner runs its command only after the start request if rec["phase"] == "Running" and rec["polls"] >= script.get("polls", 1) and started and not rec.get("sent"): rec["sent"] = True + body = json.loads((HERE / "started" / argv[2]).read_text() or "{{}}") # the start body the router saw + delivery = body.get("delivery") or {{}} if script.get("deliver", True) and "phase" not in script and not script.get("vanish"): files = script.get("files", {{}}) - base = env["OUSAST_ARTIFACT_URL"].rstrip("/") + "/inputs/" + base = env.get("OUSAST_ARTIFACT_URL", "").rstrip("/") + "/inputs/" for name, ref in json.loads(env.get("OUSAST_INPUTS", "{{}}")).items(): - fetch = urllib.request.Request(base + ref, headers={{"X-Ousast-Run": env["OUSAST_RUN"], "X-Ousast-Task": task}}) + presigned = (delivery.get("inputs") or {{}}).get(name) + fetch = urllib.request.Request(presigned) if presigned else urllib.request.Request( + base + ref, headers={{"X-Ousast-Run": env["OUSAST_RUN"], "X-Ousast-Task": task}}) try: with urllib.request.urlopen(fetch) as resp: dest = HERE / "fetched" / task / ref @@ -170,7 +177,11 @@ elif argv[:2] == ["get", "task"]: tar.addfile(info, io.BytesIO(data)) headers = {{"Content-Type": "application/x-tar", "X-Ousast-Run": env["OUSAST_RUN"]}} headers["X-Ousast-Task"] = script.get("as_task", task) - req = urllib.request.Request(env["OUSAST_ARTIFACT_URL"], data=buf.getvalue(), method="POST", headers=headers) + if delivery.get("put"): # the store path: PUT to the presigned URL, no receiver headers + tar_headers = {{"Content-Type": "application/x-tar"}} + req = urllib.request.Request(delivery["put"], data=buf.getvalue(), method="PUT", headers=tar_headers) + else: + req = urllib.request.Request(env["OUSAST_ARTIFACT_URL"], data=buf.getvalue(), method="POST", headers=headers) try: with urllib.request.urlopen(req) as resp: (HERE / "ax.log").open("a").write(f"{{time.monotonic():.4f}} delivered {{task}} {{resp.status}}\\n") @@ -282,6 +293,7 @@ SCRIPT = { class Fake: router: FakeRouter + store: HttpObjectStore def __init__(self, root: Path) -> None: self.root = root @@ -342,7 +354,8 @@ class _RouterHandler(BaseHTTPRequestHandler): self.server.starts.append({"at": time.monotonic(), "path": self.path, "target": target, "body": body, "code": code}) if code == 202: (self.server.root / "started").mkdir(exist_ok=True) - (self.server.root / "started" / name).touch() + without_secrets = {k: v for k, v in body.items() if k != "credentials"} # the fake ax reads delivery from it + (self.server.root / "started" / name).write_text(json.dumps(without_secrets)) with (self.server.root / "ax.log").open("a") as log: log.write(f"{time.monotonic():.4f} started {name} {code}\n") self.send_response(code) @@ -355,7 +368,7 @@ class _RouterHandler(BaseHTTPRequestHandler): @pytest.fixture -def fake(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, kubectl_ate: Path) -> Iterator[Fake]: +def fake(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, kubectl_ate: Path, plane_profile: Path) -> Iterator[Fake]: monkeypatch.setenv("OUSAST_RESULTS", str(tmp_path / "results")) monkeypatch.setenv("OUSAST_ARTIFACT_HOST", "127.0.0.1") monkeypatch.setenv("OUSAST_POLL_SECONDS", "0.02") @@ -365,12 +378,15 @@ def fake(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, kubectl_ate: Path) -> fake = Fake(tmp_path) fake.script(SCRIPT) fake.router = FakeRouter(tmp_path) + fake.store = HttpObjectStore() # the memory store the reconciler opens first: S3-shaped, presigned URLs served locally + monkeypatch.setattr(reconciler, "open_store", lambda spec=None, environ=None: fake.store) monkeypatch.setenv("OUSAST_ROUTER_URL", f"http://127.0.0.1:{fake.router.server_address[1]}") try: yield fake finally: fake.router.shutdown() fake.router.server_close() + fake.store.close() def write_run(tmp_path: Path, name: str, tasks: str = CHAIN) -> Path: @@ -404,13 +420,17 @@ def test_chain_runs_in_order_with_inputs_env_and_model(fake: Fake, tmp_path: Pat assert env["OUSAST_BUDGET_USD"] == "2.5" and env["OUSAST_BUDGET_CALLS"] == "40" assert env["OUSAST_INPUT_FACTS"] == "/workspace/.ousast-in/verify/repo-facts/facts.json" assert json.loads(env["OUSAST_INPUTS"]) == {"FACTS": "repo-facts/facts.json"} - assert env["OUSAST_ARTIFACT_URL"].startswith("http://127.0.0.1:") + assert "OUSAST_ARTIFACT_URL" not in env, "the store path renders no receiver address into the Task" assert env["OUSAST_MODEL"] == "deepseek-flash" assert json.loads(env["OUSAST_MODEL_PARAMS"]) == {"cache_hit_per_m": 0.07, "input_per_m": 0.27, "output_per_m": 1.1} kinds = [(d["kind"], d["metadata"]["name"]) for d in verify["docs"]] assert kinds == [("Model", "deepseek-flash"), ("Workspace", "case-pin"), ("Task", "chain-verify")], "no inputs Workspace" assert verify["docs"][-1]["spec"]["workspaces"] == [{"name": "case-pin", "path": "/workspace"}] - assert [n for _, n in fake.events("fetched")] == ["verify FACTS 200", "agree PASS 200"], "inputs come over the receiver's GET" + assert [n for _, n in fake.events("fetched")] == ["verify FACTS 200", "agree PASS 200"], "inputs come by their presigned GET" + assert ("GET", "runs/chain/repo-facts/facts.json") in fake.store.requests and ( + "PUT", + "runs/chain/verify/output.tar", + ) in fake.store.requests assert json.loads((tmp_path / "fetched" / "verify" / "repo-facts" / "facts.json").read_text()) == {"a.py": {"functions": ["f"]}} assert json.loads((tmp_path / "fetched" / "agree" / "verify" / "agreed.json").read_text()) == {"agreed": ["a.py:f"]} assert "annotations" not in verify["docs"][-1]["metadata"], "ax's ObjectMeta has no annotations" @@ -569,7 +589,15 @@ def test_start_is_sent_once_after_resume_with_the_models_credential(fake: Fake, assert [s["target"] for s in starts] == ["default/sig-repo-facts", "default/sig-verify", "default/sig-agree"], "once each" assert all(s["path"] == "/ousast/v1/start" and s["code"] == 202 for s in starts) by_task = {s["body"]["task"]: s for s in starts} - assert by_task["sig-verify"]["body"] == {"run": "sig", "task": "sig-verify", "credentials": {"DEEPSEEK_API_KEY": SECRET}} + body = by_task["sig-verify"]["body"] + assert {k: body[k] for k in ("run", "task", "credentials")} == { + "run": "sig", + "task": "sig-verify", + "credentials": {"DEEPSEEK_API_KEY": SECRET}, + } + assert set(body["delivery"]) == {"put", "inputs"} and "/runs/sig/verify/output.tar?" in body["delivery"]["put"] + assert set(body["delivery"]["inputs"]) == {"FACTS"} and "/runs/sig/repo-facts/facts.json?" in body["delivery"]["inputs"]["FACTS"] + assert all("X-Amz" not in line for line in (tmp_path / "ax.log").read_text().splitlines()), "no presigned URL in the fake's log" assert by_task["sig-repo-facts"]["body"]["credentials"] == {} and by_task["sig-agree"]["body"]["credentials"] == {}, "no Model" accepted = {} for stamp, words in fake.log(): # the last resume of each task is the one ax accepted @@ -583,8 +611,10 @@ def test_start_is_sent_once_after_resume_with_the_models_credential(fake: Fake, assert verify_model["spec"]["secretKey"] == {"name": "deepseek", "key": "DEEPSEEK_API_KEY"}, "the manifest names the key" assert "DEEPSEEK_API_KEY" not in fake.applied("sig", "verify")["env"], "the rendered Task carries no credential" rendered = str(reconciler.run_dir("sig") / "verify" / "task.yaml") - assert rendered in files_containing(tmp_path, "OUSAST_ARTIFACT_URL"), "the scan reads the rendered files it vouches for" + assert rendered in files_containing(tmp_path, "OUSAST_INPUTS"), "the scan reads the rendered files it vouches for" assert files_containing(tmp_path, SECRET) == [], "no rendered task.yaml, state, log or applied record holds the value" + leaked = [p for p in files_containing(tmp_path, "X-Amz-Signature") if "/started/" not in p] # the fake router's own records + assert leaked == [], "nor a presigned URL in any product file (the fake store signs with that name)" def test_a_model_bound_task_without_its_secret_fails_before_ax_sees_it(fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: @@ -631,7 +661,8 @@ def tar_of(files: dict[str, bytes]) -> bytes: return buf.getvalue() -def test_receiver_rejects_traversal_unknown_tasks_and_wrong_types(tmp_path: Path) -> None: +def test_receiver_rejects_traversal_unknown_tasks_and_wrong_types(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("OUSAST_ARTIFACT_HOST", "127.0.0.1") # the direct test address: no Service lookup, no context needed receiver = reconciler.Receiver("r", tmp_path / "r", 0) receiver.running.add("t") headers = {"X-Ousast-Run": "r", "X-Ousast-Task": "t", "Content-Type": "application/x-tar"} @@ -647,6 +678,7 @@ def test_receiver_rejects_traversal_unknown_tasks_and_wrong_types(tmp_path: Path def test_delivery_for_a_task_that_is_not_running_is_rejected(fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("OUSAST_DELIVERY", "receiver") # the receiver path, kept for group 2 (task 2.6 removes this test) monkeypatch.setenv("OUSAST_TASK_TIMEOUT", "1") # the rejected delivery is no delivery: the task times out script = json.loads(json.dumps(SCRIPT)) script["repo-facts"]["as_task"] = "agree" # the runner claims to be a task that has not started @@ -719,16 +751,31 @@ def test_status_of_an_unknown_run_says_so(fake: Fake) -> None: # --- doctor, workspaces, size --------------------------------------------------------------------------------------- -def test_doctor_names_the_bring_up_script_when_nothing_is_reachable(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: - monkeypatch.setenv("PATH", str(tmp_path)) # no kubectl, no curl +def test_doctor_checks_the_profiles_cluster_and_reports_every_address( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path, plane_profile: Path +) -> None: + """Req 1.2: the configured context (never kind by name), the registry's runner reference, the memory store; the + namespaces are not checked behind an unreachable context, so only that line fails for it.""" + monkeypatch.setenv("PATH", str(tmp_path)) # no kubectl, no docker + from openultrasast.plane import doctor as doctor_module + + monkeypatch.setattr(doctor_module, "_registry_has", lambda image: (False, f"registry.test:5000 unreachable for {image}")) checks = reconciler.doctor() - assert [name for name, _, _ in checks] == [ - "kind cluster", - "agent substrate (ate-system)", - "ax controller (ax-system)", - "runner image in kind registry", + names = [name for name, _, _ in checks] + assert names == [ + "profile test", + "cluster kind-test", + "runner image in registry.test:5000", + f"memory store file://{tmp_path / 'memory'}", ] - assert all(not ok and "ops/ax/up.sh" in text for _, ok, text in checks) + by_name = {name: (ok, text) for name, ok, text in checks} + assert by_name["profile test"][0] and "kube_context=kind-test" in by_name["profile test"][1] + for address in ("registry=registry.test:5000", "exec=local", "atespace=default", "router_url=(tunnel"): + assert address in by_name["profile test"][1], address + assert not by_name["cluster kind-test"][0] and "kind-test unreachable" in by_name["cluster kind-test"][1] + assert not by_name["runner image in registry.test:5000"][0] + assert by_name[f"memory store file://{tmp_path / 'memory'}"][0], "a file store opens; an S3 store is verified" + assert not any("kind-" + "ousast" in text for _, _, text in checks), "nothing names the kind cluster" def test_workspaces_one_manifest_per_case_pin(tmp_path: Path) -> None: @@ -755,6 +802,23 @@ def test_workspaces_one_manifest_per_case_pin(tmp_path: Path) -> None: assert loaded["demo-sqli-benign-tip"].pins == {"repo": "d" * 40} +def test_cli_doctor_takes_a_profile_and_fails_on_its_unreachable_context( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path, plane_profile: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """plane-on-kubernetes 1.2: ``ousast plane doctor --profile`` drives the checks from that file, nothing built.""" + from openultrasast.cli import main + from openultrasast.plane import doctor as doctor_module + + monkeypatch.setenv("PATH", str(tmp_path)) + monkeypatch.setattr(doctor_module, "_registry_has", lambda image: (True, "present")) + monkeypatch.delenv("OUSAST_PLANE_PROFILE") + assert main(["plane", "doctor", "--profile", str(plane_profile)]) == 1 + out = capsys.readouterr().out + assert "[FAIL] cluster kind-test:" in out and "[ok] profile test:" in out and "[ok] runner image in registry.test:5000" in out + assert main(["plane", "doctor", "--profile", str(tmp_path / "missing.toml")]) == 2 + assert "missing.toml" in capsys.readouterr().err + + def test_rendered_documents_carry_only_ax_fields(tmp_path: Path) -> None: """Req 1.1/1.4: whatever ``ax apply`` receives names only fields ax documents; annotations never leave.""" (tmp_path / "repo-facts").mkdir() @@ -798,9 +862,13 @@ def test_mark_done_sets_the_state_under_the_run_lock(tmp_path: Path) -> None: assert not (base / "lock").exists(), "the lock is released" -def test_reconciler_is_under_500_lines_and_holds_no_pipeline_logic() -> None: +def test_reconciler_is_under_550_lines_and_holds_no_pipeline_logic() -> None: text = RECONCILER.read_text(encoding="utf-8") - assert len(text.splitlines()) < 500, "Req 3.4: the reconciler stays under 500 lines" + assert len(text.splitlines()) < 550, ( + "Req 3.4: the reconciler stays under 500 lines; plane-on-kubernetes 1.6/2.1 moved the bound to 550 while the " + "store-delivery wiring (Delivery, state mirror and seed, the start-body delivery) sits beside the receiver path " + "that task 2.6 removes -- 2.6 records the count and returns the bound to 500 if it allows" + ) for word in ("prompt(", "openai", "ChatClient", "score(", "system_prompt"): assert word not in text, f"Req 3.4: no pipeline logic in the reconciler ({word})" assert "Popen" not in text and "openultrasast.plane.tasks" not in text, "ax is the only executor" @@ -829,7 +897,7 @@ def test_doctor_treats_an_empty_namespace_as_not_ready(monkeypatch: pytest.Monke from openultrasast.plane import doctor monkeypatch.setattr(doctor, "_sh", lambda *a: (True, "No resources found in ax-system namespace.")) - ok, text = doctor._pods_ready("ax-system", "kind-ousast") + ok, text = doctor._pods_ready("ax-system", "kind-test") assert not ok and "no pods" in text monkeypatch.setattr(doctor, "_sh", lambda *a: (True, "a-1 1/1 Running 0 1m\nb-2 0/1 Completed 0 1m")) assert doctor._pods_ready("ns", "ctx") == (True, "2 pods ready") @@ -854,11 +922,13 @@ def test_egress_policy_between_apply_and_resume_and_checked_after_delete(fake: F order.append(first(prefix, order[-1] + 1)) assert order == sorted(order) assert not (tmp_path / "policies" / "default_eg-verify.json").exists(), "the policy went with its actor" + contexts = {line.split()[-1] for line in lines if line.startswith("egress ")} + assert contexts == {PROFILE_CONTEXT}, "kubectl-ate received the profile's context, never a built one (1.2)" written = json.loads((tmp_path / "policies" / "default_eg-verify.written.json").read_text()) - tls = {"hostnames": ["api.deepseek.com", "example.invalid"], "ports": {"numbers": [443]}} - assert written["rules"] == [{"tlsPassthrough": tls}], "the Model host and the Git host, nothing else" + tls = {"hostnames": ["api.deepseek.com", "example.invalid", "localhost"], "ports": {"numbers": [443]}} + assert written["rules"] == [{"tlsPassthrough": tls}], "the Model host, the Git host and the store host, nothing else" facts = json.loads((tmp_path / "policies" / "default_eg-repo-facts.written.json").read_text()) - assert facts["rules"] == [{"tlsPassthrough": {"hostnames": ["example.invalid"], "ports": {"numbers": [443]}}}] + assert facts["rules"] == [{"tlsPassthrough": {"hostnames": ["example.invalid", "localhost"], "ports": {"numbers": [443]}}}] def test_a_refused_egress_policy_fails_the_task_before_resume(fake: Fake, tmp_path: Path, kubectl_ate: Path) -> None: @@ -943,3 +1013,82 @@ def test_ax_task_names_leave_room_for_the_template_suffix() -> None: assert other != long, "no collision between tasks sharing a long prefix" assert reconciler._ax_name("run", "facts", limit=reconciler.TASK_NAME_LIMIT) == "run-facts" assert re.fullmatch(r"[a-z0-9]([a-z0-9-]*[a-z0-9])?", long) + + +# --- the store is the state of record (plane-on-kubernetes 2.1, design section 2) ----------------------------------- + + +def test_state_is_mirrored_to_the_store_and_seeds_a_rerun_from_an_empty_results_root( + fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + assert reconciler.run(write_run(tmp_path, "mir"), ax=str(fake.ax)) == "done" + mirrored = json.loads(fake.store.objects["runs/mir/state.json"].decode()) + assert mirrored == state_of("mir") and mirrored["status"] == "done", "mirrored after the last change" + applies = len(fake.events("apply")) + monkeypatch.setenv("OUSAST_RESULTS", str(tmp_path / "elsewhere")) # another machine: no local state at all + assert reconciler.run(write_run(tmp_path, "mir"), ax=str(fake.ax)) == "done" + assert statuses("mir") == {"repo-facts": "done", "verify": "done", "agree": "done"}, "seeded from the store" + assert len(fake.events("apply")) == applies, "nothing was submitted again" + + +def test_a_store_that_cannot_open_stops_the_run_before_ax_and_leaves_no_lock( + fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + from openultrasast.plane import memory + from openultrasast.plane.memory import MemoryStoreError + + monkeypatch.setattr(reconciler, "open_store", memory.open_store) # the real one, not the fixture's fake + monkeypatch.setenv("OUSAST_MEMORY", "s3://nowhere") # the profile's override; no S3_* in the test environment + for key in ("S3_ENDPOINT", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"): + monkeypatch.delenv(key, raising=False) + with pytest.raises(MemoryStoreError, match="S3_ENDPOINT"): + reconciler.run(write_run(tmp_path, "nostore"), ax=str(fake.ax)) + assert fake.events("apply") == [] and not (reconciler.run_dir("nostore") / "lock").exists() + + +# --- the store path end to end, and the receiver path kept for group 2 (plane-on-kubernetes 2.4) -------------------- + + +def test_the_store_holds_the_tar_the_declared_outputs_and_done_json(fake: Fake, tmp_path: Path) -> None: + """Completion is the tar in the store: downloaded, extracted under the run dir, declared outputs re-put as single + objects (the consumer's GET names one artifact), done.json with the summary's status and the tar's size.""" + assert reconciler.run(write_run(tmp_path, "st"), ax=str(fake.ax)) == "done" + keys = sorted(k for k in fake.store.objects if k.startswith("runs/st/")) + assert keys == [ + "runs/st/agree/done.json", + "runs/st/agree/output.tar", + "runs/st/repo-facts/done.json", + "runs/st/repo-facts/facts.json", + "runs/st/repo-facts/output.tar", + "runs/st/state.json", + "runs/st/verify/agreed.json", + "runs/st/verify/done.json", + "runs/st/verify/output.tar", + ], "declared outputs only (summary.json is extracted locally, not declared by this Run)" + done = json.loads(fake.store.objects["runs/st/repo-facts/done.json"]) + assert done["status"] == "done" and done["bytes"] == len(fake.store.objects["runs/st/repo-facts/output.tar"]) and done["files"] == 2 + assert json.loads((reconciler.run_dir("st") / "repo-facts" / "facts.json").read_text()) == {"a.py": {"functions": ["f"]}} + assert fake.store.objects["runs/st/repo-facts/facts.json"] == (reconciler.run_dir("st") / "repo-facts" / "facts.json").read_bytes() + + +def test_a_tar_the_store_path_cannot_accept_fails_the_task(fake: Fake, tmp_path: Path) -> None: + script = json.loads(json.dumps(SCRIPT)) + script["repo-facts"]["files"] = {"../escape.json": "{}", "summary.json": FACTS_SUMMARY} + fake.script(script) + assert reconciler.run(write_run(tmp_path, "esc"), ax=str(fake.ax)) == "failed" + assert "rejected member ../escape.json" in state_of("esc")["tasks"]["repo-facts"]["reason"] + assert not (reconciler.run_dir("esc").parent / "escape.json").exists() + + +def test_receiver_mode_still_runs_through_the_host_receiver(fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """OUSAST_DELIVERY=receiver (group 2 only): the old path, with its http rule and OUSAST_ARTIFACT_URL, until 2.6.""" + monkeypatch.setenv("OUSAST_DELIVERY", "receiver") + assert reconciler.run(write_run(tmp_path, "rcv"), ax=str(fake.ax)) == "done" + assert fake.applied("rcv", "verify")["env"]["OUSAST_ARTIFACT_URL"].startswith("http://127.0.0.1:") + assert [s["body"].get("delivery") for s in fake.router.starts] == [None, None, None], "no presigned URLs in receiver mode" + assert not any(k.startswith("runs/rcv/") and k.endswith("output.tar") for k in fake.store.objects), "nothing PUT to the store" + assert "runs/rcv/state.json" in fake.store.objects, "the state is still mirrored" + written = json.loads((tmp_path / "policies" / "default_rcv-repo-facts.written.json").read_text()) + assert written["rules"] == [{"tlsPassthrough": {"hostnames": ["example.invalid"], "ports": {"numbers": [443]}}}], ( + "an IP receiver: no http rule" + ) diff --git a/tests/test_plane_router.py b/tests/test_plane_router.py index 1ff0453..a0d15c5 100644 --- a/tests/test_plane_router.py +++ b/tests/test_plane_router.py @@ -93,17 +93,17 @@ def alive(pid: int) -> bool: @pytest.fixture(autouse=True) def _clean_env(monkeypatch: pytest.MonkeyPatch) -> None: - for key in ("OUSAST_ROUTER_URL", "OUSAST_START_TIMEOUT", "KIND_CLUSTER_NAME", "FAKE_KUBECTL_MODE"): + for key in ("OUSAST_ROUTER_URL", "OUSAST_START_TIMEOUT", "FAKE_KUBECTL_MODE"): monkeypatch.delenv(key, raising=False) def test_port_forward_port_is_parsed_and_the_child_is_terminated_on_exit(tmp_path: Path, fake_router: FakeRouter) -> None: port = int(fake_router.server_address[1]) kubectl = write_kubectl(tmp_path, port) - with open_router(kubectl=str(kubectl)) as router: + with open_router("kind-test", kubectl=str(kubectl)) as router: # the profile's context, as the reconciler passes it assert router.url == f"http://127.0.0.1:{port}" seen = json.loads((tmp_path / "kubectl.json").read_text()) - assert seen["argv"] == ["--context", "kind-ousast", "-n", "ate-system", "port-forward", "svc/atenet-router", ":80"] + assert seen["argv"] == ["--context", "kind-test", "-n", "ate-system", "port-forward", "svc/atenet-router", ":80"] assert alive(seen["pid"]) router.start_task("default", "run-t", "run", {}) deadline = time.monotonic() + 5 @@ -123,15 +123,23 @@ def test_a_failing_port_forward_raises_with_kubectls_output(tmp_path: Path, monk def test_router_url_override_starts_no_port_forward(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None: monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url) - with open_router(kubectl=str(tmp_path / "no-such-kubectl")) as router: + with open_router("kind-test", kubectl=str(tmp_path / "no-such-kubectl")) as router: assert router.url == fake_router.url assert not (tmp_path / "kubectl.json").exists() + monkeypatch.delenv("OUSAST_ROUTER_URL") + with open_router("kind-test", url=fake_router.url, kubectl=str(tmp_path / "no-such-kubectl")) as router: + assert router.url == fake_router.url, "the profile's router_url is the same override" + + +def test_open_router_refuses_to_build_a_context(tmp_path: Path) -> None: + with pytest.raises(StartError, match="kube context"), open_router("", kubectl=str(tmp_path / "no-such-kubectl")): + pass @pytest.mark.parametrize("fake_router", [[(503, "no actor yet"), (502, "bad gateway"), (202, "started")]], indirect=True) def test_start_retries_503_and_502_then_succeeds(monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None: monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url) - with open_router() as router: + with open_router("kind-test") as router: router.start_task("default", "chain-verify", "chain", {"DEEPSEEK_API_KEY": SECRET}) assert len(fake_router.requests) == 3 path, headers, body = fake_router.requests[-1] @@ -142,7 +150,7 @@ def test_start_retries_503_and_502_then_succeeds(monkeypatch: pytest.MonkeyPatch @pytest.mark.parametrize("fake_router", [[(409, f"this actor runs task 'x' (echo {SECRET})")]], indirect=True) def test_a_409_is_raised_at_once_with_the_response_text_redacted(monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None: monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url) - with open_router() as router, pytest.raises(StartError, match="HTTP 409.*runs task 'x'") as caught: + with open_router("kind-test") as router, pytest.raises(StartError, match="HTTP 409.*runs task 'x'") as caught: router.start_task("default", "t", "r", {"DEEPSEEK_API_KEY": SECRET}) assert len(fake_router.requests) == 1 and SECRET not in str(caught.value) assert router.starter("default", "t", "r", {})() is not None, "the starter returns the refusal instead of raising" @@ -153,7 +161,7 @@ def test_a_restart_after_re_resume_accepts_a_running_command(monkeypatch: pytest """First start: 'already started' is a refusal. After a re-resume the actor may not have rebooted; then the runner's 'already started on this boot' means the command still runs, not a failure.""" monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url) - with open_router() as router: + with open_router("kind-test") as router: start = router.starter("default", "t", "r", {}) assert start() is not None, "a first start that finds the command running is a refusal" assert start() is None, "a re-sent start after a re-resume is not" @@ -165,7 +173,7 @@ def test_a_restart_after_re_resume_accepts_a_running_command(monkeypatch: pytest def test_start_gives_up_after_the_start_timeout(monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None: monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url) monkeypatch.setenv("OUSAST_START_TIMEOUT", "1") - with open_router() as router, pytest.raises(StartError, match="OUSAST_START_TIMEOUT.*workspaces not materialised"): + with open_router("kind-test") as router, pytest.raises(StartError, match="OUSAST_START_TIMEOUT.*workspaces not materialised"): router.start_task("default", "t", "r", {}) assert len(fake_router.requests) >= 2 @@ -173,7 +181,7 @@ def test_start_gives_up_after_the_start_timeout(monkeypatch: pytest.MonkeyPatch, def test_connection_errors_are_retried_until_the_timeout(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("OUSAST_ROUTER_URL", "http://127.0.0.1:9") monkeypatch.setenv("OUSAST_START_TIMEOUT", "1") - with open_router() as router, pytest.raises(StartError, match="within 1s"): + with open_router("kind-test") as router, pytest.raises(StartError, match="within 1s"): router.start_task("default", "t", "r", {}) diff --git a/tests/test_plane_runner.py b/tests/test_plane_runner.py index f2f2678..366dc62 100644 --- a/tests/test_plane_runner.py +++ b/tests/test_plane_runner.py @@ -23,6 +23,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path import pytest +from plane_fake_store import HttpObjectStore from openultrasast.plane import egress, runner from openultrasast.plane.manifests import GitSource @@ -36,6 +37,7 @@ from openultrasast.plane.runner import ( clone_destination, deliver, find_cached_checkout, + parse_delivery, start_health_server, ) @@ -850,3 +852,121 @@ def test_fetch_dials_the_override_and_sends_the_url_host(tmp_path: Path) -> None server.server_close() assert size == 2 and (tmp_path / "x" / "a.json").read_bytes() == b"ok" assert seen[0]["path"] == "/inputs/p/a.json" and seen[0]["Host"] == "receiver.example" and seen[0]["X-Ousast-Run"] == "r" + + +# --- store delivery through the start body (plane-on-kubernetes 2.2, design section 3) --------------------------- + + +@pytest.fixture +def object_store() -> Iterator[HttpObjectStore]: + fake = HttpObjectStore() + try: + yield fake + finally: + fake.close() + + +def test_the_start_body_may_carry_delivery_targets_and_refuses_unsafe_urls() -> None: + """``delivery {put, inputs}``: https to a hostname, never an address or http; the answer echoes no URL.""" + gate = StartGate() + gate.arm("r", "t") + bad = [ + ({"put": "http://store.example/b/k?sig=1"}, "delivery.put must be an https URL"), + ({"put": "https://10.0.0.7/b/k"}, "delivery.put must name a host, not an address"), + ({"put": "https://user:pw@store.example/k"}, "delivery.put must be an https URL"), + ({"inputs": {"FACTS": "ftp://store.example/k"}}, "delivery.inputs.FACTS must be an https URL"), + ({"inputs": {"lower": "https://store.example/k"}}, "delivery.inputs must map"), + ({"elsewhere": "x"}, "delivery must be an object"), + ("https://store.example/k", "delivery must be an object"), + ] + for delivery, message in bad: + status, text = gate.offer(json.dumps({"run": "r", "task": "t", "delivery": delivery}).encode()) + assert status == 400 and message in text, (delivery, status, text) + assert "store.example" not in text and "10.0.0.7" not in text, "the answer never echoes a URL" + body = { + "run": "r", + "task": "t", + "delivery": { + "put": "https://store.example/b/t/output.tar?X-Amz-Signature=s", + "inputs": {"FACTS": "https://store.example/b/p/facts.json?X-Amz-Signature=s"}, + }, + } + assert gate.offer(json.dumps(body).encode()) == (202, "started") + start = gate.wait(threading.Event()) + assert start is not None and start.delivery is not None + assert start.delivery.put == body["delivery"]["put"] and start.delivery.inputs == body["delivery"]["inputs"] + assert "X-Amz" not in repr(start) and "store.example" not in repr(start), "a Start never prints a URL" + assert parse_delivery(None) is None and parse_delivery({"put": "http://localhost:1/k"}, insecure=True) is not None + with pytest.raises(ValueError, match="must be an https URL"): + parse_delivery({"put": "http://localhost:1/k"}) + + +def test_output_is_put_to_the_presigned_url_and_inputs_are_fetched_from_theirs( + env: dict[str, str], object_store: HttpObjectStore, tmp_path: Path, caplog: pytest.LogCaptureFixture +) -> None: + """End to end against the S3-shaped fake: the start carries the URLs, the runner PUTs the tar and GETs the input; + no receiver, no OUSAST_ARTIFACT_URL, no URL in any log line.""" + caplog.set_level(logging.DEBUG) + object_store._put("runs/run-1/repo-facts/facts.json", b'{"a.py": ["f"]}') + put_url = object_store.presign_put("runs/run-1/stub/output.tar") + facts_url = object_store.presign_get("runs/run-1/repo-facts/facts.json") + thread, environ, codes = main_in_thread( + tmp_path, + AX_RUNNER_INSECURE_DELIVERY="1", + OUSAST_TASK="stub", + OUSAST_INPUTS=json.dumps({"FACTS": "repo-facts/facts.json"}), + OUSAST_INPUT_FACTS=str(tmp_path / "ws" / "in" / "repo-facts" / "facts.json"), + ) + body = {"run": "run-1", "task": "stub", "delivery": {"put": put_url, "inputs": {"FACTS": facts_url}}} + assert post_start(environ["AX_RUNNER_BOUND_PORT"], body) == (202, "started") + thread.join(timeout=30) + assert codes == [0] + assert (tmp_path / "ws" / "in" / "repo-facts" / "facts.json").read_bytes() == b'{"a.py": ["f"]}', "the input came by its URL" + assert ("GET", "runs/run-1/repo-facts/facts.json") in object_store.requests and ( + "PUT", + "runs/run-1/stub/output.tar", + ) in object_store.requests + with tarfile.open(fileobj=io.BytesIO(object_store.objects["runs/run-1/stub/output.tar"]), mode="r:") as tar: + names = sorted(tar.getnames()) + assert "facts.json" in names and "summary.json" in names, "the whole output directory, as a tar, in the store" + marker = json.loads((tmp_path / "state" / "stub.done.json").read_text()) + assert marker["delivered"] is True and marker["status"] == "done" + assert "X-Amz" not in caplog.text and "fake-bucket" not in caplog.text, "no presigned URL in any log line" + assert files_containing(tmp_path, "X-Amz-Signature") == [], "nor in any file" + + +def test_a_failed_put_is_retried_alone_on_the_next_start_with_a_fresh_url( + env: dict[str, str], object_store: HttpObjectStore, tmp_path: Path +) -> None: + """The marker and retry-only boot are unchanged: a PUT that fails (wrong bucket: 404) ends the task failed with + the URL's host and path only; the next boot's start brings a working URL and only the delivery runs again.""" + counter = tmp_path / "runs.txt" + wrong = object_store.presign_put("runs/run-1/stub/output.tar").replace("/fake-bucket/", "/no-such-bucket/") + thread, environ, codes = main_in_thread( + tmp_path, AX_RUNNER_INSECURE_DELIVERY="1", STUB_COUNTER=str(counter), OUSAST_DELIVERY_BACKOFF="0" + ) + assert post_start(environ["AX_RUNNER_BOUND_PORT"], {"run": "run-1", "task": "stub", "delivery": {"put": wrong}})[0] == 202 + thread.join(timeout=30) + assert codes == [EXIT_BY_STATUS["failed"]] + marker = json.loads((tmp_path / "state" / "stub.done.json").read_text()) + assert marker["delivered"] is False and marker["status"] == "failed" and marker["exit"] == 0 + assert "HTTP 404" in marker["reason"] and "X-Amz" not in marker["reason"], "the reason names the status, not the signature" + thread, environ, codes = main_in_thread( + tmp_path, AX_RUNNER_INSECURE_DELIVERY="1", STUB_COUNTER=str(counter), OUSAST_DELIVERY_BACKOFF="0" + ) + good = object_store.presign_put("runs/run-1/stub/output.tar") + assert post_start(environ["AX_RUNNER_BOUND_PORT"], {"run": "run-1", "task": "stub", "delivery": {"put": good}})[0] == 202 + thread.join(timeout=30) + assert codes == [0] and len(counter.read_text().splitlines()) == 1, "only the delivery is retried" + assert "runs/run-1/stub/output.tar" in object_store.objects + assert json.loads((tmp_path / "state" / "stub.done.json").read_text())["delivered"] is True + + +def test_deliver_puts_and_names_only_the_safe_url_on_failure(object_store: HttpObjectStore) -> None: + url = object_store.presign_put("k/output.tar") + assert runner.deliver(url, b"tar", {"Content-Type": "application/x-tar"}, backoff=0, method="PUT") == 200 + assert object_store.objects["k/output.tar"] == b"tar" + with pytest.raises(runner.DeliveryError) as caught: + runner.deliver(url.replace("/fake-bucket/", "/other/"), b"x", {}, attempts=2, backoff=0, method="PUT") + assert "HTTP 404" in str(caught.value) and "X-Amz" not in str(caught.value) and "/other/k/output.tar" in str(caught.value) + assert runner.safe_url("https://h:1/p/q?X-Amz-Signature=s") == "https://h:1/p/q"