marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 11:49:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

merge(plane-on-kubernetes): groups 1-2 -- cluster profile, no host literals, delivery through the S3 store

Browse Source 
Kind proofs: records 01 and 02. Task 2.6 (receiver removal) is paused while the
safety net and the decision engine come first. deployment.md conflicts resolved
in the plain-language style; corrects the RuntimeClass claim (Substrate runs
runsc inside its worker pod).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-03 07:57:13 +02:00
co-authored by Claude Opus 5.5
parent 432fd2947e c8b418c34c
commit b24eeb15f3
62 changed files with 3080 additions and 433 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/plane-on-kubernetes/design.md
+7
View File
@@ -254,6 +254,13 @@ containerd; `Dockerfile:1-7` already argues against it).
a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a
separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays
`alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such.
**Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.**
With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor
requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS
1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so
group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether
Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the
k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path.
- **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and
`docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose
`engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`.
.kiro/specs/plane-on-kubernetes/tasks.md
+13 -13
View File
@@ -10,7 +10,7 @@ its kind proof and the record it writes under `benchmarks/measurements/<date>-k8
go-ahead** with a ceiling; steps that need a hand on a cluster that does not exist yet are marked **manual**.
- [ ] 1. Profile and literals (design section 1; open-question checks 2, 3, 7)
- [ ] 1.1 `PlaneProfile`
- [x] 1.1 `PlaneProfile`
- `src/openultrasast/plane/profile.py`: frozen dataclass with the fields of design section 1; `load_profile()`
reads `ops/k8s/profiles/<OUSAST_PLANE_PROFILE>.toml` (a name or a path; default `kind`), then applies
`OUSAST_<FIELD>` overrides; rejects an unknown key, a secret-looking key (`*_KEY`, `*_SECRET`), an `exec`
@@ -19,7 +19,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_profile.py`: defaults; file then env precedence; each rejection names the field; both
committed profiles load; no field holds a value matching a credential pattern.
- _Requirements: 1.1_
- [ ] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
- [x] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
- `doctor(profile)`: context from `profile.kube_context`; the registry check resolves the `images` file's runner
reference (`docker manifest inspect` for a remote registry, `/v2/_catalog` for a local one); `open_store`
verifies `profile.memory`; prints every configured address (Req 1.2). `Egress(context=...)`,
@@ -29,46 +29,46 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
profile's context, never a built one.
- Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile.
- _Requirements: 1.1, 1.2_
- [ ] 1.3 Host-literal scan test
- [x] 1.3 Host-literal scan test
- `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does)
over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`,
`localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged
`ax-tunnel`; fails listing `path:line`.
- Evidence: the test passes on this commit and fails when a literal is reinserted (shown once in the record).
- _Requirements: 1.3_
- [ ] 1.4 `up.sh` by profile
- [x] 1.4 `up.sh` by profile
- `ops/ax/up.sh`: `KO_DOCKER_REPO` and the context from `kind.toml` (read with `python -m
openultrasast.plane.profile --print registry|kube_context`); writes `kind-images.json` instead of
`runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3).
- Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes.
- _Requirements: 1.1, 2.4_
- [ ] 1.5 Placement probe (open question 2, decided before the engine work)
- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any`
- `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's
engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records
with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after.
- Evidence: `record.json` says whether Substrate places by resource fit (`placement: fit|any`). `any` selects
the design's fallback for group 5 and is written into `design.md` section 4 before group 5 starts.
- _Requirements: 4.1_
- [ ] 1.6 Reconciler bound (open question 7)
- [x] 1.6 Reconciler bound (open question 7) -- 489 lines after 1.2; projected 514 during group 2: the bound moves to 550 in 2.4
- Record `wc -l src/openultrasast/plane/reconciler.py` (477 at 17ac7ad) and the lines task 2.4 may add
(receiver wiring removed: `-12`, `Delivery` calls: `+8` estimated). If the estimate crosses 500, the bound
in `tests/test_plane_reconciler.py:803` moves to 550 in task 2.4 with the reason in the assertion message.
- Evidence: the numbers in `record.json`.
- _Requirements: 2.1_
- [ ] 1.7 gvisor tarball reachability (open question 3) **manual**
- [x] 1.7 gvisor tarball reachability (open question 3) **manual** -- documented as planned in `docs/deployment.md` B.1; the live check waits for k3s
- `docs/deployment.md` (B.1, planned): the k3s node must fetch
`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd` (`substrate/manifests/ate-install/sandboxconfig-gvisor.yaml`)
or a mirrored copy named in a copied SandboxConfig; the check is `kubectl -n ate-system logs` of the worker
pod's atelet fetch line, run by hand on the day k3s exists. Labelled planned until then.
- _Requirements: 7.2_
- [ ] 1.8 Kind proof and record 01
- [x] 1.8 Kind proof and record 01
- `ousast plane doctor --profile kind` passes every line; `ousast plane doctor --profile k3s` fails only on the
unreachable context (the expected line); `test_plane_literals` 0 offenders; the probe of 1.5; the bound of 1.6.
- `benchmarks/measurements/<date>-k8s-01-profile/record.json`.
- _Requirements: 1.1, 1.2, 1.3, 7.1_
- [ ] 2. Store delivery, receiver-less egress (design section 3)
- [ ] 2.1 Presigned URLs and the store run directory
- [x] 2.1 Presigned URLs and the store run directory
- `src/openultrasast/plane/delivery.py`: `Delivery(store, run)` mints presigned PUT for
`runs/<run>/<task>/output.tar` and GET for `runs/<run>/<producer>/<artifact>` (boto3 `generate_presigned_url`
on `S3Client.sdk`, expiry `OUSAST_TASK_TIMEOUT + 600`); `delivered(task)` HEADs the tar; `collect(task)`
@@ -78,7 +78,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_delivery.py`: against a local S3-shaped fake (PUT/GET/HEAD by key); URLs never appear in
logs; expiry; member rejection; declared-only re-put; state seeding.
- _Requirements: 3.1, 2.1_
- [ ] 2.2 Runner: `delivery` in the start body
- [x] 2.2 Runner: `delivery` in the start body
- `src/openultrasast/plane/runner.py`: `StartGate.offer` validates an optional `delivery {put, inputs}` (https,
hostname, no echo); `_deliver_output` PUTs the tar to `delivery.put` (same retry and `DeliveryError`
semantics); `fetch_inputs` GETs from `delivery.inputs[name]`; `OUSAST_ARTIFACT_URL` and `OUSAST_ARTIFACT_DIAL`
@@ -86,14 +86,14 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_runner.py`: accepted body with `delivery`; 400 on an `http` or IP URL; PUT to a local
server; inputs fetched from presigned-shaped URLs; the marker and retry-only boot unchanged.
- _Requirements: 3.1_
- [ ] 2.3 Egress without the receiver
- [x] 2.3 Egress without the receiver
- `src/openultrasast/plane/egress.py`: `policy_for(task, workspaces, model, store_host)` emits one
`tlsPassthrough` rule on 443 over the sorted union of store host, Git hosts, Model hosts; no `http` rule; an
IP store host raises naming `S3_ENDPOINT`.
- `tests/test_plane_egress.py`: for every committed template under `plane/tasks/` the rendered policy is the
expected set; the IP refusal.
- _Requirements: 3.2_
- [ ] 2.4 Reconciler on `Delivery`
- [x] 2.4 Reconciler on `Delivery`
- `src/openultrasast/plane/reconciler.py`: `run()` opens the store first (no fallback), builds `Delivery`,
passes `delivery` to `router.starter`/`start_task` (body extended in `router.py`), `_await` on
`Delivery.delivered`, outcome from the collected `summary.json`; the receiver thread is kept behind
@@ -101,7 +101,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_reconciler.py`: the ax fake run completes through the delivery fake; a rerun from an empty
results root resumes from the store's `state.json`.
- _Requirements: 2.1, 3.1_
- [ ] 2.5 Kind proof and record 02
- [x] 2.5 Kind proof and record 02
- `ops/ax/smoke-run.sh` with `OUSAST_DELIVERY=store`: `facts.json` and `summary.json` arrive through the store;
the actor's policy read back has one TLS rule naming the store host; `ousast plane status ax-e2e-smoke`
prints the table. Instrument: the tar's byte size from the store equals the extracted files' total.
benchmarks/measurements/2026-09-08-module-audit.json
+27 -2
View File
@@ -1,7 +1,7 @@
{
"modules": 155,
"modules": 157,
"counts": {
"load_bearing": 150,
"load_bearing": 152,
"standalone_capability": 5
},
"orphaned": [],
@@ -855,6 +855,8 @@
"learn.examples",
"learn.experiments",
"learn.program",
"plane.delivery",
"plane.doctor",
"plane.egress",
"plane.engine_alerts",
"plane.generate",
@@ -885,6 +887,14 @@
"plane.tasks.verify"
]
},
{
"module": "plane.delivery",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"plane.reconciler"
]
},
{
"module": "plane.doctor",
"classification": "load_bearing",
@@ -932,6 +942,7 @@
"reason": "ai-service-plane Req 1.1-1.4: ax-shaped Task/Workspace/Model and the Run kind, validated before anything runs",
"importers": [
"learn.cli",
"plane.delivery",
"plane.egress",
"plane.generate",
"plane.harvest",
@@ -954,11 +965,25 @@
"learn.examples",
"learn.experiments",
"learn.program",
"plane.delivery",
"plane.doctor",
"plane.generate",
"plane.reconciler",
"plane.tasks.loop",
"plane.tasks.remember"
]
},
{
"module": "plane.profile",
"classification": "load_bearing",
"reason": "python -m openultrasast.plane.profile",
"importers": [
"cli",
"plane.doctor",
"plane.generate",
"plane.reconciler"
]
},
{
"module": "plane.reconciler",
"classification": "load_bearing",
benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-k3s.txt
+5
View File
@@ -0,0 +1,5 @@
[ok] profile k3s: exec=remote kube_context=ousast-k3s registry=ghcr.io/norandom images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=ghcr-pull
[FAIL] cluster ousast-k3s: context ousast-k3s unreachable (error: context "ousast-k3s" does not exist); docs/deployment.md lists the prerequisites
[FAIL] runner image in ghcr.io/norandom: images file /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json: No such file or directory; docs/deployment.md lists the prerequisites
[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
exit=1 seconds=1.1
benchmarks/measurements/2026-10-02-k8s-01-profile/doctor-kind.txt
+7
View File
@@ -0,0 +1,7 @@
[ok] profile kind: exec=local kube_context=kind-ousast registry=localhost:5001 images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)
[ok] cluster kind-ousast: reachable
[ok] agent substrate (ate-system): 9 pods ready
[ok] ax controller (ax-system): 4 pods ready
[ok] runner image in localhost:5001: ousast-runner@sha256:95edc7ed5621... present in localhost:5001
[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
exit=0 seconds=1.4
benchmarks/measurements/2026-10-02-k8s-01-profile/literal-reinserted-fail.txt
+19
View File
@@ -0,0 +1,19 @@
F.. [100%]
=================================== FAILURES ===================================
________ test_no_host_literal_outside_the_profile_and_the_kind_bring_up ________
def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None:
kind = load_profile("kind", environ={})
found = offenders(_files(), kind.registry)
> assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found)
E AssertionError: host-only literals (Req 1.3):
E src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration
E assert ['src/openult...emonstration'] == []
E
E Left contains one more item: 'src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration'
E Use -v to get more diff
tests/test_plane_literals.py:81: AssertionError
=========================== short test summary info ============================
FAILED tests/test_plane_literals.py::test_no_host_literal_outside_the_profile_and_the_kind_bring_up
1 failed, 2 passed in 0.26s
benchmarks/measurements/2026-10-02-k8s-01-profile/literal-restored-pass.txt
+2
View File
@@ -0,0 +1,2 @@
... [100%]
3 passed in 0.23s
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-big.yaml
+16
View File
@@ -0,0 +1,16 @@
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: ousast-probe-big-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: ousast-probe-big, atespace: default}
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "2560Mi"}, limits: {cpu: "1", memory: "3Gi"}}
workspaces: [{name: ousast-probe-big-ws, path: "/workspace"}]
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-small.yaml
+16
View File
@@ -0,0 +1,16 @@
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: ousast-probe-small-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: ousast-probe-small, atespace: default}
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "256Mi"}, limits: {cpu: "1", memory: "512Mi"}}
workspaces: [{name: ousast-probe-small-ws, path: "/workspace"}]
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/probe-pool.yaml
+26
View File
@@ -0,0 +1,26 @@
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
name: probe-pool
namespace: ax-system
labels:
workload: ousast
spec:
replicas: 1
workerImage: localhost:5001/ateom-gvisor-715889664656de67e44382a8d6ab981d@sha256:220f1cf13ea485b73a2e4270fa544901a18ae826d8f22bd54d42761944a41c5c
template:
nodeSelector:
ate.dev/substrate-version: "4f9c293"
resources:
limits:
cpu: "2"
memory: 3Gi
requests:
cpu: 250m
memory: 3Gi
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-before.json
+78
View File
@@ -0,0 +1,78 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T05:24:59.069304031Z",
"version": "1758"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:29:30.551668391Z",
"version": "2"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-big.json
+81
View File
@@ -0,0 +1,81 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T18:29:45.281958347Z",
"version": "1761"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"allocated": {
"actors": 1
},
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:29:30.551668391Z",
"version": "2"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-small.json
+81
View File
@@ -0,0 +1,81 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T18:29:53.001457323Z",
"version": "1763"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"allocated": {
"actors": 1
},
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:30:04.825058843Z",
"version": "4"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
benchmarks/measurements/2026-10-02-k8s-01-profile/record.json
+132
View File
@@ -0,0 +1,132 @@
{
"record": "k8s-01-profile",
"spec": "plane-on-kubernetes",
"tasks": ["1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8"],
"date": "2026-10-02",
"host": "the maintainer's laptop: 7.9 GiB, single-node kind cluster, RustFS store reachable over https",
"commits": {
"1.1": "8eac1b2",
"1.2": "77a03ba",
"1.3": "36f5927",
"1.4": "8099b29",
"1.5": "86ec237",
"1.6": "2afa20e",
"1.7": "2cca03c",
"1.8": "the commit that adds this line (git log: plane(1.8))"
},
"images": {
"runner_before": "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884",
"runner_after_up_sh": "localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb",
"pin_file": "ops/k8s/profiles/kind-images.json (written by ops/ax/up.sh; the templates re-pinned to it, 128 image lines)"
},
"instrument_checks": {
"code_under_test": "PYTHONPATH=<worktree>/src first; `python -c 'import openultrasast; print(openultrasast.__file__)'` printed the worktree path before every gate",
"store_reached": {
"how": "scratch store_probe.py: open_store('s3://sast-memory') through the worktree code (verify_bucket runs), then read the probe object",
"open_and_verify_seconds": 0.62,
"probe_key": "_probe/select.jsonl",
"probe_bytes": 24,
"probe_version_id_present": true
},
"smoke_delivery_sizes_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291, "task.yaml": 1148},
"registry_resolution": "doctor HEADs /v2/ousast-runner/manifests/<digest> on the kind registry: HTTP 2xx for the new digest"
},
"doctor": {
"kind": {
"command": "ousast plane doctor --profile kind",
"exit": 0,
"seconds": 1.4,
"lines": [
"[ok] profile kind: exec=local kube_context=... registry=... images=... router_url=(tunnel) ax_server=(tunnel) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)",
"[ok] cluster <kind context>: reachable",
"[ok] agent substrate (ate-system): 9 pods ready",
"[ok] ax controller (ax-system): 4 pods ready",
"[ok] runner image in <kind registry>: ousast-runner@sha256:95edc7ed5621... present",
"[ok] memory store s3://sast-memory: opens (an S3 bucket is verified)"
],
"file": "doctor-kind.txt"
},
"k3s": {
"command": "ousast plane doctor --profile k3s",
"exit": 1,
"seconds": 1.1,
"failing_lines": [
"cluster ousast-k3s: context does not exist (expected: the cluster does not exist yet)",
"runner image in ghcr.io/norandom: images file ops/k8s/profiles/images.json missing (expected: the release asset is published by task 4.1)"
],
"passing_lines": ["profile k3s: every address reported", "memory store s3://sast-memory: opens"],
"note": "two expected failures, not one: the images file is the release asset and does not exist before group 4; the namespaces are not checked behind an unreachable context, so no third line fails for the same cause",
"file": "doctor-k3s.txt"
}
},
"literal_scan": {
"test": "tests/test_plane_literals.py",
"scope": "src/, plane/, ops/ (minus ops/ax/up.sh, ops/k8s/profiles/), docs/",
"patterns": ["kind context", "kind registry", "kind gateway 172.19.", "loopback with port", "port-forward"],
"exemptions": [
"router.py for loopback-with-port and port-forward",
"lines tagged ax-tunnel or loopback (two: the runner's own health URL, the Joern session's query URL)",
"digest-pinned image: lines under plane/tasks/ whose registry equals the kind profile's (a third test asserts every one does)"
],
"offenders_at_1_3": 0,
"offenders_before_1_3": 23,
"reinsertion_demo": {
"inserted": "a kind-context literal appended to src/openultrasast/plane/workspaces.py",
"pytest_exit_with_literal": 1,
"named": "src/openultrasast/plane/workspaces.py:37: [kind context]",
"restored": "byte for byte (cmp)",
"pytest_exit_restored": 0,
"files": ["literal-reinserted-fail.txt", "literal-restored-pass.txt"]
}
},
"up_sh_rerun": {
"command": "OUSAST_PYTHON=<main venv python> ops/ax/up.sh",
"exit": 0,
"seconds": 38,
"kept": ["cluster", "substrate", "egress gateway", "ax control plane", "worker pool ousast-pool (2/2 ready)", "ax CLI (newer than the checkout)"],
"done": ["receiver Service re-applied", "runner image rebuilt and pushed (new digest above)", "kind-images.json written", "smoke Task applied and resumed"],
"first_attempt_failure": "`profile --print images` answered `no image named ''`; fixed in profile._lookup (an images path with no sub-key is the file path) and covered by test_plane_profile",
"disk_gb_free_before_after": [1.7, 1.5],
"file": "up-sh-rerun.txt"
},
"smoke_run_1_4": {
"command": "ops/ax/smoke-run.sh (receiver delivery: the path group 2 replaces)",
"exit": 0,
"seconds": 24,
"status": "done",
"delivered": "facts.json 451 B, summary.json 238 B, units.jsonl 291 B; summary.image = the new digest, units 2/2",
"file": "smoke-run-1.4.txt"
},
"placement_probe_1_5": {
"script": "ops/ax/probe-placement.sh",
"pools": {"ousast-pool": "scaled to 1 worker, 1 CPU / 1536Mi", "probe-pool": "1 worker, 2 CPU / 3Gi, the already-built ateom-gvisor image (no ko build)"},
"big_actor": {"requests_memory": "2560Mi", "limits_memory": "3Gi", "landed_on": "ousast-pool (1536Mi worker): status.allocated.actors=1; probe-pool 0"},
"small_actor": {"requests_memory": "256Mi", "limits_memory": "512Mi", "landed_on": "ousast-pool: allocated.actors=1; probe-pool 0"},
"placement": "any",
"consequence": "design.md section 4: group 5 takes the fallback (an engine atespace with engine-pool as its only pool if Substrate scopes pools by atespace; else a plain Kubernetes Job on k3s and alerts-engine on kind)",
"resume_latency_seconds": "first resume of each new template succeeded at once (golden snapshot built within the resume deadline); whole probe 46 s",
"cleanup": "probe tasks deleted, probe-pool deleted, ousast-pool back to 2/2 (verified by kubectl get workerpool)",
"files": ["probe/probe.log", "probe/workers-before.json", "probe/workers-ousast-probe-big.json", "probe/workers-ousast-probe-small.json", "probe/ousast-probe-big.yaml", "probe/ousast-probe-small.yaml", "probe/probe-pool.yaml"]
},
"reconciler_bound_1_6": {
"lines_at_spec_commit_6bd44ca": 477,
"lines_after_1_2": 489,
"added_by_1_2": "the profile parameter of run() and its docstring (+12)",
"estimate_2_4": "Delivery wiring: open the store, build Delivery, pass the delivery body to the starter, delivered() and collect() in the await/execute path, state mirroring, the OUSAST_DELIVERY=receiver switch: about +25 while the receiver is kept (group 2 keeps both paths), about -20 when 2.6 removes the receiver wiring",
"projected_during_group_2": 514,
"projected_after_2_6": 494,
"decision": "the estimate crosses 500 while both paths coexist: task 2.4 moves the bound in tests/test_plane_reconciler.py to 550 with the reason in the assertion message, and 2.6 records the final count"
},
"gvisor_tarball_1_7": {
"status": "planned, manual",
"where": "docs/deployment.md B.1",
"url": "gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd (sha256-pinned in substrate/manifests/ate-install/sandboxconfig-gvisor.yaml)",
"check": "the atelet's fetch line in `kubectl -n ate-system logs` of a worker pod on the k3s node; labelled planned until read there"
},
"gates": {
"1.1": "pytest 2002 passed + module-audit manifest regenerated (test_module_audit 6 passed), ruff 0, format 0, mypy 0",
"1.2": "pytest 2006 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"1.3": "pytest 2009 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0; mkdocs --strict 0",
"1.4": "see the commit message"
}
}
benchmarks/measurements/2026-10-02-k8s-01-profile/smoke-run-1.4.txt
+9
View File
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - 0 n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=24
benchmarks/measurements/2026-10-02-k8s-01-profile/up-sh-rerun.txt
+58
View File
@@ -0,0 +1,58 @@
== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (20:35:35)
cluster exists; kept
== agent substrate (ate-system) (20:35:35)
substrate ready; kept
NAME READY STATUS RESTARTS AGE
ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (12h ago) 3d
ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (12h ago) 3d
ate-controller-c95586684-g7nvn 1/1 Running 0 3d2h
atelet-4f9c293-9zpfg 1/1 Running 0 3d2h
atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d
atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (12h ago) 3d2h
postgres-0 2/2 Running 3 (3d2h ago) 3d2h
rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d2h ago) 3d2h
rustfs-bucket-init-nvfr6 0/1 Completed 0 3d2h
== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (20:35:35)
gateway present; kept
== ax control plane (ax-system) (20:35:35)
ax ready; kept
NAME READY STATUS RESTARTS AGE
ax-redis-86df786c87-pz5tp 1/1 Running 0 3d2h
ax-server-b479bdc76-hwkqb 1/1 Running 0 3d2h
ousast-pool-6656b55f5b-5brkn 1/1 Running 0 5m23s
ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d2h
== artifact receiver Service (ousast-receiver.ax-system -> this host) (20:35:35)
service/ousast-receiver unchanged
endpointslice.discovery.k8s.io/ousast-receiver-host configured
== worker pool (gVisor): pools.default of the profile (20:35:36)
pool ousast-pool exists (2/2 ready); kept
pod/ousast-pool-6656b55f5b-5brkn condition met
pod/ousast-pool-6656b55f5b-9wbrw condition met
== runner image localhost:5001/ousast-runner (20:35:37)
sha256:f004a5ef8f71b1cb8a5880c509389c0569181af7b19170b574958cc3ef75cb1c
localhost:5001/ousast-runner:dev
runner image localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
== ax CLI from the deployed checkout (a release CLI skews from the server) (20:36:05)
ax CLI newer than the checkout; kept
== smoke task (20:36:05)
task.ax.io/ousast-smoke created
workspace.ax.io/ousast-smoke-ws unchanged
task.ax.io/ousast-smoke resumed
NAME ATESPACE PHASE ACTOR WORKER-IP AGE
ousast-smoke default Running ousast-smoke 10.244.0.29 5s
== footprint (20:36:11)
total used free shared buff/cache available
Mem: 7877 4797 225 63 3464 3079
kind-registry 53.77MiB / 7.693GiB 0.00%
up.sh exit 0
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/egress-policy-read-back.json
+24
View File
@@ -0,0 +1,24 @@
{
"metadata": {
"atespace": "default",
"createTime": "2026-10-02T19:39:19.428045847Z",
"name": "default",
"uid": "741e52b7-aaf1-4a91-b4bf-fc3b20ba4a36",
"updateTime": "2026-10-02T19:39:19.428045847Z",
"version": "1"
},
"rules": [
{
"tlsPassthrough": {
"hostnames": [
"files.because-security.com"
],
"ports": {
"numbers": [
443
]
}
}
}
]
}
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json
+83
View File
@@ -0,0 +1,83 @@
{
"record": "k8s-02-store-delivery",
"spec": "plane-on-kubernetes",
"tasks": ["2.1", "2.2", "2.3", "2.4", "2.5", "2.6"],
"date": "2026-10-02",
"commits": {
"2.1": "35d8032",
"2.2": "9506a18",
"2.3": "c49eae4",
"2.4": "605b7af",
"2.5": "the commit that adds this file (git log: plane(2.5))",
"receiver_removed": "pending: task 2.6 (paused 2026-10-03 by the maintainer's reprioritisation; the receiver path still exists behind OUSAST_DELIVERY=receiver)"
},
"images": {
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019",
"built_by": "ops/ax/up.sh rerun (file up-sh-rebuild.txt): every step kept but the runner build; the image carries the 2.2 runner",
"disk_gb_free_after_build": 1.3
},
"instrument_checks": {
"code_under_test": "the worktree's src first on PYTHONPATH; `ousast` run as `python -m openultrasast.cli` from the worktree (ops/ax/smoke-run.sh with OUSAST_PYTHON)",
"store_read_back": {
"how": "scratch store_tar_check.py opens the store through the worktree code (verify_bucket runs), lists runs/ax-e2e-smoke/, downloads output.tar, compares its members with the files the reconciler extracted (file store-check.json)",
"tar_bytes": 10240,
"tar_members_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
"extracted_files_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
"members_equal_extracted": true,
"member_bytes_total": 980,
"note": "a tar is 512-byte blocks: 10240 B of tar carry 980 B of files; done.json records the tar's byte size"
},
"no_receiver_in_the_task": "the rendered task.yaml is 996 B against 1148 B on the receiver path: no OUSAST_ARTIFACT_URL/DIAL in the Task's env; the start body carries the URLs"
},
"smoke_runs": {
"store_1": {
"command": "ops/ax/smoke-run.sh (store delivery, the default after 2.4; OUSAST_DELIVERY unset)",
"exit": 0,
"seconds_submit_to_done": 26,
"tar_put_at": "2026-10-02T19:37:24+00:00 (the object's LastModified)",
"collected_at": "2026-10-02T19:37:26+00:00 (done.json finished)",
"put_to_collect_seconds": 2,
"poll_interval_seconds": 3,
"file": "smoke-run-store-1.txt"
},
"seeded_rerun": {
"command": "the same, with the local results root deleted and the store's runs/ax-e2e-smoke/ intact",
"exit": 0,
"seconds": 2,
"what_happened": "state.json seeded from the store: every task done, nothing submitted to ax (plane-on-kubernetes 2.1, design section 2: a rerun from another machine resumes from the store)",
"file": "smoke-run-seeded-rerun.txt"
},
"store_2": {
"command": "the same, after deleting runs/ax-e2e-smoke/ from the store (5 objects) so the task executes again, with the egress policy polled",
"exit": 0,
"seconds_submit_to_done": 24,
"tar_put_at": "2026-10-02T19:39:38+00:00",
"collected_at": "2026-10-02T19:39:40+00:00",
"put_to_collect_seconds": 2,
"file": "smoke-run-store-2.txt"
}
},
"objects_delivered_to_the_store": {
"runs/ax-e2e-smoke/facts/output.tar": 10240,
"runs/ax-e2e-smoke/facts/facts.json": 451,
"runs/ax-e2e-smoke/facts/summary.json": 238,
"runs/ax-e2e-smoke/facts/done.json": 149,
"runs/ax-e2e-smoke/state.json": 350,
"note": "units.jsonl is extracted locally but not a declared output of the smoke Run, so it is not re-put"
},
"egress_policy_read_back": {
"how": "kubectl ate get egress-policy ax-e2e-smoke-facts -a default -o json, polled while the actor lived (file egress-policy-read-back.json)",
"rules": [{"tlsPassthrough": {"hostnames": ["files.because-security.com"], "ports": {"numbers": [443]}}}],
"reading": "one TLS rule naming the store host, no http rule; the model-free smoke task names no Git host and no Model host",
"also": "Egress.apply reads every policy back and raises on a mismatch, so each completed Run is itself the equality check"
},
"status_table": "ousast plane status ax-e2e-smoke printed the attribution table (facts: done, calls 0) in every run; the table is in the smoke-run-*.txt files",
"reconciler_lines": {"after_2_4": 549, "bound": 550, "note": "both paths coexist until 2.6; the receiver wiring goes then"},
"gates": {
"2.1": "pytest 2020 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.2": "pytest 2024 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.3": "pytest 2027 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.4": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.5": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0 (first attempt 2026-10-02: 11 failed + 5 errors, all the FileStore 1 GiB free-space refusal at 0.97 GiB free; rerun after the disk was freed)"
}
}
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-seeded-rerun.txt
+9
View File
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - n/a n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=2
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-1.txt
+9
View File
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - 0 n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=26
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/smoke-run-store-2.txt
+9
View File
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - 0 n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=24
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/store-check.json
+36
View File
@@ -0,0 +1,36 @@
{
"store": "s3://sast-memory",
"objects_under_run": {
"runs/ax-e2e-smoke/facts/done.json": 149,
"runs/ax-e2e-smoke/facts/facts.json": 451,
"runs/ax-e2e-smoke/facts/output.tar": 10240,
"runs/ax-e2e-smoke/facts/summary.json": 238,
"runs/ax-e2e-smoke/state.json": 350
},
"tar_bytes": 10240,
"tar_members": {
"facts.json": 451,
"summary.json": 238,
"units.jsonl": 291
},
"extracted_files": {
"facts.json": 451,
"units.jsonl": 291,
"summary.json": 238
},
"members_equal_extracted": true,
"member_bytes_total": 980,
"extracted_bytes_total": 980,
"done_json": {
"bytes": 10240,
"files": 3,
"finished": "2026-10-02T19:39:40+00:00",
"published": [
"facts.json",
"summary.json"
],
"status": "done",
"task": "facts"
},
"tar_put_at_last_modified": "2026-10-02T19:39:38+00:00"
}
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/up-sh-rebuild.txt
+58
View File
@@ -0,0 +1,58 @@
== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (21:36:18)
cluster exists; kept
== agent substrate (ate-system) (21:36:18)
substrate ready; kept
NAME READY STATUS RESTARTS AGE
ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (13h ago) 3d1h
ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (13h ago) 3d1h
ate-controller-c95586684-g7nvn 1/1 Running 0 3d3h
atelet-4f9c293-9zpfg 1/1 Running 0 3d3h
atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d1h
atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (13h ago) 3d3h
postgres-0 2/2 Running 3 (3d3h ago) 3d3h
rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d3h ago) 3d3h
rustfs-bucket-init-nvfr6 0/1 Completed 0 3d3h
== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (21:36:18)
gateway present; kept
== ax control plane (ax-system) (21:36:19)
ax ready; kept
NAME READY STATUS RESTARTS AGE
ax-redis-86df786c87-pz5tp 1/1 Running 0 3d3h
ax-server-b479bdc76-hwkqb 1/1 Running 0 3d3h
ousast-pool-6656b55f5b-5brkn 1/1 Running 0 66m
ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d3h
== artifact receiver Service (ousast-receiver.ax-system -> this host) (21:36:19)
service/ousast-receiver unchanged
endpointslice.discovery.k8s.io/ousast-receiver-host configured
== worker pool (gVisor): pools.default of the profile (21:36:20)
pool ousast-pool exists (2/2 ready); kept
pod/ousast-pool-6656b55f5b-5brkn condition met
pod/ousast-pool-6656b55f5b-9wbrw condition met
== runner image localhost:5001/ousast-runner (21:36:21)
sha256:0501baf180b29f7aa7bbb8888bebabfc5d62429b7bdaa7f5a62b5b188f8ad5f6
localhost:5001/ousast-runner:dev
runner image localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
== ax CLI from the deployed checkout (a release CLI skews from the server) (21:36:32)
ax CLI newer than the checkout; kept
== smoke task (21:36:32)
task.ax.io/ousast-smoke created
workspace.ax.io/ousast-smoke-ws unchanged
task.ax.io/ousast-smoke resumed
NAME ATESPACE PHASE ACTOR WORKER-IP AGE
ousast-smoke default Running ousast-smoke 10.244.0.29 4s
== footprint (21:36:38)
total used free shared buff/cache available
Mem: 7877 5202 382 63 2903 2674
kind-registry 32.45MiB / 7.693GiB 0.00%
up.sh exit 0
docs/deployment.md
+44 -39
View File
@@ -31,11 +31,11 @@ flowchart LR
sub["Agent Substrate (ate-system): atenet-router, atenet-egress"]
act["Task actors in gVisor workers (WorkerPool ousast-pool)"]
svc["Service ousast-receiver + EndpointSlice to the host"]
reg["Registry localhost:5001: runner image by digest"]
reg["Registry named by the profile: runner image by digest"]
end
mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")]
rec -- "ax apply, resume, get, delete" --> ax
rec -- "EgressPolicy per actor; start request via port-forward" --> sub
rec -- "EgressPolicy per actor; start request through the kube-context tunnel" --> sub
sub --> act
act -- "HTTP by Service name through the gateway" --> svc
svc --> rcv
@@ -50,8 +50,8 @@ flowchart LR
| How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` |
| ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` |
| Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` |
| Runner image | the kind-local registry `localhost:5001`, pinned by digest (`~/.cache/ousast/ax-src/runner-image`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through a `kubectl port-forward` to `svc/atenet-router` | `src/openultrasast/plane/router.py` |
| Runner image | the registry the profile names (`registry` in `ops/k8s/profiles/kind.toml`: the kind-local one), pinned by digest in the profile's images file (`ops/k8s/profiles/kind-images.json`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through the tunnel `router.py` opens to `svc/atenet-router` in the profile's kube context | `src/openultrasast/plane/router.py` |
| Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://<bucket>[/<prefix>]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) |
| Run state and artifacts | `$OUSAST_RESULTS/plane/<run>/` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) |
@@ -80,7 +80,7 @@ files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
The script is idempotent: it skips each step that is already done. In order, it:
- creates the kind cluster (`KIND_CLUSTER_NAME`, default `ousast`) with its registry
(`KO_DOCKER_REPO`, default `localhost:5001`);
(`KO_DOCKER_REPO`, default: the `registry` of `ops/k8s/profiles/kind.toml`);
- installs Agent Substrate;
- applies the egress gateway if the install left it out (the agentgateway variant, one prebuilt image);
- deploys the ax control plane;
@@ -95,17 +95,20 @@ files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
2. **Check it.**
```bash
uv run ousast plane doctor
uv run ousast plane doctor --profile kind
```
It runs four checks:
It checks the profile you pass (`--profile`, default `kind`; the profiles are in `ops/k8s/profiles/`):
- the kind context is reachable;
- its configured addresses, printed on the first line;
- the kube context is reachable;
- every pod in `ate-system` is ready;
- every pod in `ax-system` is ready;
- the registry at `localhost:5001` holds `ousast-runner`.
- the runner image of the profile's images file is in its registry (a local registry over its v2
API, a remote one through `docker manifest inspect`);
- the memory store opens (an S3 bucket is verified).
Each failing line names `ops/ax/up.sh` as the fix. `doctor` is informational only.
Each failing line names the fix (`ops/ax/up.sh` for the kind profile). `doctor` is informational:
`ousast plane run` does not call it.
3. **Run the end-to-end smoke Run.**
@@ -165,8 +168,8 @@ The target cluster would do four things:
- keep its memory in your S3 bucket.
The provider key already travels only in the start request through `atenet-router`. That works the same
through a port-forward to any cluster. Everything else below is either a step you take or a code change
that is not made yet.
through the kube-context tunnel to any cluster. Everything else below is either a step you take or a code
change that is not made yet.
### B.1 Cluster with Agent Substrate, its egress gateway and ax
@@ -186,6 +189,14 @@ that is not made yet.
- TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
- Deploy ax (`make deploy AX_IMAGE_REPO=<your registry>` in the ax checkout) **after** B.2.
- Apply a WorkerPool (B.6). Substrate installs none by itself ("no free workers").
- **gVisor on the nodes (planned, checked by hand on the day k3s exists).** Substrate does not use a Kubernetes
`RuntimeClass`: the `ateom-gvisor` worker runs `runsc` itself, from the gVisor release tarball named in
`manifests/ate-install/sandboxconfig-gvisor.yaml` of the Substrate checkout
(`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd`, sha256-pinned per architecture), which
the atelet fetches on the worker node. The node must reach that URL, or a copied `SandboxConfig` must name a
mirrored copy with the same sha256 and the ActorTemplates must name that config. The check is the atelet's
fetch line in `kubectl -n ate-system logs` of the worker pod; it is labelled planned until that log has been
read on a k3s node (plane-on-kubernetes task 1.7, design open question 3).
### B.2 ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET`
@@ -218,31 +229,23 @@ long Runs need a janitor that removes directories no live actor owns.
- Pass `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`. They re-pin the
Tasks they generate.
- The committed templates under `plane/tasks/` and the generated `plane/tasks/validation-46.yaml` carry
`localhost:5001/ousast-runner@sha256:...`. They need the same re-pin before they run elsewhere.
the kind profile's registry and digest. They need the same re-pin before they run elsewhere.
Regenerate them, or replace the reference.
- Workers must be able to pull from the registry. Credentials, if any, are the cluster's concern. ax's
default runner image on `gcr.io` needs credentials and is not used here.
- **Not yet configurable:** `ousast plane doctor` checks the registry at `localhost:5001` by name
(`src/openultrasast/plane/doctor.py:45`). So its fourth check fails on another cluster, even when the
image is in place. `doctor` is informational; `plane run` does not depend on it.
- `ousast plane doctor --profile <name>` looks up the runner reference of the profile's images file in
the profile's `registry` (implemented 2026-10-02, plane-on-kubernetes task 1.2). `doctor` is
informational; `plane run` does not depend on it.
### B.4 One configurable kube context
### B.4 One configurable kube context (implemented 2026-10-02)
The reconciler, the egress client and the router port-forward all use one kube context. Its name is
`kind-$KIND_CLUSTER_NAME` (`kind-ousast` by default).
The reconciler, the egress client and the router tunnel take the kube context from the profile.
**Not yet configurable**: the prefix `kind-` is hard-coded in these places:
- `src/openultrasast/plane/doctor.py:39`;
- `src/openultrasast/plane/egress.py:115` and `:148`;
- `src/openultrasast/plane/router.py:145`;
- `ops/ax/up.sh:24` builds the same name.
The planned change is one variable read in those three modules, for example `OUSAST_KUBE_CONTEXT`.
Until then, there is one workaround to drive another cluster with the unchanged code. Rename its
kubeconfig context to that form: `kubectl config rename-context <ctx> kind-<name>` and
`KIND_CLUSTER_NAME=<name>`. This is a workaround, not a supported configuration.
- The setting is `kube_context` in `ops/k8s/profiles/<name>.toml`; `OUSAST_KUBE_CONTEXT` overrides it
(plane-on-kubernetes task 1.2).
- `doctor.py`, `egress.py` and `router.py` are handed the context. They build none themselves.
- `tests/test_plane_literals.py` fails on any module, manifest or page that names this host's cluster,
registry or addresses.
The egress client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names). It must be
able to use that context.
@@ -333,10 +336,10 @@ Running the reconciler in-cluster is itself planned. Today it is a host process.
| Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented |
| S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) |
| Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented |
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates still name `localhost:5001` |
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates name the kind profile's registry (`tests/test_plane_literals.py` checks it) |
| Deployment to a cluster other than this host's kind cluster | **not done** |
| One configurable kube context | **planned**; `kind-` hard-coded in `doctor.py`, `egress.py`, `router.py` |
| `doctor` registry check against a configurable registry | **planned**; `localhost:5001` hard-coded in `doctor.py:45` |
| One configurable kube context | implemented (2026-10-02): `kube_context` in the profile, `OUSAST_KUBE_CONTEXT` overrides |
| `doctor` registry check against a configurable registry | implemented (2026-10-02): the profile's `registry` and images file |
| Receiver as an in-cluster Deployment | **planned** (code change) |
| Presigned-URL delivery to the S3 store | **follow-on**, not started |
| Reconciler running in-cluster with Secrets injected into its environment | **planned** |
@@ -373,7 +376,7 @@ flowchart LR
ctl["ate-api-server, ate-controller, atelet"]
rt["atenet-router"]
eg["atenet-egress gateway"]
wp["WorkerPool: gVisor workers, runsc RuntimeClass, HPA (planned)"]
wp["WorkerPool: gVisor workers, runsc in the worker, HPA (planned)"]
ewp["Engine WorkerPool: Joern as a task (planned)"]
end
end
@@ -410,7 +413,7 @@ profile only. It does not appear in production.
- It is a Deployment `ax-server` with a Redis beside it, in the namespace `ax-system`
(`ax/deploy/ax-server.yaml` in the ax checkout).
- Its controller runs with `--template=default-template --template-atespace=ax-system`.
- The `ax` CLI reaches it through a `kubectl port-forward` to `svc/ax-server`
- The `ax` CLI reaches it through a tunnel to `svc/ax-server` over the kube context
(`ax/internal/tunnel/tunnel.go`).
- So `ousast plane run` needs a kube context or an ingress, not a public endpoint.
@@ -440,14 +443,16 @@ profile only. It does not appear in production.
- k3s runs containerd with no Docker socket. So the host-Docker path of `ousast plane alerts-engine`
does not exist there. The engine must become a task (below).
- Substrate's gVisor workers need `runsc` and a `RuntimeClass` on the nodes they run on.
- Substrate's gVisor workers need no Kubernetes `RuntimeClass`: the worker fetches a pinned gVisor
tarball and runs `runsc` inside its own pod. The node checklist is pod security, tarball
reachability and local-path storage (B.1).
- k3s ships Traefik and local-path storage. Traefik is an ingress the CLI can use instead of a kubeconfig.
- GitHub Actions builds the images from tagged commits. They are published to GitHub Container Registry
and digest-pinned in the manifests.
- A private package needs an `imagePullSecret` on the worker ServiceAccount.
- Substrate's and ax's own images are built with `ko` from their checkouts. They are pushed to the same
registry.
- The kind profile keeps `localhost:5001` and `ops/ax/up.sh`.
- The kind profile keeps its local registry (`ops/k8s/profiles/kind.toml`) and `ops/ax/up.sh`.
### One image, and what concurrency equals
@@ -486,7 +491,7 @@ Substrate ships two scaling aids:
| Component | Today (the kind profile) | Production (the k3s profile, planned) |
| --- | --- | --- |
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against `kind-ousast`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against the profile's `kube_context`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
| Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction |
| Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path |
ops/ax/README.md
+9 -6
View File
@@ -7,12 +7,15 @@ single-node kind cluster. ax is the only executor; `ousast plane run` submits, r
ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway,
# receiver Service, runner image (digest pinned), ax CLI, smoke Task
ousast plane doctor # kind, Substrate, ax controller, runner image
ousast plane doctor # the profile's cluster (ops/k8s/profiles/kind.toml), Substrate, ax, runner image, memory store
ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table
ops/ax/down.sh # deletes the cluster and its registry
Tools live in `~/go/bin` (kind, ax, ko, kubectl-ate) and `~/.local/bin` (kubectl); sources and rendered files
in `~/.cache/ousast/ax-src/` (the runner digest pin is `runner-image` there).
in `~/.cache/ousast/ax-src/`. The cluster's context, registry and pool sizes come from the kind profile
(`ops/k8s/profiles/kind.toml`); `up.sh` writes the runner's digest pin to that profile's images file
(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them.
`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`).
## How a Run executes
@@ -192,16 +195,16 @@ assume this laptop and must change first (checked 2026-10-02):
| Assumption | Where | Needed in a real cluster |
|---|---|---|
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind gateway `172.19.0.1:18090` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
| Registry `localhost:5001`, rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), `doctor.py:45` (hard-coded check), the runner digest pin, the `image` of every template under `plane/tasks/` | a registry the workers can pull from; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
| kubectl context `kind-$KIND_CLUSTER_NAME` | `doctor.py:39`, `egress.py:115` and `:148`, `router.py:145`, `up.sh:24` | one configurable context (for example `OUSAST_KUBE_CONTEXT`); not yet configurable |
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind network's gateway address on `OUSAST_ARTIFACT_PORT` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
| The kind-local registry (`registry` in `ops/k8s/profiles/kind.toml`), rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), the profile's images file, the `image` of every template under `plane/tasks/` | the `k3s` profile's registry; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
| kubectl context | `kube_context` in the profile, read by `doctor.py`, `egress.py`, `router.py` (plane-on-kubernetes 1.2); `up.sh` reads the same profile | the `k3s` profile's context; nothing else changes |
| ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` in ax's `deploy/ax-server.yaml` points at the ax authors' GCS bucket | ax deploy manifest | your own bucket, set before deploying ax |
| Egress gateway applied by hand (agentgateway variant, no Rust build) | this README | the Substrate-installed gateway; per-task EgressPolicies work unchanged |
| Worker pool of 2 x 1 CPU / 1.5 GiB for the 7 GB host | `workerpool.yaml.tmpl` | sized to the cluster; `--workers` to match |
| Plane memory store and results under `~/ousast-results/` on the host | `reconciler.py` (`OUSAST_RESULTS`), `memory.py` (`OUSAST_MEMORY`) | a persistent volume or bucket shared by the reconciler; the `s3://` store already works against any reachable S3-compatible server |
The provider key already travels only in the start request through `atenet-router`, which works the same through
a port-forward to any cluster.
the kube-context tunnel to any cluster.
## What a larger ax deployment needs
ops/ax/probe-placement.sh
Executable
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# Placement probe (plane-on-kubernetes task 1.5, design open question 2): does Agent Substrate place an actor on a
# worker that fits its memory request, or on any free worker of the sandbox class?
#
# Applies a second WorkerPool `probe-pool` (1 replica, 2 CPU / 3Gi) from ops/ax/workerpool.yaml.tmpl with the worker
# image the existing pool already built (no ko build), scales the default pool to one worker so the 7 GB host holds
# both, then runs one model-free repo-facts Task asking 2560Mi (limit 3Gi) and one asking 256Mi, recording after
# each resume which worker holds the actor (`kubectl ate get workers -o json`). Everything it created is removed
# on exit and the default pool is scaled back. Usage: ops/ax/probe-placement.sh OUT_DIR (the JSON evidence lands there).
set -euo pipefail
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
OUT="${1:?usage: probe-placement.sh OUT_DIR}"
mkdir -p "$OUT"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
CTX="$(profile kube_context)"
RUNNER_IMAGE="$(profile images.runner)"
DEFAULT_REPLICAS="$(profile pools.default.replicas)"
K() { kubectl --context "$CTX" -n ax-system "$@"; }
log() { printf '%s %s\n' "$(date +%H:%M:%S)" "$*" | tee -a "$OUT/probe.log"; }
cleanup() {
rc=$?
log "cleanup (exit $rc): delete the probe tasks and probe-pool, scale ousast-pool back to $DEFAULT_REPLICAS"
for t in ousast-probe-big ousast-probe-small; do ax delete task "$t" >/dev/null 2>&1 || true; done
K delete workerpool probe-pool --ignore-not-found >/dev/null 2>&1 || true
K patch workerpool ousast-pool --type merge -p "{\"spec\":{\"replicas\":$DEFAULT_REPLICAS}}" >/dev/null 2>&1 || true
echo "probe exit $rc"
}
trap cleanup EXIT
workers() { kubectl ate --context "$CTX" get workers -o json > "$OUT/$1.json"; kubectl ate --context "$CTX" get workers | tee -a "$OUT/probe.log"; }
export POOL_NAME=probe-pool POOL_REPLICAS=1 POOL_CPU=2 POOL_MEMORY=3Gi
WORKER_IMAGE="$(K get workerpool ousast-pool -o jsonpath='{.spec.workerImage}')" # already built and pulled by the node
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
export WORKER_IMAGE SUBSTRATE_VERSION
log "default pool to 1 replica (room for a 3Gi worker on this host)"
K patch workerpool ousast-pool --type merge -p '{"spec":{"replicas":1}}'
for i in $(seq 1 60); do [ "$(K get pods -l ate.dev/worker-pool=ousast-pool --no-headers 2>/dev/null | wc -l)" = 1 ] && break; sleep 2; done
log "apply probe-pool: $POOL_REPLICAS x $POOL_CPU CPU / $POOL_MEMORY, image $WORKER_IMAGE"
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" | tee "$OUT/probe-pool.yaml" | K apply -f -
K wait --for=condition=Ready pod -l ate.dev/worker-pool=probe-pool --timeout=600s
workers workers-before
task() { # name, request memory, limit memory
cat > "$OUT/$1.yaml" <<EOF
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: $1-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: $1, atespace: default}
spec:
image: "$RUNNER_IMAGE"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "$2"}, limits: {cpu: "1", memory: "$3"}}
workspaces: [{name: $1-ws, path: "/workspace"}]
EOF
ax delete task "$1" >/dev/null 2>&1 || true
log "apply $1 (requests $2, limits $3)"
ax apply -f "$OUT/$1.yaml"
for i in 1 2 3 4 5 6 7 8 9; do
if ax resume task "$1" 2>&1 | tee -a "$OUT/probe.log"; then break; fi # the first resume may wait for the golden snapshot
sleep 20
done
sleep 15
ax get task "$1" | tee "$OUT/$1.status.yaml" | grep -E "phase|message" || true
workers "workers-$1"
ax delete task "$1" >/dev/null 2>&1 || true
for i in $(seq 1 30); do kubectl ate --context "$CTX" get workers | grep -q " 1/1000 " || break; sleep 2; done
}
task ousast-probe-big 2560Mi 3Gi
task ousast-probe-small 256Mi 512Mi
log "done; evidence under $OUT"
ops/ax/smoke-run.sh
+9 -5
View File
@@ -1,14 +1,18 @@
#!/usr/bin/env bash
# Render the end-to-end smoke Run with the current runner digest and execute it through the reconciler on ax.
# Render the end-to-end smoke Run with the profile's runner digest and execute it through the reconciler on ax.
# The profile (OUSAST_PLANE_PROFILE, default kind; ops/k8s/profiles/) names the images file up.sh wrote.
set -euo pipefail
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
RUNNER_IMAGE="$(cat "$SRC/runner-image")"; export RUNNER_IMAGE
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO/.venv/bin/python}"
RUNNER_IMAGE="$(PYTHONPATH="$REPO/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print images.runner)"
export RUNNER_IMAGE
# Actors reach the receiver by its Service name through the egress gateway (ops/ax/receiver-service.yaml.tmpl,
# applied by up.sh, endpoint this host at OUSAST_ARTIFACT_PORT): the runner dials the Service's ClusterIP on port
# 80 with that name as Host; the reconciler looks the ClusterIP up unless OUSAST_ARTIFACT_DIAL is set.
export OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}"
envsubst < "$(dirname "$0")/smoke-run.yaml.tmpl" > "$SRC/smoke-run.yaml"
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
"$REPO/.venv/bin/ousast" plane run "$SRC/smoke-run.yaml" "$@"
"$REPO/.venv/bin/ousast" plane status ax-e2e-smoke
ousast() { PYTHONPATH="$REPO/src" "$PY" -m openultrasast.cli "$@"; } # this checkout's code, whatever python runs it
ousast plane run "$SRC/smoke-run.yaml" "$@"
ousast plane status ax-e2e-smoke
ops/ax/up.sh
+30 -12
View File
@@ -1,13 +1,21 @@
#!/usr/bin/env bash
# Bring up ax on this host: kind cluster + local registry, Agent Substrate, the ax control plane, one smoke
# Task. Idempotent enough to rerun; `down.sh` removes everything. Requires docker, go, kubectl, kind, ko, ax.
# Task. Idempotent: every step is skipped when already done; `down.sh` removes everything. Requires docker, go,
# kubectl, kind, ko, ax. The cluster's context, the registry and the pool sizes come from the kind profile
# (ops/k8s/profiles/kind.toml; OUSAST_PLANE_PROFILE selects another file with exec = "local"). This script may
# name the kind cluster because it creates it; the code reads the profile (tests/test_plane_literals.py).
set -euo pipefail
trap 'rc=$?; echo "up.sh FAILED at line $LINENO (exit $rc)"' ERR
trap 'echo "up.sh exit $?"' EXIT
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
export GOTOOLCHAIN=auto
export KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-ousast}"
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-localhost:5001}"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
CTX="$(profile kube_context)" # kind names its context kind-<cluster>
export KIND_CLUSTER_NAME="${CTX#kind-}"
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}"
IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
for t in docker go kubectl kind ko ax; do
@@ -19,9 +27,8 @@ clone substrate https://github.com/agent-substrate/substrate.git
clone ax https://github.com/google/ax.git
step() { printf '\n== %s (%s)\n' "$1" "$(date +%H:%M:%S)"; }
step "kind cluster '$KIND_CLUSTER_NAME' + registry $KO_DOCKER_REPO"
step "kind cluster '$KIND_CLUSTER_NAME' (context $CTX) + registry $KO_DOCKER_REPO"
# Each step is skipped when already done: the substrate helper DELETES an existing cluster before creating one.
CTX="kind-$KIND_CLUSTER_NAME"
ready() { kubectl --context "$CTX" -n "$1" get pods --no-headers 2>/dev/null | awk '$2 ~ /^[0-9]+\/[0-9]+$/ {n++; split($2,a,"/"); if (a[1]!=a[2] && $3!="Completed") bad++} END {exit !(n>0 && bad==0)}'; }
if kind get clusters 2>/dev/null | grep -qx "$KIND_CLUSTER_NAME"; then
echo "cluster exists; kept"
@@ -56,25 +63,36 @@ KIND_GATEWAY="$(docker network inspect kind --format '{{range .IPAM.Config}}{{if
export KIND_GATEWAY OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}" # the reconciler's fixed receiver port
envsubst < "$(dirname "$0")/receiver-service.yaml.tmpl" | kubectl --context "$CTX" apply -f -
step "worker pool (gVisor)"
step "worker pool (gVisor): pools.default of the profile"
# `ko apply` builds the worker image from the Substrate checkout (several GB of Go caches): only when the pool
# is absent. A size change in the profile is applied with `kubectl patch` or by deleting the pool first.
export POOL_NAME=ousast-pool POOL_REPLICAS="$(profile pools.default.replicas)" POOL_CPU="$(profile pools.default.cpu)" POOL_MEMORY="$(profile pools.default.memory)"
export WORKER_IMAGE="ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor"
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
export SUBSTRATE_VERSION
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" > "$SRC/workerpool.yaml"
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l ate.dev/worker-pool=ousast-pool --timeout=600s \
if kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" >/dev/null 2>&1; then
echo "pool $POOL_NAME exists ($(kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" -o jsonpath='{.status.readyReplicas}/{.spec.replicas}') ready); kept"
else
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
fi
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l "ate.dev/worker-pool=$POOL_NAME" --timeout=600s \
|| kubectl --context "$CTX" -n ax-system get pods
step "runner image $KO_DOCKER_REPO/ousast-runner"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
docker build -q -f "$REPO_ROOT/plane/Dockerfile.runner" -t "$KO_DOCKER_REPO/ousast-runner:dev" "$REPO_ROOT"
docker push -q "$KO_DOCKER_REPO/ousast-runner:dev"
RUNNER_IMAGE="$(docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$KO_DOCKER_REPO/ousast-runner:dev" | grep "^$KO_DOCKER_REPO/ousast-runner@" | head -1)"
[ -n "$RUNNER_IMAGE" ] || { echo "no digest for the runner image" >&2; exit 2; }
export RUNNER_IMAGE; echo "$RUNNER_IMAGE" > "$SRC/runner-image" # the reconciler reads this pin
echo "runner image $RUNNER_IMAGE"
export RUNNER_IMAGE
printf '{\n "runner": "%s"\n}\n' "$RUNNER_IMAGE" > "$IMAGES_FILE" # the profile's images file: commit it with the templates
echo "runner image $RUNNER_IMAGE -> $IMAGES_FILE"
echo "re-pin the templates with: ousast plane workspaces ... --runner-image $IMAGES_FILE (or ousast plane repin, task 4.3)"
step "ax CLI from the deployed checkout (a release CLI skews from the server)"
(cd "$SRC/ax" && go install ./cmd/ax)
if [ -x "$HOME/go/bin/ax" ] && [ "$HOME/go/bin/ax" -nt "$SRC/ax/.git/HEAD" ]; then echo "ax CLI newer than the checkout; kept"; else
(cd "$SRC/ax" && go install ./cmd/ax)
fi
step "smoke task"
ax delete task ousast-smoke >/dev/null 2>&1 || true
ops/ax/workerpool.yaml.tmpl
+12 -9
View File
@@ -1,23 +1,26 @@
# Warm gVisor workers for ax tasks on this host (Agent Substrate WorkerPool). ax's per-task ActorTemplates
# set no workerSelector, so every pool of the matching sandbox class is eligible. Two workers of 1 CPU /
# 1.5 GiB fit the 7 GB host beside the control planes; an actor occupies a whole worker.
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
name: ousast-pool
name: ${POOL_NAME}
namespace: ax-system
labels:
workload: ousast
spec:
replicas: 2
workerImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor
replicas: ${POOL_REPLICAS}
workerImage: ${WORKER_IMAGE}
template:
nodeSelector:
ate.dev/substrate-version: "${SUBSTRATE_VERSION}"
resources:
limits:
cpu: "1"
memory: 1536Mi
cpu: "${POOL_CPU}"
memory: ${POOL_MEMORY}
requests:
cpu: 250m
memory: 1536Mi
memory: ${POOL_MEMORY}
ops/k8s/profiles/k3s.toml
+26
View File
@@ -0,0 +1,26 @@
# The k3s profile: the plane on the maintainer's production cluster (plane-on-kubernetes design section 1;
# requirements "Target Environment"). Documented values, not secrets: the kubeconfig for the context, the GHCR
# pull secret's content and the store's keys live outside this file. Every field is overridable by OUSAST_<FIELD>,
# a pool by OUSAST_POOL_<NAME>_<FIELD>. Nothing has been applied to a k3s cluster yet; docs/deployment.md says
# which items are planned.
exec = "remote" # the same CLI from a laptop or a CI job, through the cluster's kube context
kube_context = "ousast-k3s" # the maintainer's context name for the k3s cluster
registry = "ghcr.io/norandom" # images built by GitHub Actions from tagged commits (design section 8)
images = "images.json" # the release asset {"runner": "ghcr.io/norandom/ousast-runner@sha256:...", "engine": ...}
router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel)
ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel)
memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from the CI secrets or .env
atespace = "default"
snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch)
image_pull_secret = "ghcr-pull" # only needed while the GHCR packages are private
[pools.default] # production sizing: four verify-class workers; the HPA (design section 5) scales it
replicas = 4
cpu = "1"
memory = "1536Mi"
[pools.engine] # two engine workers sized from the measured 2.1 GiB peak under a 3 GiB limit
replicas = 2
cpu = "2"
memory = "4Gi"
ops/k8s/profiles/kind-images.json
+3
View File
@@ -0,0 +1,3 @@
{
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
}
ops/k8s/profiles/kind.toml
+25
View File
@@ -0,0 +1,25 @@
# The kind profile: the plane on this laptop's single-node kind cluster, which ops/ax/up.sh creates and which is
# the development default (plane-on-kubernetes design section 1). Values only; a credential never goes here
# (keys stay in the environment or .env). Every field is overridable by OUSAST_<FIELD> in the environment, a pool
# by OUSAST_POOL_<NAME>_<FIELD>. This file may name the kind cluster; the code must not (tests/test_plane_literals.py).
exec = "local" # the reconciler runs on this host against the cluster below
kube_context = "kind-ousast" # kind's context for KIND_CLUSTER_NAME=ousast
registry = "localhost:5001" # the kind-local registry up.sh creates (Substrate rewrites it for the node)
images = "kind-images.json" # digest-pinned image references, written by up.sh after each build
router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel)
ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel)
memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from .env
atespace = "default"
snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch)
image_pull_secret = "" # the kind registry needs none
[pools.default] # ousast-pool: two gVisor workers fit the 7 GB host beside the control planes
replicas = 2
cpu = "1"
memory = "1536Mi"
[pools.engine] # engine-pool (design section 4): one worker for the Joern engine task
replicas = 1
cpu = "2"
memory = "4Gi"
plane/tasks/agree.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: agree
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["agree"]
resources:
requests:
plane/tasks/alerts.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: alerts
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["alerts"]
resources:
requests:
plane/tasks/features.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: features
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["features"]
resources:
requests:
plane/tasks/loop.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: loop
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["loop"]
resources:
requests:
plane/tasks/remember.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: remember
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["remember"]
resources:
requests:
plane/tasks/repo-facts.yaml
+1 -1
View File
@@ -8,7 +8,7 @@ metadata:
name: repo-facts
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["repo-facts"]
resources:
requests:
plane/tasks/roles.yaml
+1 -1
View File
@@ -11,7 +11,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["roles"]
resources:
requests:
plane/tasks/validation-46.yaml
+120 -120
View File
@@ -5,9 +5,9 @@ metadata:
name: repo-facts-budibase-mongo-template-nosqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}'
openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -36,7 +36,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -67,7 +67,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -96,7 +96,7 @@ metadata:
name: agree-budibase-mongo-template-nosqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -123,9 +123,9 @@ metadata:
name: repo-facts-cct-studio-server-cmdi
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}'
openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -154,7 +154,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -185,7 +185,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -214,7 +214,7 @@ metadata:
name: agree-cct-studio-server-cmdi
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -241,9 +241,9 @@ metadata:
name: repo-facts-contextforge-prompt-ssti
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}'
openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -272,7 +272,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -303,7 +303,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -332,7 +332,7 @@ metadata:
name: agree-contextforge-prompt-ssti
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -359,9 +359,9 @@ metadata:
name: repo-facts-froxlor-admin-ip-second-order-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}'
openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -390,7 +390,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -421,7 +421,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -450,7 +450,7 @@ metadata:
name: agree-froxlor-admin-ip-second-order-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -477,9 +477,9 @@ metadata:
name: repo-facts-fuxa-tdengine-escape-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}'
openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -508,7 +508,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -539,7 +539,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -568,7 +568,7 @@ metadata:
name: agree-fuxa-tdengine-escape-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -595,9 +595,9 @@ metadata:
name: repo-facts-lmdeploy-media-url-ssrf
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}'
openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -626,7 +626,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -657,7 +657,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -686,7 +686,7 @@ metadata:
name: agree-lmdeploy-media-url-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -713,9 +713,9 @@ metadata:
name: repo-facts-medplum-external-auth-open-redirect
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}'
openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -744,7 +744,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -775,7 +775,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -804,7 +804,7 @@ metadata:
name: agree-medplum-external-auth-open-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -831,9 +831,9 @@ metadata:
name: repo-facts-pgadmin-maintenance-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}'
openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -862,7 +862,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -893,7 +893,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -922,7 +922,7 @@ metadata:
name: agree-pgadmin-maintenance-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -949,9 +949,9 @@ metadata:
name: repo-facts-qwed-sympy-eval
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}'
openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -980,7 +980,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1011,7 +1011,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1040,7 +1040,7 @@ metadata:
name: agree-qwed-sympy-eval
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1067,9 +1067,9 @@ metadata:
name: repo-facts-reactpress-findall-column-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}'
openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1098,7 +1098,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1129,7 +1129,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1158,7 +1158,7 @@ metadata:
name: agree-reactpress-findall-column-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1185,9 +1185,9 @@ metadata:
name: repo-facts-unleash-addon-webhook-ssrf
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}'
openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1216,7 +1216,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1247,7 +1247,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1276,7 +1276,7 @@ metadata:
name: agree-unleash-addon-webhook-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1303,9 +1303,9 @@ metadata:
name: repo-facts-winml-cli-serve-cors
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}'
openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1334,7 +1334,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1365,7 +1365,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1394,7 +1394,7 @@ metadata:
name: agree-winml-cli-serve-cors
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1421,9 +1421,9 @@ metadata:
name: repo-facts-wp-directorist-avatar-ssrf
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}'
openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1452,7 +1452,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1483,7 +1483,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1512,7 +1512,7 @@ metadata:
name: agree-wp-directorist-avatar-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1539,9 +1539,9 @@ metadata:
name: repo-facts-wp-gopay-log-filter-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}'
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1570,7 +1570,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1601,7 +1601,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1630,7 +1630,7 @@ metadata:
name: agree-wp-gopay-log-filter-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1657,9 +1657,9 @@ metadata:
name: repo-facts-wp-groundhogg-confirm-redirect
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}'
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1688,7 +1688,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1719,7 +1719,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1748,7 +1748,7 @@ metadata:
name: agree-wp-groundhogg-confirm-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1777,7 +1777,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1808,7 +1808,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1839,7 +1839,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1870,7 +1870,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1901,7 +1901,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1932,7 +1932,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1963,7 +1963,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1994,7 +1994,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2025,7 +2025,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2056,7 +2056,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2087,7 +2087,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2118,7 +2118,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2149,7 +2149,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2180,7 +2180,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2211,7 +2211,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2240,7 +2240,7 @@ metadata:
name: features-budibase-mongo-template-nosqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2263,7 +2263,7 @@ metadata:
name: remember-budibase-mongo-template-nosqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2288,7 +2288,7 @@ metadata:
name: features-cct-studio-server-cmdi
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2311,7 +2311,7 @@ metadata:
name: remember-cct-studio-server-cmdi
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2336,7 +2336,7 @@ metadata:
name: features-contextforge-prompt-ssti
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2359,7 +2359,7 @@ metadata:
name: remember-contextforge-prompt-ssti
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2384,7 +2384,7 @@ metadata:
name: features-froxlor-admin-ip-second-order-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2407,7 +2407,7 @@ metadata:
name: remember-froxlor-admin-ip-second-order-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2432,7 +2432,7 @@ metadata:
name: features-fuxa-tdengine-escape-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2455,7 +2455,7 @@ metadata:
name: remember-fuxa-tdengine-escape-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2480,7 +2480,7 @@ metadata:
name: features-lmdeploy-media-url-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2503,7 +2503,7 @@ metadata:
name: remember-lmdeploy-media-url-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2528,7 +2528,7 @@ metadata:
name: features-medplum-external-auth-open-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2551,7 +2551,7 @@ metadata:
name: remember-medplum-external-auth-open-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2576,7 +2576,7 @@ metadata:
name: features-pgadmin-maintenance-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2599,7 +2599,7 @@ metadata:
name: remember-pgadmin-maintenance-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2624,7 +2624,7 @@ metadata:
name: features-qwed-sympy-eval
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2647,7 +2647,7 @@ metadata:
name: remember-qwed-sympy-eval
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2672,7 +2672,7 @@ metadata:
name: features-reactpress-findall-column-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2695,7 +2695,7 @@ metadata:
name: remember-reactpress-findall-column-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2720,7 +2720,7 @@ metadata:
name: features-unleash-addon-webhook-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2743,7 +2743,7 @@ metadata:
name: remember-unleash-addon-webhook-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2768,7 +2768,7 @@ metadata:
name: features-winml-cli-serve-cors
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2791,7 +2791,7 @@ metadata:
name: remember-winml-cli-serve-cors
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2816,7 +2816,7 @@ metadata:
name: features-wp-directorist-avatar-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2839,7 +2839,7 @@ metadata:
name: remember-wp-directorist-avatar-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2864,7 +2864,7 @@ metadata:
name: features-wp-gopay-log-filter-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2887,7 +2887,7 @@ metadata:
name: remember-wp-gopay-log-filter-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2912,7 +2912,7 @@ metadata:
name: features-wp-groundhogg-confirm-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2935,7 +2935,7 @@ metadata:
name: remember-wp-groundhogg-confirm-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
plane/tasks/verify.yaml
+1 -1
View File
@@ -10,7 +10,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["verify"]
resources:
requests:
src/openultrasast/cli.py
+29 -10
View File
@@ -323,16 +323,21 @@ def _main(argv: list[str] | None) -> int:
learn_labels.add_argument("--include-title", action="store_true", help="the title-tier pairs as well (a sensitivity arm)")
learn_labels.add_argument("--no-assumed-benign", action="store_true", help="skip mining the assumed-benign history source")
add_learn_commands(learn_sub)
plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on this host's kind cluster")
plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on the profile's cluster (ops/k8s/profiles/)")
profile_help = "the PlaneProfile: a name under ops/k8s/profiles/ or a .toml path (default OUSAST_PLANE_PROFILE, else kind)"
plane_sub = plane.add_subparsers(dest="plane_command", required=True)
plane_run = plane_sub.add_parser("run", help="execute a Run manifest on ax; a rerun skips done tasks")
plane_run.add_argument("run_manifest", type=Path)
plane_run.add_argument("--workers", type=int, default=1, help="tasks in flight at once (serialize labels never overlap)")
plane_run.add_argument("--ax", default="ax", help="path of the ax CLI")
plane_run.add_argument("--profile", help=profile_help)
plane_status = plane_sub.add_parser("status", help="per-task status and the token attribution table of a run")
plane_status.add_argument("run")
plane_status.add_argument("--units", action="store_true", help="per-unit rows from each task's units.jsonl")
plane_sub.add_parser("doctor", help="check kind, Agent Substrate, the ax controller and the runner image")
plane_doctor = plane_sub.add_parser(
"doctor", help="check the profile's cluster, Agent Substrate, the ax controller, the runner image and the memory store"
)
plane_doctor.add_argument("--profile", help=profile_help)
plane_remember = plane_sub.add_parser("remember", help="ingest a run's memory rows into the store (OUSAST_MEMORY)")
plane_remember.add_argument("run")
plane_remember.add_argument("--plane", type=Path, default=Path("plane"), help="where runs/<run>.yaml names population and split")
@@ -370,9 +375,11 @@ def _main(argv: list[str] | None) -> int:
)
plane_harvest.add_argument("--ceiling", type=float, default=10.0, help="USD: each Run's task budgets sum to at most this")
plane_harvest.add_argument("--runner-image", type=Path, help="re-pin the task templates to the image in this file first (ops/ax/up.sh)")
plane_harvest.add_argument("--profile", help=profile_help)
plane_ws = plane_sub.add_parser("workspaces", help="one Workspace manifest per case pin of a population file")
plane_ws.add_argument("population", type=Path)
plane_ws.add_argument("--out", type=Path, default=Path("plane/workspaces"))
plane_ws.add_argument("--profile", help=profile_help)
plane_ws.add_argument("--validation-set", type=Path, help="JSON {case: [[path, function], ...]}: emit that set's increment instead")
plane_ws.add_argument("--candidates", type=Path, help="with --validation-set: per-case candidate records (independent-v2-modelsinks)")
plane_ws.add_argument("--triage", type=Path, help="with --validation-set: recorded batched-check scans (<case>--vulnerable_a.jsonl)")
@@ -1529,10 +1536,22 @@ def _learn_labels(args: argparse.Namespace) -> int:
def _plane(args: argparse.Namespace) -> int:
from .plane import reconciler # lazy: the plane is a maintainer surface, not the scan path
from .plane.profile import ProfileError, load_profile
try:
profile = load_profile(getattr(args, "profile", None))
except ProfileError as exc:
print(f"plane: {exc}", file=sys.stderr)
return 2
if args.plane_command == "run":
from .plane.memory import MemoryStoreError
_plane_memory("seed", args.run_manifest)
result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax)
try:
result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax, profile=profile)
except MemoryStoreError as exc: # the store is the state of record: no fallback, the Run is not submitted
print(f"memory store {profile.memory}: {exc}", file=sys.stderr)
return 1
print(f"run finished: {result}")
_plane_memory("ingest", args.run_manifest)
return 0 if result == "done" else 1
@@ -1547,14 +1566,14 @@ def _plane(args: argparse.Namespace) -> int:
if args.plane_command == "alerts-engine":
return _plane_alerts_engine(args)
if args.plane_command == "harvest":
return _plane_harvest(args)
return _plane_harvest(args, profile.atespace)
if args.plane_command == "doctor":
checks = reconciler.doctor()
checks = reconciler.doctor(profile)
for name, ok, text in checks:
print(f"[{'ok' if ok else 'FAIL'}] {name}: {text}")
return 0 if all(ok for _, ok, _ in checks) else 1
if args.validation_set is not None:
return _plane_increment(args)
return _plane_increment(args, profile.atespace)
from .plane.workspaces import workspaces
written = workspaces(args.population, args.out)
@@ -1562,7 +1581,7 @@ def _plane(args: argparse.Namespace) -> int:
return 0
def _plane_harvest(args: argparse.Namespace) -> int:
def _plane_harvest(args: argparse.Namespace, atespace: str) -> int:
from .plane.generate import repin_templates
from .plane.harvest import write_harvest
@@ -1571,7 +1590,7 @@ def _plane_harvest(args: argparse.Namespace) -> int:
command = f"ousast plane harvest --labels {args.labels.name} --name {args.name} --ceiling {args.ceiling:g}"
files = write_harvest(
args.labels, args.plane, name=args.name, command=command, catalog=args.catalog, cache=args.cache, templates=args.templates,
ceiling=args.ceiling,
ceiling=args.ceiling, atespace=atespace,
) # fmt: skip
for relative in sorted(k for k in files if k.startswith("runs/")):
print(f"{args.plane / relative}: {files[relative].splitlines()[1].lstrip('# ')}")
@@ -1652,7 +1671,7 @@ def _plane_memory(
return 0
def _plane_increment(args: argparse.Namespace) -> int:
def _plane_increment(args: argparse.Namespace, atespace: str) -> int:
from .plane.generate import LOOP_MANIFEST, Loop, increment
if args.candidates is None or args.triage is None:
@@ -1678,7 +1697,7 @@ def _plane_increment(args: argparse.Namespace) -> int:
command += "".join(f" --qualify-population {p}" for p in loop.populations)
written = increment(
args.population, args.validation_set, args.candidates, args.triage, plane=args.plane, run_name=args.run, command=command,
repos=args.repos, runner_image=args.runner_image, loop=loop,
repos=args.repos, runner_image=args.runner_image, loop=loop, atespace=atespace,
) # fmt: skip
print(f"{len(written)} manifests written under {args.plane} (Run {args.run})")
return 0
src/openultrasast/cpg/session.py
+3 -1
View File
@@ -162,7 +162,9 @@ class EngineSession:
return None
payload = json.dumps({"query": code}).encode()
call = urllib.request.Request(
f"http://127.0.0.1:{self._port}/query-sync", data=payload, headers={"Content-Type": "application/json"}
f"http://127.0.0.1:{self._port}/query-sync", # loopback: the Joern server this process started
data=payload,
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(call, timeout=timeout) as response: # noqa: S310 -- fixed loopback URL
src/openultrasast/plane/delivery.py
+192
View File
@@ -0,0 +1,192 @@
"""Task artifacts through the memory store (plane-on-kubernetes design section 3, Req 3.1).
A task's runner PUTs its output tar to a presigned URL the reconciler minted, and GETs each declared input from
another; the URLs travel in the start request's ``delivery`` field (a presigned URL is a bearer capability, so it
goes where the credentials go, never into an ActorTemplate's env). The reconciler polls the store for the tar,
extracts it under the run directory with the member checks the old receiver had, re-puts the task's declared
``outputs[]`` as single objects so a consumer's URL names exactly one artifact, and writes ``done.json``. The run's
``state.json`` is mirrored to the store after each change and seeded from it on a rerun elsewhere.
Store layout under the 30-day ``runs/`` expiry rule (``memory.RUNS_RULE_ID``)::
runs/<run>/state.json the reconciler's state, mirrored
runs/<run>/<task>/output.tar the runner's PUT
runs/<run>/<task>/<artifact> each declared output, re-put by the reconciler
runs/<run>/<task>/done.json status, bytes, file count, finished
runs/<run>/inputs/<name> an input no task produces (``put_input``)
Nothing here logs or prints a URL.
"""
from __future__ import annotations
import io
import json
import os
import re
import tarfile
from collections.abc import Iterable, Mapping
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from urllib.parse import urlsplit
from .manifests import Run
from .memory import RUNS_PREFIX, MemoryStore
__all__ = ["DONE", "GRACE_SECONDS", "OUTPUT_TAR", "STATE", "Delivery", "DeliveryError", "declared", "expiry", "input_var"]
OUTPUT_TAR = "output.tar"
DONE = "done.json"
STATE = "state.json"
INPUTS_DIR = "inputs"
GRACE_SECONDS = 600 # a URL outlives the task timeout by this much; a re-start after a re-resume re-mints anyway
_NAME = re.compile(r"[^A-Z0-9]+")
class DeliveryError(RuntimeError):
"""A delivered tar that cannot be accepted, or an object that is not there; the message never carries a URL."""
def expiry(environ: Mapping[str, str] | None = None) -> timedelta:
"""``OUSAST_TASK_TIMEOUT`` (7200 s) plus :data:`GRACE_SECONDS`."""
env = os.environ if environ is None else environ
return timedelta(seconds=float(env.get("OUSAST_TASK_TIMEOUT") or 7200) + GRACE_SECONDS)
def input_var(name: str) -> str:
"""The ``OUSAST_INPUT_<NAME>`` suffix of a Run input name, as the reconciler renders it."""
return _NAME.sub("_", name.upper())
def _now() -> str:
return datetime.now(UTC).isoformat(timespec="seconds")
def declared(run: Run) -> dict[str, list[str]]:
"""Per Run task, the artifacts to publish as single objects: its ``outputs[]`` plus every ``<producer>/<artifact>``
a consumer declares as an input on it (``summary.json`` of a producer is a usual one), in declaration order."""
wanted: dict[str, list[str]] = {entry.name: list(entry.outputs) for entry in run.tasks}
for entry in run.tasks:
for ref in entry.inputs.values():
producer, _, artifact = ref.partition("/")
if producer in wanted and artifact and artifact not in wanted[producer]:
wanted[producer].append(artifact)
return wanted
class Delivery:
"""Presigned URLs, polling, collection and state mirroring for one Run on one store."""
def __init__(self, store: MemoryStore, run: str, base: Path, *, expires: timedelta | None = None) -> None:
self.store, self.run, self.base = store, run, base
self.expires = expires or expiry()
def key(self, *parts: str) -> str:
return f"{RUNS_PREFIX}{self.run}/" + "/".join(parts)
@property
def host(self) -> str | None:
"""The hostname the runner will dial (for the task's egress policy); None for a store without one."""
return urlsplit(self.store.presign_get(self.key(STATE), timedelta(seconds=60))).hostname
# --- the start body -------------------------------------------------------------------------------------------
def put_url(self, task: str) -> str:
return self.store.presign_put(self.key(task, OUTPUT_TAR), self.expires)
def input_urls(self, inputs: Mapping[str, str]) -> dict[str, str]:
"""``{OUSAST_INPUT_<NAME> suffix: presigned GET of runs/<run>/<producer>/<artifact>}``."""
out: dict[str, str] = {}
for name, ref in inputs.items():
producer, _, artifact = ref.partition("/")
if not producer or not artifact:
raise DeliveryError(f"input {name}: {ref!r} is not <producer>/<artifact>")
out[input_var(name)] = self.store.presign_get(self.key(producer, artifact), self.expires)
return out
def body(self, task: str, inputs: Mapping[str, str]) -> dict[str, Any]:
"""The start request's ``delivery`` field for ``task``."""
return {"put": self.put_url(task), "inputs": self.input_urls(inputs)}
# --- completion ----------------------------------------------------------------------------------------------
def reset(self, task: str) -> None:
"""Before an attempt: remove the previous attempt's tar and ``done.json`` so a rerun (an unfinished task
resumed with a larger budget, an interrupted attempt) never completes on a stale object."""
for name in (OUTPUT_TAR, DONE):
self.store._delete(self.key(task, name))
def delivered(self, task: str) -> bool:
"""Whether the runner's tar is in the store (an exact-key listing, the store's HEAD)."""
key = self.key(task, OUTPUT_TAR)
return key in self.store._keys(key)
def collect(self, task: str, outputs: Iterable[str]) -> tuple[dict[str, Any] | None, int]:
"""Download and extract the tar under ``<base>/<task>/``, re-put the declared ``outputs``, write
``done.json``; returns (the extracted ``summary.json`` or None, the tar's byte size)."""
got = self.store._get(self.key(task, OUTPUT_TAR))
if got is None:
raise DeliveryError(f"task {task}: output.tar is not in the store")
data = got[0]
dest = (self.base / task).resolve()
try:
with tarfile.open(fileobj=io.BytesIO(data), mode="r:") as tar:
members = tar.getmembers()
for m in members:
if m.issym() or m.islnk() or not (dest / m.name).resolve().is_relative_to(dest):
raise DeliveryError(f"task {task}: rejected member {m.name}")
dest.mkdir(parents=True, exist_ok=True)
tar.extractall(dest, members) # every member checked above
except tarfile.TarError as exc:
raise DeliveryError(f"task {task}: bad tar: {exc}") from None
published = self.publish(task, outputs)
summary = _read_json(dest / "summary.json")
done = {
"task": task,
"status": summary.get("status") if isinstance(summary, dict) else None,
"bytes": len(data),
"files": sum(1 for m in members if m.isfile()),
"published": published,
"finished": _now(),
}
self.store._put(self.key(task, DONE), json.dumps(done, sort_keys=True).encode("utf-8"))
return (summary if isinstance(summary, dict) else None), len(data)
def publish(self, task: str, outputs: Iterable[str]) -> list[str]:
"""Re-put the task's declared outputs that exist under ``<base>/<task>/`` as single objects; the names put."""
put: list[str] = []
root = (self.base / task).resolve()
for artifact in outputs:
path = (root / artifact).resolve()
if path.is_relative_to(root) and path.is_file():
self.store._put(self.key(task, artifact), path.read_bytes())
put.append(artifact)
return put
def put_input(self, name: str, path: Path) -> str:
"""Store a file no task produces as ``runs/<run>/inputs/<name>``; returns its presigned GET URL."""
self.store._put(self.key(INPUTS_DIR, name), Path(path).read_bytes())
return self.store.presign_get(self.key(INPUTS_DIR, name), self.expires)
# --- state ---------------------------------------------------------------------------------------------------
def save_state(self, data: Mapping[str, Any]) -> None:
self.store._put(self.key(STATE), (json.dumps(data, indent=2) + "\n").encode("utf-8"))
def load_state(self) -> dict[str, Any] | None:
got = self.store._get(self.key(STATE))
if got is None:
return None
try:
loaded = json.loads(got[0].decode("utf-8"))
except ValueError:
return None
return loaded if isinstance(loaded, dict) else None
def _read_json(path: Path) -> Any:
try:
return json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError):
return None
src/openultrasast/plane/doctor.py
+83 -13
View File
@@ -1,15 +1,35 @@
"""``ousast plane doctor``: four checks of this host's ax before a Run is submitted (ai-service-plane Req 3.5).
"""``ousast plane doctor``: checks of the configured cluster before a Run is submitted (ai-service-plane Req 3.5,
plane-on-kubernetes Req 1.2).
Everything is read from the :class:`~openultrasast.plane.profile.PlaneProfile`: the kube context is checked for
reachability and, when reachable, the two namespaces for ready pods; the ``images`` file's runner reference is
resolved in its registry (a local registry over its v2 API, a remote one through ``docker manifest inspect``); the
memory store is opened, which verifies an S3 bucket; and the first line reports every configured address. Nothing
here names a cluster, a registry or an address of its own: those are the profile's.
A sibling of ``reconciler.py`` (tasks.md task 0 allows either), which re-exports :func:`doctor`.
"""
from __future__ import annotations
import os
import ipaddress
import re
import subprocess
import urllib.error
import urllib.request
from .profile import PlaneProfile, ProfileError, load_profile
UP = "ops/ax/up.sh brings it up"
RUNBOOK = "docs/deployment.md lists the prerequisites"
_MANIFEST_TYPES = ", ".join(
(
"application/vnd.oci.image.index.v1+json",
"application/vnd.oci.image.manifest.v1+json",
"application/vnd.docker.distribution.manifest.list.v2+json",
"application/vnd.docker.distribution.manifest.v2+json",
)
)
# --- checks ----------------------------------------------------------------------------------------------
@@ -34,18 +54,68 @@ def _pods_ready(namespace: str, context: str) -> tuple[bool, str]:
return not bad, f"not ready: {', '.join(bad)}" if bad else f"{len(rows)} pods ready"
def doctor() -> list[tuple[str, bool, str]]:
"""Four checks of the host's ax: kind, Agent Substrate, the ax controller, the runner image (Req 3.5)."""
context = "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast")
def _is_local(registry: str) -> bool:
host = registry.rsplit(":", 1)[0] if re.search(r":\d+$", registry) else registry
if host == "localhost":
return True
try:
return ipaddress.ip_address(host.strip("[]")).is_loopback or ipaddress.ip_address(host.strip("[]")).is_private
except ValueError:
return False
def _registry_has(image: str) -> tuple[bool, str]:
"""Resolve a digest-pinned reference ``<registry>/<repository>@sha256:<hex>``: a local registry answers its v2
API for the manifest by digest (plain HTTP, as kind's registry speaks); a remote one is asked through
``docker manifest inspect``, which carries the operator's registry login."""
name, _, digest = image.partition("@")
registry, _, repository = name.partition("/")
short = f"{repository}@{digest[:19]}..."
if _is_local(registry):
url = f"http://{registry}/v2/{repository}/manifests/{digest}"
request = urllib.request.Request(url, method="HEAD", headers={"Accept": _MANIFEST_TYPES})
try:
with urllib.request.urlopen(request, timeout=20) as response: # noqa: S310 - the profile's registry
found = 200 <= int(response.status) < 300
except urllib.error.HTTPError as exc:
return False, f"{short} missing from {registry} (HTTP {exc.code})"
except (urllib.error.URLError, OSError) as exc:
return False, f"{registry} unreachable ({str(exc)[:80]})"
return found, f"{short} present in {registry}" if found else f"{short} missing from {registry}"
ok, out = _sh("docker", "manifest", "inspect", image)
return ok, f"{short} present in {registry}" if ok else f"{short} not resolved in {registry} ({out[:120]})"
def doctor(profile: PlaneProfile | None = None) -> list[tuple[str, bool, str]]:
"""The checks of the profile's cluster: ``(name, ok, text)`` per line, the configured addresses first."""
profile = profile or load_profile()
hint = UP if profile.name == "kind" else RUNBOOK
context = profile.kube_context
checks = [(f"profile {profile.name}", True, " ".join(f"{k}={v}" for k, v in profile.addresses().items()))]
ok, out = _sh("kubectl", "--context", context, "cluster-info")
checks = [("kind cluster", ok, "reachable" if ok else f"context {context} unreachable ({out[:80]}); {UP}")]
for label, ns in (("agent substrate (ate-system)", "ate-system"), ("ax controller (ax-system)", "ax-system")):
ok, out = _pods_ready(ns, context)
checks.append((label, ok, out if ok else f"{out}; {UP}"))
ok, out = _sh("curl", "-s", "localhost:5001/v2/_catalog")
present = ok and "ousast-runner" in out
text = "ousast-runner present" if present else f"missing from localhost:5001 ({out[:80]}); {UP} and loads it"
checks.append(("runner image in kind registry", present, text))
checks.append((f"cluster {context}", ok, "reachable" if ok else f"context {context} unreachable ({out[:80]}); {hint}"))
if ok: # behind an unreachable context the namespaces cannot be told apart from it
for label, ns in (("agent substrate (ate-system)", "ate-system"), ("ax controller (ax-system)", "ax-system")):
ok, out = _pods_ready(ns, context)
checks.append((label, ok, out if ok else f"{out}; {hint}"))
try:
images = profile.load_images()
runner = images["runner"]
except ProfileError as exc:
checks.append((f"runner image in {profile.registry}", False, f"{exc}; {hint}"))
except KeyError:
checks.append((f"runner image in {profile.registry}", False, f"{profile.images} names no runner image; {hint}"))
else:
ok, out = _registry_has(runner)
checks.append((f"runner image in {profile.registry}", ok, out if ok else f"{out}; {hint} and loads it"))
from .memory import MemoryStoreError, open_store # lazy: boto3 only when the profile names an S3 store
try:
store = open_store(profile.memory)
except (MemoryStoreError, OSError, ValueError) as exc:
checks.append((f"memory store {profile.memory}", False, str(exc)[:300]))
else:
checks.append((f"memory store {profile.memory}", True, f"{store.describe()} opens (an S3 bucket is verified)"))
return checks
src/openultrasast/plane/egress.py
+41 -19
View File
@@ -84,17 +84,32 @@ def git_hosts(workspaces: Iterable[Workspace]) -> tuple[list[str], list[str]]:
return sorted(tls), sorted(plain)
def policy_for(task: Task, workspaces: Iterable[Workspace], model: Model | None, receiver_host: str | None) -> dict[str, Any]:
"""The task's EgressPolicy body: the receiver over HTTP/80, Git and Model hosts as TLS passthrough on 443."""
def policy_for(
task: Task,
workspaces: Iterable[Workspace],
model: Model | None,
store_host: str | None,
receiver_host: str | None = None,
) -> dict[str, Any]:
"""The task's EgressPolicy body (plane-on-kubernetes design section 3): one ``tlsPassthrough`` rule on 443 over
the sorted union of the store's hostname (``store_host``: where the runner PUTs its tar and GETs its inputs), the
bound Workspaces' https Git hosts and the bound Model's ``openultrasast.io/egress-hosts``. No ``http`` rule, unless
``receiver_host`` names the old receiver (the path task 2.6 removes). An http Git repo cannot be reached under
this policy and is refused; a store given as an address is refused naming ``S3_ENDPOINT`` (the gateway allows
hostnames only)."""
tls, plain = git_hosts(workspaces)
tls = sorted(set(tls) | set(model.egress_hosts if model else ()))
if receiver_host and not _is_ip(receiver_host): # an address cannot be allowed; only the direct test mode has one
plain = sorted({*plain, receiver_host.lower()})
rules: list[dict[str, Any]] = []
if plain:
rules.append({"http": {"hostnames": plain, "ports": {"numbers": [80]}}})
if tls:
rules.append({"tlsPassthrough": {"hostnames": tls, "ports": {"numbers": [443]}}})
raise ValueError(f"Task/{task.metadata.name}: http Git repos ({', '.join(plain)}) cannot be reached: the policy grants TLS only")
hosts = set(tls) | set(model.egress_hosts if model else ())
if store_host:
if _is_ip(store_host):
raise ValueError(f"Task/{task.metadata.name}: the store endpoint is an address ({store_host}); S3_ENDPOINT must be a hostname")
hosts.add(store_host.lower())
rules: list[dict[str, Any]] = []
if receiver_host and not _is_ip(receiver_host): # the receiver path, kept for group 2 only (task 2.6 deletes it)
rules.append({"http": {"hostnames": [receiver_host.lower()], "ports": {"numbers": [80]}}})
if hosts:
rules.append({"tlsPassthrough": {"hostnames": sorted(hosts), "ports": {"numbers": [443]}}})
return {"rules": rules}
@@ -110,9 +125,11 @@ def rules_of(policy: Mapping[str, Any]) -> list[dict[str, Any]]:
return sorted(out, key=lambda r: json.dumps(r, sort_keys=True))
def receiver_cluster_ip(context: str | None = None) -> str | None:
"""The ClusterIP of the ``ousast-receiver`` Service (``kubectl get svc``); None when kubectl cannot say."""
context = context or "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast")
def receiver_cluster_ip(context: str) -> str | None:
"""The ClusterIP of the ``ousast-receiver`` Service (``kubectl get svc`` in the profile's ``context``); None when
kubectl cannot say."""
if not context:
raise RuntimeError("receiver_cluster_ip needs the profile's kube context")
argv = ["kubectl", "--context", context, "-n", "ax-system", "get", "svc", "ousast-receiver", "-o", "jsonpath={.spec.clusterIP}"]
try:
proc = subprocess.run(argv, capture_output=True, text=True, check=False, timeout=20)
@@ -122,8 +139,9 @@ def receiver_cluster_ip(context: str | None = None) -> str | None:
return address if proc.returncode == 0 and _is_ip(address) else None
def receiver_address(port: int) -> tuple[str, str | None]:
"""(``OUSAST_ARTIFACT_URL``, ``OUSAST_ARTIFACT_DIAL``) for a receiver bound on ``port``.
def receiver_address(port: int, context: str | None) -> tuple[str, str | None]:
"""(``OUSAST_ARTIFACT_URL``, ``OUSAST_ARTIFACT_DIAL``) for a receiver bound on ``port``; ``context`` is the
profile's kube context the Service's ClusterIP is looked up in.
``OUSAST_ARTIFACT_HOST`` set: the old direct URL ``http://<host>:<port>/`` and no dial (tests, a host the actor
reaches without the gateway). Otherwise the receiver Service's name, dialled at ``OUSAST_ARTIFACT_DIAL`` or the
@@ -135,7 +153,9 @@ def receiver_address(port: int) -> tuple[str, str | None]:
return f"http://{host}:{port}/", None
dial = os.environ.get("OUSAST_ARTIFACT_DIAL")
if not dial:
address = receiver_cluster_ip()
if not context:
raise RuntimeError("receiver_address needs the profile's kube context to look the receiver Service up")
address = receiver_cluster_ip(context)
dial = f"{address}:80" if address else None
return f"http://{RECEIVER_HOST}/", dial
@@ -143,9 +163,11 @@ def receiver_address(port: int) -> tuple[str, str | None]:
class Egress:
"""``kubectl ate`` for one actor's egress policy; the executable is injectable so tests use a fake."""
def __init__(self, executable: str | None = None, context: str | None = None) -> None:
def __init__(self, executable: str | None, context: str) -> None:
if not context:
raise ValueError("Egress needs the profile's kube context; it never builds one")
self.executable = executable or os.environ.get("OUSAST_KUBECTL_ATE") or "kubectl-ate"
self.context = context or "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast")
self.context = context
def _run(self, *args: str, stdin: str | None = None) -> subprocess.CompletedProcess[str]:
argv = [self.executable, "--context", self.context, *args]
@@ -214,12 +236,12 @@ class Receiver(ThreadingHTTPServer):
daemon_threads = True
def __init__(self, run_name: str, base: Path, port: int) -> None:
def __init__(self, run_name: str, base: Path, port: int, context: str | None = None) -> None:
super().__init__(("0.0.0.0", port), _Handler)
self.run_name, self.base, self.running, self.delivered = run_name, base, set[str](), set[str]()
self.inputs: dict[str, frozenset[str]] = {}
self.lock = threading.Lock()
self.url, self.dial = receiver_address(self.server_address[1])
self.url, self.dial = receiver_address(self.server_address[1], context)
def _task(self, headers: Any) -> str | None:
"""The ``X-Ousast-Task`` of a request naming this run and a task that is running now, else None."""
src/openultrasast/plane/generate.py
+39 -20
View File
@@ -45,8 +45,9 @@ The recorded triage cost is derived per file: a batched-check record's ``usd`` p
hunts, and its ``usage`` counts only the hunts (``verify_sinks._Recording`` slices the client's usage from the
first hunt on), so the triage share is ``usd`` minus that usage priced at the reference model's list price.
``--runner-image FILE`` (``~/.cache/ousast/ax-src/runner-image``, written by ``ops/ax/up.sh``) re-pins the
templates' ``image:`` to the digest in that file before generating, so a rebuilt image is one command.
``--runner-image FILE`` (the profile's images file, ``ops/k8s/profiles/kind-images.json``, written by ``ops/ax/up.sh``;
the older one-line pin file still reads) re-pins the templates' ``image:`` to the runner digest in that file before
generating, so a rebuilt image is one command.
"""
from __future__ import annotations
@@ -82,7 +83,7 @@ TRIAGE_BASIS = (
RANGES_BASIS = "git diff -U0 <vulnerable> <fixed>, old side: benchmarks/independent/evaluate.hunks(case, 'old')"
HEADROOM = 1.5
CALLS_PER_HUNT = MAX_STEPS + 2 # every tool step, the answer, one empty-content retry
ATESPACE = "default"
ATESPACE = "default" # the default when no profile is given; ``increment`` takes ``profile.atespace``
CASE_PATH = "/workspace/case"
INPUTS_PATH = "/workspace/inputs"
FIXED_PATH = "/workspace/fixed"
@@ -236,10 +237,10 @@ def _case_record(item: CaseInputs) -> dict[str, Any]:
return record
def _workspaces(item: CaseInputs) -> tuple[dict[str, Any], dict[str, Any]]:
def _workspaces(item: CaseInputs, atespace: str = ATESPACE) -> tuple[dict[str, Any], dict[str, Any]]:
case = item.case
git = [{"name": "repo", "repo": case["repo"], "dir": "repo", "depth": 1}]
pinned = _doc("Workspace", _ax_name(case["id"], "vulnerable"), {"git": git}, ATESPACE)
pinned = _doc("Workspace", _ax_name(case["id"], "vulnerable"), {"git": git}, atespace)
pinned["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"repo={case['vulnerable']}"}
files = {
"candidates.json": _json({"id": case["id"], "family": case["family"], "candidates": [list(c) for c in item.kept]}),
@@ -249,7 +250,7 @@ def _workspaces(item: CaseInputs) -> tuple[dict[str, Any], dict[str, Any]]:
{"candidates": [list(c) for c in item.candidates], "kept": [list(c) for c in item.kept], "triage": dict(item.triage)}
),
}
inputs = _doc("Workspace", _ax_name(case["id"], "inputs"), {"files": [{"path": k, "content": v} for k, v in files.items()]}, ATESPACE)
inputs = _doc("Workspace", _ax_name(case["id"], "inputs"), {"files": [{"path": k, "content": v} for k, v in files.items()]}, atespace)
return pinned, inputs
@@ -339,11 +340,11 @@ def _remember(item: CaseInputs, templates: Mapping[str, Task], population: str,
return task, entry
def _alerts(item: CaseInputs, templates: Mapping[str, Task]) -> tuple[dict, dict, dict]:
def _alerts(item: CaseInputs, templates: Mapping[str, Task], atespace: str = ATESPACE) -> tuple[dict, dict, dict]:
"""(fixed-pin Workspace, Task, Run entry) of the case's `alerts`: quick-mode rules on both pins."""
case = item.case
git = [{"name": "repo", "repo": case["repo"], "dir": "repo", "depth": 1}]
fixed = _doc("Workspace", _ax_name(case["id"], "fixed"), {"git": git}, ATESPACE)
fixed = _doc("Workspace", _ax_name(case["id"], "fixed"), {"git": git}, atespace)
fixed["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"repo={case['fixed']}"}
bindings = [(_ax_name(case["id"], "vulnerable"), CASE_PATH), (fixed["metadata"]["name"], FIXED_PATH)]
bindings.append((_ax_name(case["id"], "inputs"), INPUTS_PATH))
@@ -358,10 +359,12 @@ def _alerts(item: CaseInputs, templates: Mapping[str, Task]) -> tuple[dict, dict
return fixed, task, entry
def _loop(cases: Sequence[CaseInputs], templates: Mapping[str, Task], loop: Loop) -> tuple[dict, list[dict], list[dict]]:
def _loop(
cases: Sequence[CaseInputs], templates: Mapping[str, Task], loop: Loop, atespace: str = ATESPACE
) -> tuple[dict, list[dict], list[dict]]:
"""(project Workspace, Tasks, Run entries) of the loop's singletons: snapshot -> measure -> propose -> improve."""
git = [{"name": "repo", "repo": loop.repo, "dir": "repo", "depth": 1}]
project = _doc("Workspace", _ax_name("openultrasast", loop.commit[:12]), {"git": git}, ATESPACE)
project = _doc("Workspace", _ax_name("openultrasast", loop.commit[:12]), {"git": git}, atespace)
project["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"repo={loop.commit}"}
bound = [(project["metadata"]["name"], PROJECT_PATH)]
env = {
@@ -399,6 +402,7 @@ def render(
population: str = "",
split: str = "",
loop: Loop | None = None,
atespace: str = ATESPACE,
) -> dict[str, str]:
"""Relative path under the plane root -> file text, for every generated manifest. ``population`` and ``split``
(the population file's and the set's stems) become the Run's annotations, which the memory rows carry. Each
@@ -411,7 +415,7 @@ def render(
tiebreak: list[dict[str, Any]] = []
for item in cases:
case_id = item.case["id"]
pinned, inputs = _workspaces(item)
pinned, inputs = _workspaces(item, atespace)
for doc in (pinned, inputs):
out[f"workspaces/{doc['metadata']['name']}.yaml"] = _dump(header, [doc])
repo = [(pinned["metadata"]["name"], CASE_PATH), (inputs["metadata"]["name"], INPUTS_PATH)]
@@ -451,7 +455,7 @@ def render(
entries.extend(tiebreak) # appended: the first Run's entries stay byte-identical, so a rerun keeps their state
for item in cases: # appended after the tie-break for the same reason
if loop is not None:
fixed, alerts_task, alerts_entry = _alerts(item, templates)
fixed, alerts_task, alerts_entry = _alerts(item, templates, atespace)
out[f"workspaces/{fixed['metadata']['name']}.yaml"] = _dump(header, [fixed])
tasks.append(alerts_task)
entries.append(alerts_entry)
@@ -462,7 +466,7 @@ def render(
tasks.append(remember_task)
entries.append(remember_entry)
if loop is not None:
project, loop_tasks, loop_entries = _loop(cases, templates, loop)
project, loop_tasks, loop_entries = _loop(cases, templates, loop, atespace)
out[f"workspaces/{project['metadata']['name']}.yaml"] = _dump(header, [project])
tasks.extend(loop_tasks)
entries.extend(loop_entries)
@@ -482,11 +486,18 @@ _IMAGE_LINE = re.compile(r'^(\s*image:\s*)"?[^"\s]+"?\s*$', re.MULTILINE)
def repin_templates(plane: Path, image_file: Path) -> str:
"""Set every template's ``image:`` to the digest-pinned reference in ``image_file`` (as ``ops/ax/up.sh``
writes it); comments and layout are kept. Returns the reference."""
image = image_file.read_text(encoding="utf-8").strip()
if not re.fullmatch(r"\S+@sha256:[0-9a-f]{64}", image):
raise ValueError(f"{image_file}: expected a digest-pinned image reference, got {image!r}")
"""Set every template's ``image:`` to the runner reference of ``image_file``: a profile images file
(``{"runner": "<registry>/<name>@sha256:..."}``, as ``ops/ax/up.sh`` writes it) or the older one-line pin; a
reference without a digest is refused. Comments and layout are kept. Returns the reference. (Task 4.3 gives
the engine template ``images["engine"]``; until then only the runner key is read.)"""
from .profile import ProfileError, load_images
try:
image = load_images(image_file)["runner"]
except ProfileError as exc:
raise ValueError(str(exc)) from None
except KeyError:
raise ValueError(f"{image_file}: no runner image in the images file") from None
for name in TEMPLATES:
path = plane / "tasks" / f"{name}.yaml"
text = path.read_text(encoding="utf-8")
@@ -509,16 +520,24 @@ def increment(
repos: Path | None = None,
runner_image: Path | None = None,
loop: Loop | None = None,
atespace: str | None = None,
) -> list[Path]:
"""Write the generated manifests under ``plane`` from the templates in ``plane/tasks`` (re-pinned first when
``runner_image`` is given), with the improvement loop when ``loop`` is given; returns the paths."""
``runner_image`` is given), with the improvement loop when ``loop`` is given; returns the paths. Workspaces go
to ``atespace`` (default: the profile's)."""
if atespace is None:
from .profile import load_profile
atespace = load_profile().atespace
if loop is not None and repos is None:
raise ValueError("--loop needs --repos: the alerts task marks fixed-pin alerts by the fix's new-side ranges")
if runner_image is not None:
repin_templates(plane, runner_image)
loaded = load_manifests([plane / "tasks" / f"{name}.yaml" for name in TEMPLATES])
cases = read_cases(validation_set, candidates_dir, triage_dir, population, repos, fixed_side=loop is not None)
files = render(cases, loaded.tasks, run_name, command, population=population.stem, split=validation_set.stem, loop=loop)
files = render(
cases, loaded.tasks, run_name, command, population=population.stem, split=validation_set.stem, loop=loop, atespace=atespace
)
written: list[Path] = []
for relative, text in files.items():
path = plane / relative
src/openultrasast/plane/harvest.py
+26 -15
View File
@@ -210,17 +210,19 @@ def _reader(cache: Path, source: str, ref: str, pin: str) -> Any:
# --- manifests -------------------------------------------------------------------------------------------------------
def _git_workspace(unit: Unit) -> dict[str, Any]:
doc = _doc("Workspace", _ax_name(unit.name, "pin"), {"git": [{"name": REPO, "repo": unit.repo, "dir": REPO, "depth": 1}]}, ATESPACE)
def _git_workspace(unit: Unit, atespace: str = ATESPACE) -> dict[str, Any]:
doc = _doc("Workspace", _ax_name(unit.name, "pin"), {"git": [{"name": REPO, "repo": unit.repo, "dir": REPO, "depth": 1}]}, atespace)
doc["metadata"]["annotations"] = {GIT_COMMITS_ANNOTATION: f"{REPO}={unit.pin}"}
return doc
def _files_workspace(name: str, files: Mapping[str, str]) -> dict[str, Any]:
return _doc("Workspace", name, {"files": [{"path": k, "content": v} for k, v in sorted(files.items())]}, ATESPACE)
def _files_workspace(name: str, files: Mapping[str, str], atespace: str = ATESPACE) -> dict[str, Any]:
return _doc("Workspace", name, {"files": [{"path": k, "content": v} for k, v in sorted(files.items())]}, atespace)
def _verify_docs(unit: Unit, templates: Mapping[str, Task], usd_per_hunt: float) -> tuple[list[dict], list[dict], list[dict]]:
def _verify_docs(
unit: Unit, templates: Mapping[str, Task], usd_per_hunt: float, atespace: str = ATESPACE
) -> tuple[list[dict], list[dict], list[dict]]:
"""(Workspaces, Tasks, Run entries) of one verify unit."""
asked = list(unit.verify or unit.candidates)
candidates = {"family": unit.family, "candidates": [list(c) for c in asked]}
@@ -232,15 +234,15 @@ def _verify_docs(unit: Unit, templates: Mapping[str, Task], usd_per_hunt: float)
entries: list[dict] = []
if unit.files:
files = {f"{REPO}/{path}": text for path, text in unit.files} | {"inputs/candidates.json": _json(candidates)}
workspace = _files_workspace(_ax_name(unit.name, "side"), files)
workspace = _files_workspace(_ax_name(unit.name, "side"), files, atespace)
workspaces.append(workspace)
bindings = [(workspace["metadata"]["name"], CASE_PATH)]
inputs = f"{CASE_PATH}/inputs/candidates.json"
else:
pinned = _git_workspace(unit)
pinned = _git_workspace(unit, atespace)
functions = [{"path": p, "function": fn, "line": ln} for p, fn, ln in asked]
files_doc = _files_workspace(
_ax_name(unit.name, "inputs"), {"candidates.json": _json(candidates), "functions.json": _json(functions)}
_ax_name(unit.name, "inputs"), {"candidates.json": _json(candidates), "functions.json": _json(functions)}, atespace
)
workspaces += [pinned, files_doc]
bindings = [(pinned["metadata"]["name"], CASE_PATH), (files_doc["metadata"]["name"], INPUTS_PATH)]
@@ -309,7 +311,7 @@ def _roles_groups(units: Sequence[Unit]) -> list[list[Unit]]:
def render_harvest(
units: Sequence[Unit], templates: Mapping[str, Task], name: str, command: str, *, ceiling: float = 10.0
units: Sequence[Unit], templates: Mapping[str, Task], name: str, command: str, *, ceiling: float = 10.0, atespace: str = ATESPACE
) -> dict[str, str]:
"""Relative path under the plane root -> file text: both Runs, their Workspaces and Tasks, and ``units.json``.
A population file whose size is unknown is estimated at :data:`UNKNOWN_SIZE`."""
@@ -323,7 +325,7 @@ def render_harvest(
tasks: list[dict] = []
entries: list[dict] = []
for unit in verify_units:
w, t, e = _verify_docs(unit, templates, USD_PER_HUNT[_kind(unit)] * scale)
w, t, e = _verify_docs(unit, templates, USD_PER_HUNT[_kind(unit)] * scale, atespace)
workspaces += w
tasks += t
entries += e
@@ -333,7 +335,7 @@ def render_harvest(
for group in _roles_groups([u for u in units if u.files and _inline(u, roles=True) <= MAX_INLINE_BYTES]):
files = {f"{REPO}/{u.name}/{p}": t for u in group for p, t in u.files if p in u.paths}
listing = sorted(f.removeprefix(f"{REPO}/") for f in files)
workspace = _files_workspace(_ax_name(name, "roles", group[0].name), files | {"inputs/files.json": _json(listing)})
workspace = _files_workspace(_ax_name(name, "roles", group[0].name), files | {"inputs/files.json": _json(listing)}, atespace)
workspaces.append(workspace)
estimates.append(_roles_estimate((len(text), len(text.splitlines())) for text in files.values()))
bound = [(workspace["metadata"]["name"], CASE_PATH)]
@@ -350,8 +352,8 @@ def render_harvest(
same = [u for u in units if not u.files and (u.repo, u.pin) == (unit.repo, unit.pin)]
paths = sorted({p for u in same for p in u.paths})
known = {p: (chars, lines) for u in same for p, chars, lines in u.sizes}
pinned = _git_workspace(unit)
files_doc = _files_workspace(_ax_name(unit.name, "roles-in"), {"files.json": _json(paths)})
pinned = _git_workspace(unit, atespace)
files_doc = _files_workspace(_ax_name(unit.name, "roles-in"), {"files.json": _json(paths)}, atespace)
workspaces += [w for w in (pinned, files_doc) if w["metadata"]["name"] not in emitted]
estimates.append(_roles_estimate(known.get(p, UNKNOWN_SIZE) for p in paths))
bound = [(pinned["metadata"]["name"], CASE_PATH), (files_doc["metadata"]["name"], INPUTS_PATH)]
@@ -380,7 +382,16 @@ def _run_files(header: str, run: str, workspaces: list[dict], tasks: list[dict],
def write_harvest(
labels: Path, plane: Path, *, name: str, command: str, catalog: Path, cache: Path, templates: Path, ceiling: float = 10.0
labels: Path,
plane: Path,
*,
name: str,
command: str,
catalog: Path,
cache: Path,
templates: Path,
ceiling: float = 10.0,
atespace: str = ATESPACE,
) -> dict[str, str]:
"""Build the units, render, write under ``plane`` (with the Model manifests copied beside); returns the rendered
files."""
@@ -388,7 +399,7 @@ def write_harvest(
units = pair_units(labels, catalog) + population_units(labels, cache)
loaded = load_manifests([templates / "tasks" / f"{n}.yaml" for n in ("repo-facts", "verify", "agree", "roles")]).tasks
files = render_harvest(units, loaded, name, command, ceiling=ceiling)
files = render_harvest(units, loaded, name, command, ceiling=ceiling, atespace=atespace)
for path in sorted((templates / "models").glob("*.yaml")):
files[f"models/{path.name}"] = path.read_text(encoding="utf-8")
for relative, text in files.items():
src/openultrasast/plane/profile.py
+266
View File
@@ -0,0 +1,266 @@
"""The plane's configuration surface: one ``PlaneProfile`` per cluster (plane-on-kubernetes design section 1).
A profile is a TOML file under ``ops/k8s/profiles/`` (``kind.toml`` for the laptop's kind cluster, ``k3s.toml`` for
the production cluster) holding every value that differs between clusters: the execution mode, the kube context,
the registry and the digest-pinned images file, the router and ax-server addresses (empty: tunnel through the
Kubernetes API), the memory store, the atespace, ax's snapshot bucket, the worker pools and the image pull secret.
:func:`load_profile` reads the file ``OUSAST_PLANE_PROFILE`` names (a profile name or a path; default ``kind``) and
then applies the environment: every scalar field is overridable by ``OUSAST_<FIELD>`` (``OUSAST_KUBE_CONTEXT``,
``OUSAST_REGISTRY``, ``OUSAST_MEMORY``, ``OUSAST_ROUTER_URL``, ...), a pool by ``OUSAST_POOL_<NAME>_<FIELD>``.
A profile holds no secret: a key whose name ends in ``key``, ``secret``, ``token`` or ``password`` is refused, and so
is any unknown key, an ``exec`` outside ``local``/``remote``, and an images file whose references are not pinned by
digest (Agent Substrate rejects tags). Provider and store credentials stay in the environment or ``.env``.
``python -m openultrasast.plane.profile --print FIELD`` prints one value for shell scripts (``ops/ax/up.sh``):
``registry``, ``kube_context``, ``images.runner``, ``pools.default.replicas``, ...
"""
from __future__ import annotations
import json
import os
import re
import sys
import tomllib
from collections.abc import Mapping
from dataclasses import dataclass, field, fields
from pathlib import Path
from typing import Any
__all__ = ["DEFAULT_PROFILE", "PROFILES_DIR", "PlaneProfile", "Pool", "ProfileError", "load_images", "load_profile", "profiles_dir"]
DEFAULT_PROFILE = "kind"
PROFILES_DIR = Path("ops/k8s/profiles")
EXEC_MODES = ("local", "remote")
_DIGEST = re.compile(r"\S+@sha256:[0-9a-f]{64}")
_SECRET_KEY = re.compile(r"(key|secret|token|password)$", re.IGNORECASE)
_SECRET_VALUE = re.compile(r"(sk-[A-Za-z0-9]{8,}|AKIA[0-9A-Z]{12,}|[A-Za-z0-9+/]{40,}={0,2}$)")
_ENV_PREFIX = "OUSAST_"
class ProfileError(ValueError):
"""A profile that cannot be used; the message names the file and the field."""
@dataclass(frozen=True)
class Pool:
"""One WorkerPool's size: how many warm workers, and each worker's CPU and memory limit."""
replicas: int
cpu: str
memory: str
@dataclass(frozen=True)
class PlaneProfile:
"""The values of one cluster. ``images`` is resolved against the profile file's directory."""
name: str
path: Path
exec: str = "local"
kube_context: str = ""
registry: str = ""
images: Path = Path("images.json")
router_url: str = ""
ax_server: str = ""
memory: str = ""
atespace: str = "default"
snapshots_bucket: str = ""
image_pull_secret: str = ""
pools: Mapping[str, Pool] = field(default_factory=dict)
def load_images(self) -> dict[str, str]:
"""The digest-pinned image references of ``images`` (``{"runner": ..., "engine": ...}``)."""
return load_images(self.images)
def addresses(self) -> dict[str, str]:
"""Every configured address, for ``ousast plane doctor`` (Req 1.2); an empty router or ax-server address
means the Kubernetes API tunnel."""
return {
"exec": self.exec,
"kube_context": self.kube_context,
"registry": self.registry,
"images": str(self.images),
"router_url": self.router_url or "(tunnel through the kube context)",
"ax_server": self.ax_server or "(tunnel through the kube context)",
"memory": self.memory,
"atespace": self.atespace,
"snapshots_bucket": self.snapshots_bucket,
"image_pull_secret": self.image_pull_secret or "(none)",
}
_SCALARS = {f.name: str(f.type) for f in fields(PlaneProfile) if f.name not in ("name", "path", "pools")}
_POOL_FIELDS = {f.name: str(f.type) for f in fields(Pool)}
def profiles_dir() -> Path:
"""``ops/k8s/profiles`` of this checkout (``OUSAST_PLANE_PROFILES`` overrides; an installed package falls back
to the working directory)."""
override = os.environ.get("OUSAST_PLANE_PROFILES")
if override:
return Path(override)
checkout = Path(__file__).resolve().parents[3] / PROFILES_DIR
return checkout if checkout.is_dir() else PROFILES_DIR
def load_images(path: Path) -> dict[str, str]:
"""``{"runner": "<registry>/<name>@sha256:<64 hex>", ...}`` from a JSON images file; every value must be
digest-pinned. A one-line file holding a single reference (``ops/ax/up.sh`` before this profile) reads as
``{"runner": <line>}``."""
try:
text = path.read_text(encoding="utf-8").strip()
except OSError as exc:
raise ProfileError(f"images file {path}: {exc.strerror or exc}") from None
try:
loaded: Any = json.loads(text) if text.startswith("{") else {"runner": text}
except ValueError as exc:
raise ProfileError(f"images file {path}: not JSON ({exc})") from None
if not isinstance(loaded, dict) or not loaded or not all(isinstance(k, str) and isinstance(v, str) for k, v in loaded.items()):
raise ProfileError(f"images file {path}: expected an object of image names to references")
for name, reference in loaded.items():
if not _DIGEST.fullmatch(reference):
raise ProfileError(f"images file {path}: {name} is not digest-pinned ({reference!r}); Agent Substrate rejects tags")
return dict(loaded)
def _reject_secret(where: str, key: str, value: object, declared: bool = False) -> None:
"""A key that is not a declared field and is named like a credential, or any value shaped like one, is refused.
``image_pull_secret`` is declared: its value is the *name* of a Kubernetes Secret, never its content."""
if not declared and _SECRET_KEY.search(key):
raise ProfileError(f"{where}: {key} looks like a credential; profiles hold no secret (use the environment or .env)")
if isinstance(value, str) and _SECRET_VALUE.search(value):
raise ProfileError(f"{where}: the value of {key} looks like a credential; profiles hold no secret")
def _coerce(where: str, key: str, value: object, kind: str) -> Any:
if kind == "int":
if isinstance(value, bool) or not isinstance(value, int):
raise ProfileError(f"{where}: {key} must be an integer, got {value!r}")
return value
if kind == "Path":
if not isinstance(value, str) or not value:
raise ProfileError(f"{where}: {key} must be a file name")
return Path(value)
if not isinstance(value, str):
raise ProfileError(f"{where}: {key} must be a string, got {value!r}")
return value
def _pools(where: str, raw: object) -> dict[str, Pool]:
if not isinstance(raw, dict):
raise ProfileError(f"{where}: pools must be a table of pool names")
out: dict[str, Pool] = {}
for name, spec in raw.items():
if not isinstance(spec, dict):
raise ProfileError(f"{where}: pools.{name} must be a table")
unknown = sorted(set(spec) - set(_POOL_FIELDS))
if unknown:
raise ProfileError(f"{where}: pools.{name} has unknown keys: {', '.join(unknown)}")
missing = sorted(set(_POOL_FIELDS) - set(spec))
if missing:
raise ProfileError(f"{where}: pools.{name} lacks {', '.join(missing)}")
for key, value in spec.items():
_reject_secret(where, f"pools.{name}.{key}", value, declared=True)
out[str(name)] = Pool(**{k: _coerce(where, f"pools.{name}.{k}", spec[k], _POOL_FIELDS[k]) for k in _POOL_FIELDS})
return out
def _env_overrides(environ: Mapping[str, str], where: str, values: dict[str, Any], pools: dict[str, Pool]) -> None:
for name, kind in _SCALARS.items():
raw = environ.get(_ENV_PREFIX + name.upper())
if raw is not None and raw != "":
value: Any = raw
if kind == "int":
try:
value = int(raw)
except ValueError:
raise ProfileError(f"{where}: {_ENV_PREFIX}{name.upper()} must be an integer, got {raw!r}") from None
values[name] = _coerce(where, _ENV_PREFIX + name.upper(), value, kind)
pattern = re.compile(rf"^{_ENV_PREFIX}POOL_([A-Z0-9]+)_(REPLICAS|CPU|MEMORY)$")
for variable, raw in environ.items():
found = pattern.match(variable)
if not found or raw == "":
continue
pool_name, key = found.group(1).lower(), found.group(2).lower()
current = pools.get(pool_name)
if current is None:
raise ProfileError(f"{where}: {variable} names a pool the profile does not declare ({pool_name})")
value = int(raw) if key == "replicas" and raw.isdigit() else raw
pools[pool_name] = Pool(**{**vars(current), key: _coerce(where, variable, value, _POOL_FIELDS[key])})
def load_profile(name: str | None = None, environ: Mapping[str, str] | None = None) -> PlaneProfile:
"""The profile ``name`` (a name under :func:`profiles_dir` or a path to a ``.toml``; default
``OUSAST_PLANE_PROFILE``, else ``kind``), with the environment's overrides applied; :class:`ProfileError`
names what is wrong."""
env = os.environ if environ is None else environ
chosen = name or env.get("OUSAST_PLANE_PROFILE") or DEFAULT_PROFILE
path = Path(chosen) if chosen.endswith(".toml") or "/" in chosen else profiles_dir() / f"{chosen}.toml"
where = str(path)
try:
raw = tomllib.loads(path.read_text(encoding="utf-8"))
except OSError as exc:
raise ProfileError(f"profile {where}: {exc.strerror or exc}") from None
except tomllib.TOMLDecodeError as exc:
raise ProfileError(f"profile {where}: {exc}") from None
for key, value in raw.items(): # a credential-shaped key is named as such before it is merely unknown
_reject_secret(where, key, value, declared=key in _SCALARS)
unknown = sorted(set(raw) - set(_SCALARS) - {"pools"})
if unknown:
raise ProfileError(f"{where}: unknown keys: {', '.join(unknown)}")
values: dict[str, Any] = {}
for key, value in raw.items():
if key != "pools":
values[key] = _coerce(where, key, value, _SCALARS[key])
pools = _pools(where, raw.get("pools", {}))
_env_overrides(env, where, values, pools)
if values.get("exec", "local") not in EXEC_MODES:
raise ProfileError(f"{where}: exec must be one of {', '.join(EXEC_MODES)}, got {values.get('exec')!r}")
images = Path(values.get("images", "images.json"))
if not images.is_absolute():
images = path.resolve().parent / images
values["images"] = images
if images.is_file():
load_images(images) # a committed file must be pinned; a file the release has not published yet is doctor's finding
return PlaneProfile(name=path.stem, path=path, pools=pools, **values)
def _lookup(profile: PlaneProfile, dotted: str) -> str:
head, _, rest = dotted.partition(".")
if head == "images" and rest:
images = profile.load_images()
if rest not in images:
raise ProfileError(f"{profile.images}: no image named {rest!r} (have {', '.join(sorted(images))})")
return images[rest]
if head == "pools":
pool_name, _, key = rest.partition(".")
if pool_name not in profile.pools or key not in _POOL_FIELDS:
raise ProfileError(f"{profile.path}: no such pool field {dotted!r}")
return str(getattr(profile.pools[pool_name], key))
if head not in _SCALARS or rest:
raise ProfileError(f"{profile.path}: no such field {dotted!r}")
return str(getattr(profile, head))
def main(argv: list[str] | None = None) -> int:
"""``python -m openultrasast.plane.profile [--profile NAME] --print FIELD [FIELD...]``: one value per line."""
import argparse
parser = argparse.ArgumentParser(prog="openultrasast.plane.profile", description=main.__doc__)
parser.add_argument("--profile", help="profile name or path (default OUSAST_PLANE_PROFILE, else kind)")
parser.add_argument("--print", dest="fields", nargs="+", required=True, metavar="FIELD")
args = parser.parse_args(argv)
try:
profile = load_profile(args.profile)
for dotted in args.fields:
print(_lookup(profile, dotted))
except ProfileError as exc:
print(f"profile: {exc}", file=sys.stderr)
return 2
return 0
if __name__ == "__main__":
sys.exit(main())
src/openultrasast/plane/reconciler.py
+113 -41
View File
@@ -1,23 +1,26 @@
"""ax-backed reconciler of a ``Run``: submission, resume, start, completion by delivery (ai-service-plane Req 3, 4, 7).
ax executes every task on this host's kind cluster; nothing here runs a task locally or knows a prompt, a score or
a model call. Per ready task: one rendered YAML file (the Task with its env extended, its Workspaces, the bound
Model), ``ax apply -f``, ``ax resume task`` (retried on DeadlineExceeded/Unavailable -- a new image's first resume
waits for Agent Substrate's golden snapshot -- for up to ``OUSAST_RESUME_TIMEOUT`` s, 900), and once ax says
ax executes every task on the profile's cluster (``profile.py``: kube context, registry, store); nothing here runs a
task locally or knows a prompt, a score or a model call. Per ready task: one rendered YAML file (the Task with its
env extended, its Workspaces, the bound Model), ``ax apply -f``, ``ax resume task`` (retried on
DeadlineExceeded/Unavailable -- a new image's first resume waits for Agent Substrate's golden snapshot -- for up to
``OUSAST_RESUME_TIMEOUT`` s, 900), and once ax says
``Running`` one start request through Substrate's router (``router.py``, Req 4.5) carrying the bound Model's
credential from this process's environment, never from a file (Req 4.6). Completion is the runner's artifact
delivery (ax has no Completed phase); ``Failed``, the task vanishing, a refused start or ``OUSAST_TASK_TIMEOUT`` s
(7200) without delivery fail it; ``ax delete task`` follows.
credential from this process's environment, never from a file (Req 4.6), and the presigned store URLs of
``delivery.py`` (plane-on-kubernetes design section 3). Completion is the runner's output tar appearing in the
memory store (ax has no Completed phase), which the reconciler polls, downloads, extracts and re-publishes per
declared output; ``Failed``, the task vanishing, a refused start or ``OUSAST_TASK_TIMEOUT`` s (7200) without
delivery fail it; ``ax delete task`` follows. ``OUSAST_DELIVERY=receiver`` keeps the host receiver of the kind
profile for group 2 only (task 2.6 removes it).
ax rejects unknown fields, so rendered documents carry only ax's fields; ``openultrasast.io/model`` reaches the
task as ``OUSAST_MODEL`` plus ``OUSAST_MODEL_PARAMS`` (Req 7.1), each Workspace's pin annotation as
``OUSAST_GIT_PINS``. Inputs other tasks produced are never rendered: ``OUSAST_INPUTS`` maps each input's name to
``<producer>/<artifact>``, which the runner fetches after the start from the receiver's ``GET /inputs/...`` to
``OUSAST_INPUT_<NAME>`` under ``/workspace/.ousast-in/<task>`` (the receiver serves a running task its declared
inputs only). The receiver takes the runner's tar (``X-Ousast-Run``, ``X-Ousast-Task``) on
``OUSAST_ARTIFACT_PORT`` (18090), reached by its Service name through the egress gateway; between ``ax apply`` and
resume the task's egress policy is written (``egress.py``).
State: ``~/ousast-results/plane/<run>/`` (``OUSAST_RESULTS``), polled every ``OUSAST_POLL_SECONDS`` (3).
``<producer>/<artifact>``, fetched after the start by the presigned GET of the start body to ``OUSAST_INPUT_<NAME>``
under ``/workspace/.ousast-in/<task>``. Between ``ax apply`` and resume the task's egress policy is written
(``egress.py``). State: ``~/ousast-results/plane/<run>/`` (``OUSAST_RESULTS``), polled every ``OUSAST_POLL_SECONDS``
(3), mirrored to ``runs/<run>/state.json`` on the store after every change and seeded from there by a rerun with no
local state (design section 2); the store is opened first and required, there is no fallback.
"""
from __future__ import annotations
@@ -41,9 +44,12 @@ from urllib.parse import urlsplit
import yaml
from .delivery import Delivery, DeliveryError, declared
from .doctor import doctor
from .egress import RECEIVER_PORT, Egress, Receiver, policy_for
from .manifests import AX_API_VERSION, Manifests, Model, Run, RunTask, Task, Workspace, load_manifests
from .memory import open_store
from .profile import PlaneProfile, load_profile
from .router import Router, credentials, open_router
MODEL_ANNOTATION = "openultrasast.io/model"
@@ -136,15 +142,16 @@ def _model_doc(model: Model) -> dict[str, Any]:
return _doc("Model", model.metadata.name, spec, model.metadata.atespace)
def render_task(run: Run, entry: RunTask, manifests: Manifests, artifact_url: str, dial: str | None = None) -> list[dict[str, Any]]:
def render_task(run: Run, entry: RunTask, manifests: Manifests, artifact_url: str = "", dial: str | None = None) -> list[dict[str, Any]]:
"""The documents ``ax apply`` receives for one Run task: its Model, its Workspaces, the Task last. Task-produced
inputs are never rendered: ax puts every bound Workspace into one env value, which Substrate caps at 32768
characters, so the runner fetches them from the receiver (``OUSAST_INPUTS``) after the start."""
inputs are never rendered (Substrate caps one env value at 32768 characters): the runner fetches them
(``OUSAST_INPUTS``) after the start by presigned URL, or from the receiver when ``artifact_url`` is given."""
task: Task = manifests.tasks[entry.task]
ax_name = _ax_name(run.metadata.name, entry.name, limit=TASK_NAME_LIMIT)
env = {e.name: e.value for e in task.env}
env.update(OUSAST_RUN=run.metadata.name, OUSAST_TASK=entry.name, OUSAST_OUTPUT_DIR=f"{OUTPUT_ROOT}/{entry.name}")
env.update(OUSAST_ARTIFACT_URL=artifact_url, **({"OUSAST_ARTIFACT_DIAL": dial} if dial else {}))
if artifact_url: # receiver mode only (task 2.6 removes it): the store path carries no URL in the env
env.update(OUSAST_ARTIFACT_URL=artifact_url, **({"OUSAST_ARTIFACT_DIAL": dial} if dial else {}))
budget = _fields(entry.budget) if entry.budget else {}
env.update({f"OUSAST_BUDGET_{k.upper()}": str(v) for k, v in budget.items()})
inputs = {re.sub(r"[^A-Z0-9]+", "_", name.upper()): ref for name, ref in entry.inputs.items()}
@@ -224,12 +231,25 @@ class _State:
path: Path
data: dict[str, Any]
lock: threading.Lock = field(default_factory=threading.Lock)
mirror: Callable[[Mapping[str, Any]], None] | None = None # the store copy; a failure is reported, never fatal
def set(self, task: str, **fields: object) -> None:
with self.lock:
self.data["tasks"].setdefault(task, {"status": "pending", "started": None, "finished": None, "usd": None, "model": None})
self.data["tasks"][task].update(fields)
_write_json(self.path, self.data)
self._flush()
def flush(self) -> None:
with self.lock:
self._flush()
def _flush(self) -> None:
_write_json(self.path, self.data)
if self.mirror is not None:
try:
self.mirror(self.data)
except Exception as exc: # noqa: BLE001 - the local file is written; the mirror is reported, the run goes on
print(f"state mirror to the store failed: {exc}", file=sys.stderr)
def status(self, task: str) -> str:
return str(self.data["tasks"].get(task, {}).get("status", "pending"))
@@ -313,25 +333,46 @@ def _await(ax: Ax, name: str, delivered: Callable[[], bool], start: Callable[[],
def _execute(
ax: Ax, egress: Egress, run: Run, entry: RunTask, manifests: Manifests, base: Path, receiver: Receiver, router: Router
ax: Ax,
egress: Egress,
run: Run,
entry: RunTask,
manifests: Manifests,
base: Path,
delivery: Delivery,
router: Router,
receiver: Receiver | None = None,
) -> tuple[str, str]:
ax_name, task = _ax_name(run.metadata.name, entry.name, limit=TASK_NAME_LIMIT), manifests.tasks[entry.task]
atespace, manifest = task.metadata.atespace or "default", base / entry.name / "task.yaml"
manifest.parent.mkdir(parents=True, exist_ok=True)
docs = render_task(run, entry, manifests, receiver.url, receiver.dial)
docs = render_task(run, entry, manifests, receiver.url if receiver else "", receiver.dial if receiver else None)
manifest.write_text(yaml.safe_dump_all(docs, sort_keys=False), encoding="utf-8")
model = manifests.models.get(task.metadata.annotations.get(MODEL_ANNOTATION) or "")
policy = policy_for(task, [manifests.workspaces[b.name] for b in task.workspaces], model, urlsplit(receiver.url).hostname)
workspaces = [manifests.workspaces[b.name] for b in task.workspaces]
body = None if receiver else delivery.body(entry.name, entry.inputs) # presigned URLs ride in the start body
store_host, receiver_host = (None, urlsplit(receiver.url).hostname) if receiver else (delivery.host, None)
if receiver is None:
delivery.reset(entry.name) # a previous attempt's tar must not complete this one
def delivered() -> bool:
return entry.name in receiver.delivered if receiver is not None else delivery.delivered(entry.name)
try: # Req 4.6: the bound Model's credential is read here and only ever sent in the start request
start = router.starter(atespace, ax_name, run.metadata.name, credentials(model))
policy = policy_for(task, workspaces, model, store_host, receiver_host)
start = router.starter(atespace, ax_name, run.metadata.name, credentials(model), body)
ax.apply(manifest) # creates the Substrate actor; its egress is denied until the policy exists
egress.apply(ax_name, atespace, policy)
egress.settle()
except RuntimeError as exc: # StartError included: a Model-bound task whose secret is unset
except (RuntimeError, ValueError) as exc: # StartError (a Model's secret unset) and a policy that cannot be granted
ax.delete(ax_name)
return "failed", str(exc)
try:
failure = _await(ax, ax_name, lambda: entry.name in receiver.delivered, start)
failure = _await(ax, ax_name, delivered, start)
if failure is None and receiver is None:
delivery.collect(entry.name, declared(run)[entry.name]) # download, extract, re-publish what is declared
except DeliveryError as exc:
failure = str(exc)
finally:
ax.delete(ax_name)
leftover = egress.delete(ax_name, atespace) # the policy goes with its actor; a survivor is reported
@@ -340,20 +381,44 @@ def _execute(
return ("failed", failure) if failure else outcome_of(_read_json(base / entry.name / "summary.json"))
def run(run_manifest: Path, *, workers: int = 1, ax: str = "ax", results_root: Path | None = None, kubectl_ate: str | None = None) -> str:
"""Execute a Run on ax and return its status: ``done``, ``unfinished`` or ``failed`` (Req 3.1-3.3, 2.2)."""
def run(
run_manifest: Path,
*,
workers: int = 1,
ax: str = "ax",
results_root: Path | None = None,
kubectl_ate: str | None = None,
profile: PlaneProfile | None = None,
) -> str:
"""Execute a Run on ax and return its status: ``done``, ``unfinished`` or ``failed`` (Req 3.1-3.3, 2.2). Every
address comes from ``profile`` (default: the one ``OUSAST_PLANE_PROFILE`` names)."""
profile = profile or load_profile()
run_spec, manifests = load_run(Path(run_manifest))
base = run_dir(run_spec.metadata.name, results_root)
base.mkdir(parents=True, exist_ok=True)
store = open_store(profile.memory) # first, before the lock and before ax sees anything: no fallback
delivery = Delivery(store, run_spec.metadata.name, base)
if not (base / "state.json").exists():
seeded = delivery.load_state() # a rerun from another machine or an empty results root
if seeded is not None:
_write_json(base / "state.json", seeded)
lock = _acquire_lock(base)
state = _State(base / "state.json", _read_json(base / "state.json") or {"run": run_spec.metadata.name, "started": _now(), "tasks": {}})
port = os.environ.get("OUSAST_ARTIFACT_PORT") or (0 if os.environ.get("OUSAST_ARTIFACT_HOST") else RECEIVER_PORT)
receiver = Receiver(run_spec.metadata.name, base, int(port)) # the receiver Service targets RECEIVER_PORT
threading.Thread(target=receiver.serve_forever, daemon=True).start()
cli, egress, finished = Ax(ax), Egress(kubectl_ate), queue.Queue[tuple[str, str, str]]()
state = _State(
base / "state.json",
_read_json(base / "state.json") or {"run": run_spec.metadata.name, "started": _now(), "tasks": {}},
mirror=delivery.save_state,
)
receiver: Receiver | None = None
if os.environ.get("OUSAST_DELIVERY") == "receiver": # the kind profile's host receiver, group 2 only (2.6 removes it)
port = os.environ.get("OUSAST_ARTIFACT_PORT") or (0 if os.environ.get("OUSAST_ARTIFACT_HOST") else RECEIVER_PORT)
receiver = Receiver(run_spec.metadata.name, base, int(port), profile.kube_context) # the Service targets RECEIVER_PORT
threading.Thread(target=receiver.serve_forever, daemon=True).start()
cli, egress, finished = Ax(ax), Egress(kubectl_ate, profile.kube_context), queue.Queue[tuple[str, str, str]]()
pending = [t for t in run_spec.tasks if state.status(t.name) != "done"]
for entry in run_spec.tasks:
state.set(entry.name, **({"status": "pending"} if entry in pending else {}))
if entry not in pending:
delivery.publish(entry.name, declared(run_spec)[entry.name]) # a seeded or earlier task's artifacts reach its consumers
running: dict[str, RunTask] = {}
halted = False
@@ -361,10 +426,14 @@ def run(run_manifest: Path, *, workers: int = 1, ax: str = "ax", results_root: P
if state.status(entry.name) in ("running", "suspended"):
cli.delete(_ax_name(run_spec.metadata.name, entry.name, limit=TASK_NAME_LIMIT)) # an interrupted attempt; ax must not keep it
state.set(entry.name, status="running", started=_now(), finished=None)
finished.put((entry.name, *_execute(cli, egress, run_spec, entry, manifests, base, receiver, router)))
try:
outcome = _execute(cli, egress, run_spec, entry, manifests, base, delivery, router, receiver)
except Exception as exc: # noqa: BLE001 - a worker that dies silently would hang the run on the queue
outcome = ("failed", f"{type(exc).__name__}: {exc}")
finished.put((entry.name, *outcome))
try:
with open_router() as router: # one port-forward to Substrate's router for the whole run
with open_router(profile.kube_context, url=profile.router_url or None) as router: # one tunnel per run (ax-tunnel)
while pending or running:
blocked = {t.serialize for t in running.values() if t.serialize}
for entry in list(pending):
@@ -375,29 +444,32 @@ def run(run_manifest: Path, *, workers: int = 1, ax: str = "ax", results_root: P
pending.remove(entry)
running[entry.name] = entry
blocked.add(entry.serialize or "")
with receiver.lock: # the receiver serves a running task its declared inputs, nothing else
receiver.running.add(entry.name)
receiver.inputs[entry.name] = frozenset(entry.inputs.values())
if receiver is not None:
with receiver.lock: # the receiver serves a running task its declared inputs, nothing else
receiver.running.add(entry.name)
receiver.inputs[entry.name] = frozenset(entry.inputs.values())
threading.Thread(target=worker, args=(entry,), daemon=True).start()
if not running:
break # nothing can start: a producer stopped short
name, status, reason = finished.get()
del running[name]
with receiver.lock:
receiver.running.discard(name)
receiver.inputs.pop(name, None)
if receiver is not None:
with receiver.lock:
receiver.running.discard(name)
receiver.inputs.pop(name, None)
summary = _read_json(base / name / "summary.json") or {}
state.set(name, status=status, finished=_now(), usd=summary.get("usd"), model=summary.get("model"), reason=reason or None)
halted = halted or status == "failed"
if status == "unfinished":
pending = [t for t in pending if name not in t.producers]
finally:
receiver.shutdown()
if receiver is not None:
receiver.shutdown()
lock.unlink(missing_ok=True)
statuses = [state.status(t.name) for t in run_spec.tasks]
result = "failed" if "failed" in statuses else "done" if all(s == "done" for s in statuses) else "unfinished"
state.data.update(status=result, finished=_now())
_write_json(state.path, state.data)
state.flush()
return result
src/openultrasast/plane/router.py
+26 -11
View File
@@ -28,6 +28,7 @@ import time
import urllib.error
import urllib.request
from collections.abc import Callable, Iterable, Iterator, Mapping
from typing import Any
from .manifests import Model
@@ -77,7 +78,9 @@ class Router:
def __init__(self, url: str) -> None:
self.url = url.rstrip("/")
def starter(self, atespace: str, task: str, run: str, credentials: Mapping[str, str]) -> Callable[[], str | None]:
def starter(
self, atespace: str, task: str, run: str, credentials: Mapping[str, str], delivery: Mapping[str, Any] | None = None
) -> Callable[[], str | None]:
""":meth:`start_task` bound to one task, returning the refusal text instead of raising (None once accepted).
The reconciler calls it again after a re-resume, since a resumed actor may have rebooted and wait for a
@@ -90,20 +93,30 @@ class Router:
nonlocal calls
calls += 1
try:
self.start_task(atespace, task, run, credentials)
self.start_task(atespace, task, run, credentials, delivery)
except StartError as exc:
return None if calls > 1 and ALREADY_STARTED in str(exc) else str(exc)
return None
return start
def start_task(self, atespace: str, task: str, run: str, credentials: Mapping[str, str]) -> None:
"""Start ``<atespace>/<task>``'s command once; :class:`StartError` on 409, another 4xx, or the timeout."""
body = json.dumps({"run": run, "task": task, "credentials": dict(credentials)}).encode("utf-8")
def start_task(
self, atespace: str, task: str, run: str, credentials: Mapping[str, str], delivery: Mapping[str, Any] | None = None
) -> None:
"""Start ``<atespace>/<task>``'s command once; :class:`StartError` on 409, another 4xx, or the timeout.
``delivery`` (``{put, inputs}`` presigned URLs, ``delivery.py``) rides in the body like the credentials and is
redacted like them."""
payload: dict[str, Any] = {"run": run, "task": task, "credentials": dict(credentials)}
if delivery:
payload["delivery"] = dict(delivery)
body = json.dumps(payload).encode("utf-8")
headers = {"Content-Type": "application/json", TARGET_HEADER: f"{atespace}/{task}"}
limit = float(os.environ.get("OUSAST_START_TIMEOUT") or 300)
deadline, backoff, last = time.monotonic() + limit, 0.5, "no attempt made"
secrets = list(credentials.values())
if delivery:
secrets.append(str(delivery.get("put") or ""))
secrets.extend(str(u) for u in (delivery.get("inputs") or {}).values())
while True:
request = urllib.request.Request(self.url + START_PATH, data=body, method="POST", headers=headers)
try:
@@ -131,18 +144,20 @@ def _drain(stream: Iterable[str], lines: queue.Queue[str | None]) -> None:
@contextlib.contextmanager
def open_router(context: str | None = None, *, kubectl: str = "kubectl", ready_timeout: float = 30.0) -> Iterator[Router]:
def open_router(context: str, *, url: str | None = None, kubectl: str = "kubectl", ready_timeout: float = 30.0) -> Iterator[Router]:
"""A :class:`Router` for the duration of the block.
``OUSAST_ROUTER_URL`` set: that URL, nothing started. Otherwise ``kubectl port-forward svc/atenet-router :80``
in ``ate-system`` of ``context`` (default ``kind-$KIND_CLUSTER_NAME``, ``kind-ousast``), the local port read
from kubectl's ``Forwarding from 127.0.0.1:NNNN`` line; on exit that child process is terminated.
``url`` (the profile's ``router_url``) or ``OUSAST_ROUTER_URL`` set: that URL, nothing started. Otherwise
``kubectl port-forward svc/atenet-router :80`` in ``ate-system`` of the profile's ``context`` (ax-tunnel: the
Kubernetes API is the one authenticated door), the local port read from kubectl's ``Forwarding from
127.0.0.1:NNNN`` line; on exit that child process is terminated. No context is ever built here.
"""
override = os.environ.get("OUSAST_ROUTER_URL")
override = url or os.environ.get("OUSAST_ROUTER_URL")
if override:
yield Router(override)
return
context = context or "kind-" + (os.environ.get("KIND_CLUSTER_NAME") or "ousast")
if not context:
raise StartError("open_router needs the profile's kube context (or router_url)")
argv = [kubectl, "--context", context, "-n", NAMESPACE, "port-forward", SERVICE, ":80"]
proc = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
try:
src/openultrasast/plane/runner.py
+111 -34
View File
@@ -10,18 +10,23 @@ not (ax docs/runner.md).
The command never starts by itself (Requirements 4.5, 4.6). Agent Substrate boots every new ActorTemplate once as a
"golden" actor to snapshot it, with this image and the same ``AX_TASK_YAML``/``AX_WORKSPACES_YAML``/env, and inside
the sandbox nothing tells that boot from the real one. So after the workspaces are ready the runner waits for
``POST /ousast/v1/start`` with ``{"run", "task", "credentials"}``, which the reconciler sends through Substrate's
``POST /ousast/v1/start`` with ``{"run", "task", "credentials", "delivery"}``, which the reconciler sends through Substrate's
router to the task's actor only (``plane/router.py``). ``run`` and ``task`` must match ``OUSAST_RUN`` and the Task's
name (409 otherwise); one request per boot is accepted (202), later ones are 409, and a boot whose completion marker
records a delivered run refuses every request (409 with the reason). Before the workspaces are ready the answer is
503, which the reconciler retries. The credentials live in this process's memory and go into the command's
environment only; they are never logged or written, and their values are redacted from the command's echoed stderr.
``delivery`` (plane-on-kubernetes design section 3) is ``{"put": <presigned PUT URL for the output tar>, "inputs":
{"<NAME>": <presigned GET URL>}}``: a presigned URL is a bearer capability, so it travels where the credentials do.
Each URL must be ``https`` to a hostname (never an address; ``AX_RUNNER_INSECURE_DELIVERY=1`` admits ``http`` in unit
tests only), the answer never echoes one, and no URL is logged.
The command runs as a child process, ``python -m openultrasast.plane.tasks.<spec.command[0]> <spec.command[1:]>``,
in its own process group, with the first workspace as its working directory and ``AX_METADATA_URL`` pointing at
this server. ax has no artifact channel and never reports that a command finished, so after the child exits the
runner reads ``summary.json`` (a crash without one becomes ``failed`` with the exit code and the stderr tail),
posts the whole ``OUSAST_OUTPUT_DIR`` as a tar to ``OUSAST_ARTIFACT_URL`` -- that delivery is the task's
PUTs the whole ``OUSAST_OUTPUT_DIR`` as a tar to ``delivery.put`` (or POSTs it to ``OUSAST_ARTIFACT_URL``, the receiver
path kept until task 2.6 removes it) -- that delivery is the task's
completion (Requirement 4.4); a failed one is reported ``failed``, never done -- and writes the completion marker
``<OUSAST_STATE_DIR>/<task>.done.json`` (``exit``, ``status``, ``delivered``). Then it keeps serving until SIGTERM,
as ax requires of PID 1. A boot that finds the marker never reruns the command (a resumed actor must not repeat
@@ -38,6 +43,7 @@ Environment the runner reads, all optional except the two ax variables:
AX_RUNNER_AUTOSTART "1" runs the command without waiting for a start request (unit tests only;
required with AX_RUNNER_HTTP=0, where no request could arrive)
AX_RUNNER_TASK_PACKAGE package the command's module is looked up in (unit tests only)
AX_RUNNER_INSECURE_DELIVERY "1" admits http delivery URLs to a hostname (unit tests only; the image never sets it)
OUSAST_STATE_DIR completion markers, default ``/workspace/.ousast-state`` (the durable volume)
OUSAST_TERM_GRACE seconds between SIGTERM and SIGKILL of the command's group, default 10
OUSAST_CASE_CACHE a directory of git checkouts (``benchmarks/independent`` cache layout)
@@ -47,8 +53,9 @@ Environment the runner reads, all optional except the two ax variables:
``Host`` (the egress gateway decides on and resolves that name)
OUSAST_DELIVERY_BACKOFF seconds before the second delivery attempt, doubled per retry
OUSAST_INPUTS JSON ``{"<NAME>": "<producer>/<artifact>"}``: after the start and before the
command each is fetched from ``<OUSAST_ARTIFACT_URL>/inputs/<ref>`` (same
dial, headers and retries as delivery) to ``OUSAST_INPUT_<NAME>``; a failed
command each is fetched from ``delivery.inputs[NAME]`` (else from
``<OUSAST_ARTIFACT_URL>/inputs/<ref>`` with the receiver's dial and headers)
to ``OUSAST_INPUT_<NAME>``, with the same retries as delivery; a failed
fetch delivers a failed summary naming the input, the command never runs
The runner exports ``OUSAST_WORKSPACE_DIR`` (the first bound workspace) when the Task's env leaves it unset,
@@ -60,6 +67,7 @@ from __future__ import annotations
import contextlib
import http.client
import io
import ipaddress
import json
import logging
import os
@@ -76,7 +84,7 @@ import urllib.parse
import urllib.request
from collections import deque
from collections.abc import Mapping, MutableMapping, Sequence
from dataclasses import dataclass
from dataclasses import dataclass, field
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
@@ -206,14 +214,58 @@ _CREDENTIAL_NAME = re.compile(r"[A-Z][A-Z0-9_]*")
_RESERVED_PREFIXES = ("AX_", "OUSAST_") # the runner's own contract; a credential must not redirect it
@dataclass(frozen=True)
class DeliveryTargets:
"""Where this task PUTs its output tar and GETs each input (presigned URLs from the start request)."""
put: str | None = None
inputs: Mapping[str, str] = field(default_factory=dict)
def __repr__(self) -> str: # a presigned URL is a credential: never print one
return f"DeliveryTargets(put={'yes' if self.put else 'no'}, inputs={sorted(self.inputs)})"
@dataclass(frozen=True)
class Start:
"""An accepted start request: the credentials go to the command's environment and nowhere else."""
"""An accepted start request: the credentials go to the command's environment and nowhere else; the delivery
targets go to the artifact PUT and the input GETs and nowhere else."""
credentials: Mapping[str, str]
delivery: DeliveryTargets | None = None
def __repr__(self) -> str: # never print a value by accident
return f"Start(credentials={sorted(self.credentials)})"
return f"Start(credentials={sorted(self.credentials)}, delivery={self.delivery!r})"
def _check_url(url: object, where: str, insecure: bool) -> str:
"""``url`` is a string, ``https`` (``http`` only when ``insecure``) to a hostname that is not an address, without
userinfo; the error names the field and never echoes the value."""
if not isinstance(url, str):
raise ValueError(f"{where} must be a URL string")
parts = urlsplit(url)
allowed = ("https", "http") if insecure else ("https",)
host = parts.hostname or ""
if parts.scheme not in allowed or not host or parts.username is not None or parts.password is not None:
raise ValueError(f"{where} must be an https URL to a hostname")
try:
ipaddress.ip_address(host.strip("[]"))
except ValueError:
return url
raise ValueError(f"{where} must name a host, not an address")
def parse_delivery(raw: object, insecure: bool = False) -> DeliveryTargets | None:
"""The start body's ``delivery`` field validated: None when absent; :class:`ValueError` names the bad field."""
if raw is None:
return None
if not isinstance(raw, dict) or not set(raw) <= {"put", "inputs"}:
raise ValueError("delivery must be an object with put and/or inputs")
put = _check_url(raw["put"], "delivery.put", insecure) if raw.get("put") is not None else None
inputs_raw = raw.get("inputs") or {}
if not isinstance(inputs_raw, dict) or not all(isinstance(k, str) and _CREDENTIAL_NAME.fullmatch(k) for k in inputs_raw):
raise ValueError("delivery.inputs must map ^[A-Z][A-Z0-9_]*$ names to URLs")
inputs = {name: _check_url(url, f"delivery.inputs.{name}", insecure) for name, url in inputs_raw.items()}
return DeliveryTargets(put, inputs)
class StartGate:
@@ -225,7 +277,8 @@ class StartGate:
logged, and no answer echoes it.
"""
def __init__(self) -> None:
def __init__(self, insecure: bool = False) -> None:
self._insecure = insecure # AX_RUNNER_INSECURE_DELIVERY=1: http delivery URLs (unit tests only)
self._lock = threading.Lock()
self._accepted = threading.Event()
self._expected: tuple[str, str] | None = None
@@ -267,10 +320,14 @@ class StartGate:
for k, v in creds.items()
):
return 400, "credentials must map ^[A-Z][A-Z0-9_]*$ names (not AX_/OUSAST_) to strings"
try:
delivery = parse_delivery(request.get("delivery"), self._insecure)
except ValueError as exc:
return 400, str(exc)
run, task = self._expected
if request.get("run") != run or request.get("task") != task:
return 409, f"this actor runs task {task!r} of run {run!r}"
self._start = Start(dict(creds))
self._start = Start(dict(creds), delivery)
self._accepted.set()
return 202, "started"
@@ -653,6 +710,12 @@ def _dialled(url: str, headers: Mapping[str, str], dial: str | None) -> tuple[st
return parts._replace(netloc=dial).geturl(), out
def safe_url(url: str) -> str:
"""``scheme://host/path`` of a URL: what a log line or an error may carry (a presigned query is a credential)."""
parts = urlsplit(url)
return f"{parts.scheme}://{parts.netloc}{parts.path}"
def deliver(
url: str,
payload: bytes,
@@ -661,15 +724,17 @@ def deliver(
attempts: int = DELIVERY_ATTEMPTS,
backoff: float = 1.0,
dial: str | None = None,
method: str = "POST",
) -> int:
"""POST ``payload`` to ``url`` with ``headers``; retry with doubling backoff; return the 2xx status.
"""Send ``payload`` to ``url`` (``POST`` to the receiver, ``PUT`` to a presigned store URL) with ``headers``; retry
with doubling backoff; return the 2xx status. Log lines and errors carry the URL without its query.
``dial`` (``host:port``): connect there instead and send the URL's host as ``Host``, as the actor must -- it
cannot resolve the receiver's cluster name, and the egress gateway decides on ``Host`` and resolves it itself.
"""
last, (target, headers) = "no attempt made", _dialled(url, headers, dial)
for attempt in range(1, attempts + 1):
request = urllib.request.Request(target, data=payload, method="POST", headers=headers)
request = urllib.request.Request(target, data=payload, method=method, headers=headers)
try:
with urllib.request.urlopen(request, timeout=60) as response: # noqa: S310 - the reconciler's URL
status = int(response.status)
@@ -680,10 +745,10 @@ def deliver(
last = f"HTTP {exc.code}"
except (urllib.error.URLError, http.client.HTTPException, OSError, ValueError) as exc:
last = f"{type(exc).__name__}: {exc}"
log.warning("artifact delivery attempt %d/%d to %s failed: %s", attempt, attempts, url, last)
log.warning("artifact delivery attempt %d/%d to %s failed: %s", attempt, attempts, safe_url(url), last)
if attempt < attempts:
time.sleep(backoff * (2 ** (attempt - 1)))
raise DeliveryError(f"{last} after {attempts} attempts to {url}")
raise DeliveryError(f"{last} after {attempts} attempts to {safe_url(url)}")
def fetch(
@@ -711,14 +776,15 @@ def fetch(
except (urllib.error.URLError, http.client.HTTPException, OSError, ValueError) as exc:
last = f"{type(exc).__name__}: {exc}"
partial.unlink(missing_ok=True)
log.warning("input fetch attempt %d/%d from %s failed: %s", attempt, attempts, url, last)
log.warning("input fetch attempt %d/%d from %s failed: %s", attempt, attempts, safe_url(url), last)
if attempt < attempts:
time.sleep(backoff * (2 ** (attempt - 1)))
raise DeliveryError(f"{last} after {attempts} attempts to {url}")
raise DeliveryError(f"{last} after {attempts} attempts to {safe_url(url)}")
def fetch_inputs(environ: Mapping[str, str]) -> list[Path]:
"""Fetch every ``OUSAST_INPUTS`` entry from the receiver to its ``OUSAST_INPUT_<NAME>`` path.
def fetch_inputs(environ: Mapping[str, str], delivery: DeliveryTargets | None = None) -> list[Path]:
"""Fetch every ``OUSAST_INPUTS`` entry to its ``OUSAST_INPUT_<NAME>`` path: from ``delivery.inputs[NAME]`` (a
presigned GET, no headers) when the start request carries one, else from the receiver.
:class:`RunnerError` names the input when one cannot be fetched: a task never runs with an input missing."""
raw = environ.get("OUSAST_INPUTS")
@@ -736,11 +802,15 @@ def fetch_inputs(environ: Mapping[str, str]) -> list[Path]:
fetched: list[Path] = []
for name, ref in sorted(inputs.items()):
dest = environ.get(f"OUSAST_INPUT_{name}")
if not url or not dest or not isinstance(ref, str):
raise RunnerError(f"input {name} ({ref}) not fetched: OUSAST_ARTIFACT_URL or OUSAST_INPUT_{name} is unset")
source = f"{url.rstrip('/')}/inputs/{urllib.parse.quote(ref)}"
presigned = delivery.inputs.get(name) if delivery is not None else None
if not dest or not isinstance(ref, str) or not (presigned or url):
raise RunnerError(f"input {name} ({ref}) not fetched: no delivery URL, OUSAST_ARTIFACT_URL or OUSAST_INPUT_{name}")
try:
size = fetch(source, headers, Path(dest), backoff=backoff, dial=environ.get("OUSAST_ARTIFACT_DIAL") or None)
if presigned:
size = fetch(presigned, {}, Path(dest), backoff=backoff)
else:
source = f"{str(url).rstrip('/')}/inputs/{urllib.parse.quote(ref)}"
size = fetch(source, headers, Path(dest), backoff=backoff, dial=environ.get("OUSAST_ARTIFACT_DIAL") or None)
except DeliveryError as exc:
raise RunnerError(f"input {name} ({ref}) not fetched: {exc}") from None
log.info("input %s (%s): %d bytes to %s", name, ref, size, dest)
@@ -748,8 +818,13 @@ def fetch_inputs(environ: Mapping[str, str]) -> list[Path]:
return fetched
def _deliver_output(task: Task, output_dir: Path, environ: Mapping[str, str]) -> bool:
"""POST the output directory; False when there is nowhere to deliver; :class:`DeliveryError` when it fails."""
def _deliver_output(task: Task, output_dir: Path, environ: Mapping[str, str], delivery: DeliveryTargets | None = None) -> bool:
"""PUT the output tar to ``delivery.put`` when the start request carried one, else POST it to the receiver;
False when there is nowhere to deliver; :class:`DeliveryError` when it fails."""
backoff = float(environ.get("OUSAST_DELIVERY_BACKOFF", "1.0"))
if delivery is not None and delivery.put:
deliver(delivery.put, build_tar(output_dir), {"Content-Type": "application/x-tar"}, backoff=backoff, method="PUT")
return True
url = environ.get("OUSAST_ARTIFACT_URL")
if not url:
return False
@@ -758,7 +833,6 @@ def _deliver_output(task: Task, output_dir: Path, environ: Mapping[str, str]) ->
run = environ.get("OUSAST_RUN")
if run:
headers["X-Ousast-Run"] = run
backoff = float(environ.get("OUSAST_DELIVERY_BACKOFF", "1.0"))
deliver(url, build_tar(output_dir), headers, backoff=backoff, dial=environ.get("OUSAST_ARTIFACT_DIAL") or None)
return True
@@ -810,14 +884,16 @@ def _exit_of(marker: Mapping[str, Any]) -> int:
return EXIT_BY_STATUS.get(str(marker.get("status")), EXIT_BY_STATUS["failed"])
def _finish(session: _Session, env: Mapping[str, str], exit_code: int | None, summary: Mapping[str, Any]) -> dict[str, Any]:
def _finish(
session: _Session, env: Mapping[str, str], exit_code: int | None, summary: Mapping[str, Any], delivery: DeliveryTargets | None = None
) -> dict[str, Any]:
"""Deliver the output directory and write the completion marker; a failed delivery reports ``failed``.
The marker keeps the task's own summary so a later boot can put it back and retry the delivery alone.
"""
marker: dict[str, Any] = {"exit": exit_code, "status": str(summary.get("status")), "delivered": False, "summary": dict(summary)}
try:
marker["delivered"] = _deliver_output(session.task, session.output_dir, env)
marker["delivered"] = _deliver_output(session.task, session.output_dir, env, delivery)
except DeliveryError as exc:
log.error("artifact delivery failed: %s", exc)
marker.update(status="failed", reason=f"artifact delivery failed: {exc}")
@@ -857,24 +933,24 @@ def _run_once(env: MutableMapping[str, str], ready: threading.Event, stop: threa
stored = done.get("summary")
kept: dict[str, Any] = stored if isinstance(stored, dict) else read_summary(session.output_dir) or {}
write_summary(session.output_dir, kept)
return _exit_of(_finish(session, env, done.get("exit"), kept))
return _exit_of(_finish(session, env, done.get("exit"), kept, start.delivery)) # a re-minted URL, if any
if stop.is_set():
return 0
try:
_materialise_once(session, env)
fetch_inputs(env) # after the start as well: the receiver serves a task its inputs only while it runs
fetch_inputs(env, start.delivery) # after the start: the URLs arrive with it (and a receiver serves a running task only)
except RunnerError as exc:
log.error("runner failed: %s", exc)
failed = _failed(f"runner failed: {exc}", read_summary(session.output_dir))
write_summary(session.output_dir, failed)
return _exit_of(_finish(session, env, None, failed))
return _exit_of(_finish(session, env, None, failed, start.delivery))
log.info("workspaces and inputs ready; running task %s: %s", session.task.metadata.name, list(session.task.command))
grace = float(env.get("OUSAST_TERM_GRACE") or TERM_GRACE)
code, summary = run_command(session.task, session.output_dir, env, stop, grace, start.credentials)
if code is None:
log.info("task %s was stopped before its command finished; a resume runs it again", session.task.metadata.name)
return 0
return _exit_of(_finish(session, env, code, summary))
return _exit_of(_finish(session, env, code, summary, start.delivery))
except RunnerError as exc:
# The failure is reported, never acted on unasked: a start request is answered with it (409 without a
# session, else 202 and the failed summary is delivered), so the golden boot delivers nothing either.
@@ -887,9 +963,10 @@ def _run_once(env: MutableMapping[str, str], ready: threading.Event, stop: threa
write_summary(session.output_dir, summary)
gate.arm(env.get("OUSAST_RUN", ""), session.task.metadata.name)
ready.set()
if env.get("AX_RUNNER_AUTOSTART") != "1" and gate.wait(stop) is None:
late = gate.autostart() if env.get("AX_RUNNER_AUTOSTART") == "1" else gate.wait(stop)
if late is None:
return 0
return _exit_of(_finish(session, env, None, summary))
return _exit_of(_finish(session, env, None, summary, late.delivery))
def _materialise_once(session: _Session, env: Mapping[str, str]) -> None:
@@ -918,14 +995,14 @@ def main(argv: Sequence[str] | None = None, environ: MutableMapping[str, str] |
env = os.environ if environ is None else environ
if argv:
log.warning("ax-task-runner ignores command-line arguments %r; the Task manifest is the only input", list(argv))
ready, stop, gate = threading.Event(), threading.Event(), StartGate()
ready, stop, gate = threading.Event(), threading.Event(), StartGate(insecure=env.get("AX_RUNNER_INSECURE_DELIVERY") == "1")
previous = _on_sigterm(stop)
server: HealthServer | None = None
if env.get("AX_RUNNER_HTTP", "1") != "0":
metadata = {"task": env.get("AX_TASK_YAML", ""), "workspaces": env.get("AX_WORKSPACES_YAML", "")}
server = start_health_server(int(env.get("AX_RUNNER_PORT", str(DEFAULT_PORT))), ready, metadata, gate)
env["AX_RUNNER_BOUND_PORT"] = str(server.port)
env["AX_METADATA_URL"] = f"http://127.0.0.1:{server.port}"
env["AX_METADATA_URL"] = f"http://127.0.0.1:{server.port}" # loopback: this runner's own health server
log.info("health and metadata server on port %d", server.port)
try:
if server is None and env.get("AX_RUNNER_AUTOSTART") != "1":
tests/conftest.py
+26 -2
View File
@@ -65,11 +65,12 @@ from pathlib import Path
HERE = Path(__file__).resolve().parent
argv = sys.argv[1:]
context = ""
if argv[:1] == ["--context"]:
argv = argv[2:]
context, argv = argv[1], argv[2:]
verb, actor, space = argv[0], argv[2], argv[argv.index("-a") + 1]
with (HERE / "ax.log").open("a") as log:
log.write(f"{time.monotonic():.4f} egress {verb} {actor}\n")
log.write(f"{time.monotonic():.4f} egress {verb} {actor} {context}\n")
record = HERE / "applied" / f"{actor}.json"
alive = record.exists() and not json.loads(record.read_text()).get("gone")
store = HERE / "policies" / f"{space}_{actor}.json"
@@ -103,3 +104,26 @@ def kubectl_ate(tmp_path, monkeypatch: pytest.MonkeyPatch):
monkeypatch.setenv("OUSAST_KUBECTL_ATE", str(path))
monkeypatch.setenv("OUSAST_EGRESS_SETTLE", "0")
return path
PROFILE_CONTEXT = "kind-test" # the fakes must receive this, never a context a module built (plane-on-kubernetes 1.2)
PROFILE_TOML = (
'exec = "local"\nkube_context = "{context}"\nregistry = "registry.test:5000"\nimages = "images.json"\n'
'memory = "file://{memory}"\natespace = "default"\n[pools.default]\nreplicas = 1\ncpu = "1"\nmemory = "1Gi"\n'
)
@pytest.fixture
def plane_profile(tmp_path, monkeypatch: pytest.MonkeyPatch):
"""A throwaway PlaneProfile under ``tmp_path`` (context ``kind-test``, a file:// memory store, a digest-pinned
images file), selected through ``OUSAST_PLANE_PROFILE`` so every plane module loads it."""
import json
root = tmp_path / "profile"
root.mkdir()
(root / "test.toml").write_text(PROFILE_TOML.format(context=PROFILE_CONTEXT, memory=tmp_path / "memory"), encoding="utf-8")
(root / "images.json").write_text(json.dumps({"runner": "registry.test:5000/ousast-runner@sha256:" + "ab" * 32}), encoding="utf-8")
monkeypatch.setenv("OUSAST_PLANE_PROFILE", str(root / "test.toml"))
for key in ("OUSAST_KUBE_CONTEXT", "OUSAST_MEMORY", "OUSAST_REGISTRY", "OUSAST_IMAGES", "OUSAST_EXEC"):
monkeypatch.delenv(key, raising=False)
return root / "test.toml"
tests/plane_fake_store.py
+109
View File
@@ -0,0 +1,109 @@
"""An S3-shaped memory store for the delivery tests: objects in a dict, presigned URLs served by a local HTTP server
that accepts PUT, GET and HEAD by key (``/<bucket>/<key>?X-Amz-Expires=N&X-Amz-Signature=fake``), the way a
presigned S3 URL looks to the runner. Nothing here is imported by the product."""
from __future__ import annotations
import threading
from collections.abc import Mapping
from datetime import timedelta
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import unquote, urlsplit
from openultrasast.plane.memory import MemoryStore
class HttpObjectStore(MemoryStore):
"""``presign_*`` hand out URLs on a local server; ``requests`` records every HTTP call (method, key)."""
def __init__(self, bucket: str = "fake-bucket", host: str = "localhost") -> None:
self.objects: dict[str, bytes] = {}
self.bucket, self.host = bucket, host
self.requests: list[tuple[str, str]] = []
self.presigned: list[tuple[str, str, int]] = []
self.lock = threading.Lock()
self.server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
self.server.store = self # type: ignore[attr-defined]
threading.Thread(target=self.server.serve_forever, daemon=True).start()
def close(self) -> None:
self.server.shutdown()
self.server.server_close()
def describe(self) -> str:
return f"fake://{self.bucket}"
def _get(self, key: str) -> tuple[bytes, str | None] | None:
with self.lock:
data = self.objects.get(key)
return None if data is None else (data, "v1")
def _put(self, key: str, data: bytes, labels: Mapping[str, str] | None = None) -> None:
with self.lock:
self.objects[key] = bytes(data)
def _delete(self, key: str) -> None:
with self.lock:
self.objects.pop(key, None)
def _keys(self, prefix: str) -> list[str]:
with self.lock:
return sorted(k for k in self.objects if k.startswith(prefix))
def _url(self, method: str, key: str, expires: timedelta) -> str:
self.presigned.append((method, key, int(expires.total_seconds())))
port = self.server.server_address[1]
return f"http://{self.host}:{port}/{self.bucket}/{key}?X-Amz-Expires={int(expires.total_seconds())}&X-Amz-Signature=fake"
def presign_put(self, key: str, expires: timedelta = timedelta(hours=1)) -> str:
return self._url("PUT", key, expires)
def presign_get(self, key: str, expires: timedelta = timedelta(hours=1)) -> str:
return self._url("GET", key, expires)
class _Handler(BaseHTTPRequestHandler):
def _key(self) -> str | None:
store: HttpObjectStore = self.server.store # type: ignore[attr-defined]
path = unquote(urlsplit(self.path).path)
prefix = f"/{store.bucket}/"
return path[len(prefix) :] if path.startswith(prefix) else None
def do_PUT(self) -> None: # noqa: N802
store: HttpObjectStore = self.server.store # type: ignore[attr-defined]
key = self._key()
body = self.rfile.read(int(self.headers.get("Content-Length") or 0))
store.requests.append(("PUT", key or self.path))
if key is None:
return self._answer(404)
store._put(key, body)
self._answer(200)
def do_GET(self) -> None: # noqa: N802
store: HttpObjectStore = self.server.store # type: ignore[attr-defined]
key = self._key()
store.requests.append(("GET", key or self.path))
got = store._get(key) if key else None
if got is None:
return self._answer(404)
self.send_response(200)
self.send_header("Content-Length", str(len(got[0])))
self.end_headers()
self.wfile.write(got[0])
def do_HEAD(self) -> None: # noqa: N802
store: HttpObjectStore = self.server.store # type: ignore[attr-defined]
key = self._key()
store.requests.append(("HEAD", key or self.path))
got = store._get(key) if key else None
self.send_response(404 if got is None else 200)
self.send_header("Content-Length", "0" if got is None else str(len(got[0])))
self.end_headers()
def _answer(self, code: int) -> None:
self.send_response(code)
self.send_header("Content-Length", "0")
self.end_headers()
def log_message(self, format: str, *args: object) -> None: # noqa: A002
pass
tests/test_plane_delivery.py
+164
View File
@@ -0,0 +1,164 @@
"""plane-on-kubernetes task 2.1 (Req 3.1, 2.1): ``Delivery`` against an S3-shaped fake -- presigned URLs in the start
body, expiry, polling, extraction with the receiver's member checks, declared-only re-put, state seeding, and no URL
in any log line."""
from __future__ import annotations
import io
import json
import logging
import tarfile
import urllib.request
from collections.abc import Iterator
from pathlib import Path
import pytest
from plane_fake_store import HttpObjectStore
from openultrasast.plane.delivery import GRACE_SECONDS, Delivery, DeliveryError, declared, expiry, input_var
@pytest.fixture
def store() -> Iterator[HttpObjectStore]:
fake = HttpObjectStore()
try:
yield fake
finally:
fake.close()
def tar_of(files: dict[str, bytes], *, symlink: str | None = None) -> bytes:
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w") as tar:
for name, data in files.items():
info = tarfile.TarInfo(name)
info.size = len(data)
tar.addfile(info, io.BytesIO(data))
if symlink:
info = tarfile.TarInfo(symlink)
info.type = tarfile.SYMTYPE
info.linkname = "/etc/passwd"
tar.addfile(info)
return buffer.getvalue()
def put_via_url(url: str, data: bytes) -> int:
request = urllib.request.Request(url, data=data, method="PUT", headers={"Content-Type": "application/x-tar"})
with urllib.request.urlopen(request, timeout=10) as response: # noqa: S310 - the fake's local URL
return int(response.status)
def test_the_start_body_names_exactly_the_tar_and_the_declared_inputs(store: HttpObjectStore, tmp_path: Path, monkeypatch) -> None:
monkeypatch.setenv("OUSAST_TASK_TIMEOUT", "100")
delivery = Delivery(store, "run-1", tmp_path)
body = delivery.body("verify", {"facts": "repo-facts/facts.json", "pass a": "va/units.jsonl"})
assert set(body) == {"put", "inputs"}
assert body["put"].startswith(f"http://localhost:{store.server.server_address[1]}/fake-bucket/runs/run-1/verify/output.tar?")
assert set(body["inputs"]) == {"FACTS", "PASS_A"} and input_var("pass a") == "PASS_A"
assert "/runs/run-1/repo-facts/facts.json?" in body["inputs"]["FACTS"] and "/runs/run-1/va/units.jsonl?" in body["inputs"]["PASS_A"]
assert {m for m, _, _ in store.presigned} == {"PUT", "GET"}
assert {e for _, _, e in store.presigned} == {100 + GRACE_SECONDS}, "expiry = OUSAST_TASK_TIMEOUT + 600"
assert expiry({"OUSAST_TASK_TIMEOUT": "7200"}).total_seconds() == 7800 and expiry({}).total_seconds() == 7800
assert delivery.host == "localhost", "the hostname the egress policy must allow"
with pytest.raises(DeliveryError, match="not <producer>/<artifact>"):
delivery.input_urls({"x": "no-slash"})
def test_delivered_polls_the_store_and_collect_extracts_republishes_and_marks_done(
store: HttpObjectStore, tmp_path: Path, caplog: pytest.LogCaptureFixture
) -> None:
caplog.set_level(logging.DEBUG)
delivery = Delivery(store, "r", tmp_path / "r")
assert delivery.delivered("facts") is False
summary = {"status": "done", "units_done": 1, "units_total": 1}
tar = tar_of({"facts.json": b'{"a": 1}', "summary.json": json.dumps(summary).encode(), "scratch/notes.txt": b"not declared"})
assert put_via_url(delivery.put_url("facts"), tar) == 200, "the runner's PUT to the presigned URL"
assert delivery.delivered("facts") is True
got, size = delivery.collect("facts", ["facts.json", "summary.json", "missing.json"])
assert got == summary and size == len(tar)
assert (tmp_path / "r" / "facts" / "facts.json").read_bytes() == b'{"a": 1}'
assert (tmp_path / "r" / "facts" / "scratch" / "notes.txt").exists(), "extracted locally"
assert store.objects["runs/r/facts/facts.json"] == b'{"a": 1}' and "runs/r/facts/summary.json" in store.objects
assert "runs/r/facts/scratch/notes.txt" not in store.objects and "runs/r/facts/missing.json" not in store.objects, "declared only"
done = json.loads(store.objects["runs/r/facts/done.json"])
assert (
done["status"] == "done"
and done["bytes"] == len(tar)
and done["files"] == 3
and done["published"] == ["facts.json", "summary.json"]
)
# a consumer's presigned GET names one artifact and fetches it
urls = delivery.input_urls({"facts": "facts/facts.json"})
with urllib.request.urlopen(urls["FACTS"], timeout=10) as response: # noqa: S310
assert response.read() == b'{"a": 1}'
assert "X-Amz" not in caplog.text and "fake-bucket" not in caplog.text, "URLs never appear in logs"
delivery.reset("facts") # the next attempt must not complete on this attempt's tar
assert delivery.delivered("facts") is False and "runs/r/facts/done.json" not in store.objects
assert "runs/r/facts/facts.json" in store.objects, "a re-put output stays for its consumers until it is re-published"
@pytest.mark.parametrize("bad", ["../escape.json", "/abs.json"])
def test_a_tar_member_that_escapes_is_rejected_and_nothing_is_written(store: HttpObjectStore, tmp_path: Path, bad: str) -> None:
delivery = Delivery(store, "r", tmp_path / "r")
store._put(delivery.key("t", "output.tar"), tar_of({bad: b"{}", "summary.json": b"{}"}))
with pytest.raises(DeliveryError, match="rejected member"):
delivery.collect("t", ["summary.json"])
assert not (tmp_path / "r" / "t").exists() and not (tmp_path / "escape.json").exists()
assert "runs/r/t/done.json" not in store.objects and "runs/r/t/summary.json" not in store.objects
def test_a_symlink_member_and_a_non_tar_are_rejected(store: HttpObjectStore, tmp_path: Path) -> None:
delivery = Delivery(store, "r", tmp_path / "r")
store._put(delivery.key("t", "output.tar"), tar_of({"summary.json": b"{}"}, symlink="link"))
with pytest.raises(DeliveryError, match="rejected member link"):
delivery.collect("t", ["summary.json"])
store._put(delivery.key("t", "output.tar"), b"not a tar at all")
with pytest.raises(DeliveryError, match="bad tar"):
delivery.collect("t", ["summary.json"])
with pytest.raises(DeliveryError, match="not in the store"):
delivery.collect("never", [])
def test_state_is_mirrored_and_seeds_a_rerun_elsewhere(store: HttpObjectStore, tmp_path: Path) -> None:
first = Delivery(store, "r", tmp_path / "machine-a")
assert first.load_state() is None
first.save_state({"run": "r", "tasks": {"facts": {"status": "done"}}})
second = Delivery(store, "r", tmp_path / "machine-b") # another results root, the same store
assert second.load_state() == {"run": "r", "tasks": {"facts": {"status": "done"}}}
store._put(second.key("state.json"), b"not json")
assert second.load_state() is None
def test_put_input_stores_a_file_no_task_produces(store: HttpObjectStore, tmp_path: Path) -> None:
delivery = Delivery(store, "r", tmp_path / "r")
source = tmp_path / "candidates.json"
source.write_bytes(b"[1, 2, 3]")
url = delivery.put_input("candidates.json", source)
assert store.objects["runs/r/inputs/candidates.json"] == b"[1, 2, 3]"
with urllib.request.urlopen(url, timeout=10) as response: # noqa: S310
assert response.read() == b"[1, 2, 3]"
def test_declared_artifacts_are_outputs_plus_what_consumers_name(tmp_path: Path) -> None:
"""A consumer may declare a producer's summary.json (every task writes one) without the producer listing it."""
from openultrasast.plane.manifests import load_manifests
text = (
"apiVersion: ax.io/v1alpha1\nkind: Task\nmetadata: {name: t}\nspec: {command: [repo-facts]}\n---\n"
"apiVersion: openultrasast.io/v1alpha1\nkind: Run\nmetadata: {name: r}\nspec:\n tasks:\n"
" - {name: facts, task: t, outputs: [facts.json]}\n"
" - {name: verify, task: t, inputs: {facts: facts/facts.json, facts_summary: facts/summary.json}, outputs: [units.jsonl]}\n"
" - {name: agree, task: t, inputs: {pass: verify/units.jsonl, pass_summary: verify/summary.json}}\n"
)
(tmp_path / "run.yaml").write_text(text, encoding="utf-8")
run = next(iter(load_manifests([tmp_path / "run.yaml"]).runs.values()))
assert declared(run) == {"facts": ["facts.json", "summary.json"], "verify": ["units.jsonl", "summary.json"], "agree": []}
def test_publish_reputs_only_declared_files_under_the_task_directory(store: HttpObjectStore, tmp_path: Path) -> None:
delivery = Delivery(store, "r", tmp_path / "r")
(tmp_path / "r" / "seeded").mkdir(parents=True)
(tmp_path / "r" / "seeded" / "facts.json").write_bytes(b"{}")
(tmp_path / "r" / "secret.txt").write_bytes(b"outside")
assert delivery.publish("seeded", ["facts.json", "../secret.txt", "absent.json"]) == ["facts.json"]
assert set(store.objects) == {"runs/r/seeded/facts.json"}
tests/test_plane_egress.py
+58 -17
View File
@@ -1,8 +1,10 @@
"""ai-service-plane task 0: the per-task egress policy, its kubectl-ate calls, and the runner's dialled delivery.
Agent Substrate's egress gateway denies by default; :func:`egress.policy_for` must grant exactly the receiver (HTTP
by name), the bound Workspaces' Git hosts and the bound Model's API host (TLS passthrough), and nothing a gateway
would reject or that would intercept TLS: no IP address, no wildcard, no ``https`` rule.
Agent Substrate's egress gateway denies by default; :func:`egress.policy_for` must grant exactly the memory store's
host (where the runner PUTs its tar and GETs its inputs), the bound Workspaces' Git hosts and the bound Model's API
host, all as one TLS passthrough rule on 443 (plane-on-kubernetes 2.3), and nothing a gateway would reject or that
would intercept TLS: no IP address, no wildcard, no ``https`` rule, no ``http`` rule (the receiver path, kept for
group 2 only, is the one exception).
"""
from __future__ import annotations
@@ -70,24 +72,59 @@ def assert_tight(policy: dict) -> None:
ipaddress.ip_address(host)
def test_model_free_task_gets_only_the_receiver(manifests) -> None:
policy = egress.policy_for(manifests.tasks["facts"], [], None, egress.RECEIVER_HOST)
assert policy == {"rules": [{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}]}
STORE = "files.store.example"
def test_model_free_task_gets_only_the_store(manifests) -> None:
policy = egress.policy_for(manifests.tasks["facts"], [], None, STORE)
assert policy == {"rules": [{"tlsPassthrough": {"hostnames": [STORE], "ports": {"numbers": [443]}}}]}
assert_tight(policy)
assert egress.policy_for(manifests.tasks["facts"], [], None, None) == {"rules": []}, "no store, no Git, no Model: nothing"
def test_the_receiver_path_adds_the_one_http_rule_until_it_is_removed(manifests) -> None:
policy = egress.policy_for(manifests.tasks["facts"], [], None, None, receiver_host=egress.RECEIVER_HOST)
assert policy == {"rules": [{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}]}
assert egress.policy_for(manifests.tasks["facts"], [], None, None, receiver_host="127.0.0.1") == {"rules": []}
def test_model_bound_task_with_git_workspaces(manifests) -> None:
task = manifests.tasks["verify"]
policy = egress.policy_for(task, [manifests.workspaces["cases"]], manifests.models["deepseek-flash"], egress.RECEIVER_HOST)
policy = egress.policy_for(task, [manifests.workspaces["cases"]], manifests.models["deepseek-flash"], STORE)
assert policy["rules"] == [
{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}},
{"tlsPassthrough": {"hostnames": ["api.deepseek.com", "github.com"], "ports": {"numbers": [443]}}},
], "one github.com for two repos; the ssh and IP-address repos add nothing"
{"tlsPassthrough": {"hostnames": ["api.deepseek.com", STORE, "github.com"], "ports": {"numbers": [443]}}},
], "one rule: the store, one github.com for two repos, the Model host; the ssh and IP-address repos add nothing"
assert_tight(policy)
def test_an_ip_receiver_is_not_granted(manifests) -> None:
assert egress.policy_for(manifests.tasks["facts"], [], None, "127.0.0.1") == {"rules": []}
def test_an_ip_store_endpoint_is_refused_naming_the_variable(manifests) -> None:
with pytest.raises(ValueError, match="S3_ENDPOINT must be a hostname"):
egress.policy_for(manifests.tasks["facts"], [], None, "127.0.0.1")
def test_an_http_git_repo_is_refused(tmp_path: Path) -> None:
path = tmp_path / "m.yaml"
path.write_text(MANIFESTS.replace("https://github.com/example-org/alpha", "http://mirror.example/alpha"), encoding="utf-8")
loaded = load_manifests([path])
with pytest.raises(ValueError, match="http Git repos .*mirror.example.* cannot be reached"):
egress.policy_for(loaded.tasks["verify"], [loaded.workspaces["cases"]], None, STORE)
def test_every_committed_template_renders_the_expected_policy() -> None:
"""For each template under plane/tasks/: the store, the Model's hosts when bound, nothing else; one TLS rule."""
from openultrasast.plane.generate import TEMPLATES
plane = Path("plane")
loaded = load_manifests([plane / "tasks" / f"{name}.yaml" for name in TEMPLATES] + sorted((plane / "models").glob("*.yaml")))
assert set(loaded.tasks) == set(TEMPLATES)
for name, task in loaded.tasks.items():
model = loaded.models.get(task.metadata.annotations.get("openultrasast.io/model") or "")
policy = egress.policy_for(task, [], model, STORE)
expected = sorted({STORE, *(model.egress_hosts if model else ())})
assert policy == {"rules": [{"tlsPassthrough": {"hostnames": expected, "ports": {"numbers": [443]}}}]}, name
assert_tight(policy)
bound = {n for n, t in loaded.tasks.items() if "openultrasast.io/model" in t.metadata.annotations}
assert bound == {"verify", "roles"}, "the model-bound templates, and only they, get the provider host"
def test_deepseek_model_manifest_names_the_endpoint_host() -> None:
@@ -121,7 +158,7 @@ def calls(tmp_path: Path) -> list[str]:
return [" ".join(line.split()[2:4]) for line in (tmp_path / "ax.log").read_text().splitlines()]
POLICY = {"rules": [{"http": {"hostnames": [egress.RECEIVER_HOST], "ports": {"numbers": [80]}}}]}
POLICY = {"rules": [{"tlsPassthrough": {"hostnames": ["files.store.example"], "ports": {"numbers": [443]}}}]}
def test_apply_creates_reads_back_and_delete_confirms_the_cascade(tmp_path: Path, kubectl_ate: Path) -> None:
@@ -134,10 +171,11 @@ def test_apply_creates_reads_back_and_delete_confirms_the_cascade(tmp_path: Path
actor(tmp_path, "t1", gone=True) # ax delete task: the actor, and with it the policy, is gone
assert cli.delete("t1", "default") is None
assert calls(tmp_path) == ["get t1", "create t1", "get t1", "get t1", "get t1", "get t1"]
assert {line.split()[-1] for line in (tmp_path / "ax.log").read_text().splitlines()} == {"kind-test"}, "the given context"
def test_apply_replaces_an_existing_policy(tmp_path: Path, kubectl_ate: Path) -> None:
cli = egress.Egress(str(kubectl_ate))
cli = egress.Egress(str(kubectl_ate), context="kind-test")
actor(tmp_path, "t2")
cli.apply("t2", "default", {"rules": []})
cli.apply("t2", "default", POLICY)
@@ -146,7 +184,7 @@ def test_apply_replaces_an_existing_policy(tmp_path: Path, kubectl_ate: Path) ->
def test_apply_waits_for_the_actor_then_gives_up(tmp_path: Path, kubectl_ate: Path) -> None:
cli = egress.Egress(str(kubectl_ate))
cli = egress.Egress(str(kubectl_ate), context="kind-test")
with pytest.raises(RuntimeError, match="not found"):
cli.apply("missing", "default", POLICY, timeout=0.2)
timer = threading.Timer(0.3, actor, args=(tmp_path, "late"))
@@ -157,10 +195,13 @@ def test_apply_waits_for_the_actor_then_gives_up(tmp_path: Path, kubectl_ate: Pa
def test_receiver_address_modes(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OUSAST_ARTIFACT_HOST", "127.0.0.1")
assert egress.receiver_address(4242) == ("http://127.0.0.1:4242/", None)
assert egress.receiver_address(4242, None) == ("http://127.0.0.1:4242/", None)
monkeypatch.delenv("OUSAST_ARTIFACT_HOST")
monkeypatch.setenv("OUSAST_ARTIFACT_DIAL", "172.19.0.1:80")
assert egress.receiver_address(18090) == (f"http://{egress.RECEIVER_HOST}/", "172.19.0.1:80")
assert egress.receiver_address(18090, "kind-test") == (f"http://{egress.RECEIVER_HOST}/", "172.19.0.1:80")
monkeypatch.delenv("OUSAST_ARTIFACT_DIAL")
with pytest.raises(RuntimeError, match="kube context"):
egress.receiver_address(18090, None) # no context to look the Service up in: never a built one
# --- the runner dials the gateway address with the receiver's name as Host ----------------------------------------
tests/test_plane_increment.py
+39
View File
@@ -290,3 +290,42 @@ def test_case_record_carries_sites_in_set_and_cost_and_the_image_is_repinned(tmp
(tmp_path / "runner-image").write_text("localhost:5001/ousast-runner:dev\n")
with pytest.raises(ValueError, match="digest-pinned"):
repin_templates(paths["plane"], tmp_path / "runner-image")
def test_repin_reads_the_profiles_images_file(tmp_path: Path) -> None:
"""plane-on-kubernetes 1.4: ``ops/ax/up.sh`` writes ``kind-images.json`` ({"runner": ...}); repin reads it, the
older one-line pin still reads, and a file without a runner key or without a digest is refused by name."""
paths = _fixture(tmp_path)
digest = "localhost:5001/ousast-runner@sha256:" + "cd" * 32
images = tmp_path / "kind-images.json"
images.write_text(json.dumps({"runner": digest}) + "\n")
assert repin_templates(paths["plane"], images) == digest
assert {t.image for t in load_manifests(sorted((paths["plane"] / "tasks").glob("*.yaml"))).tasks.values()} == {digest}
images.write_text(json.dumps({"engine": digest}))
with pytest.raises(ValueError, match="no runner image"):
repin_templates(paths["plane"], images)
images.write_text(json.dumps({"runner": "localhost:5001/ousast-runner:dev"}))
with pytest.raises(ValueError, match="not digest-pinned"):
repin_templates(paths["plane"], images)
def test_workspaces_go_to_the_profiles_atespace(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""plane-on-kubernetes 1.2: ``generate.ATESPACE`` is a default; ``increment`` takes the profile's atespace."""
paths = _fixture(tmp_path)
(tmp_path / "p.toml").write_text('exec = "local"\nkube_context = "c"\nregistry = "r"\nimages = "i.json"\natespace = "team-a"\n')
monkeypatch.setenv("OUSAST_PLANE_PROFILE", str(tmp_path / "p.toml"))
increment(paths["population"], paths["set"], paths["candidates"], paths["scans"], plane=paths["plane"], run_name="sp", command="t")
manifests = load_manifests(sorted((paths["plane"] / "workspaces").glob("*.yaml")))
assert manifests.workspaces and {w.metadata.atespace for w in manifests.workspaces.values()} == {"team-a"}
increment(
paths["population"],
paths["set"],
paths["candidates"],
paths["scans"],
plane=paths["plane"],
run_name="sp",
command="t",
atespace="z",
)
manifests = load_manifests(sorted((paths["plane"] / "workspaces").glob("*.yaml")))
assert {w.metadata.atespace for w in manifests.workspaces.values()} == {"z"}, "an explicit atespace wins over the profile"
tests/test_plane_literals.py
+112
View File
@@ -0,0 +1,112 @@
"""plane-on-kubernetes Req 1.3: no module, manifest or page assumes this host.
The scan covers ``src/``, ``plane/``, ``ops/`` (minus ``ops/ax/up.sh``, which creates the kind cluster and may name
it, and ``ops/k8s/profiles/``, which *are* the configuration) and ``docs/`` for the kind context, the kind registry,
the kind network's gateway, a loopback address with a port, and ``port-forward``. Allowed: ``port-forward`` and the
loopback address inside ``router.py`` (the one module that opens the tunnel), and any line tagged ``ax-tunnel``
(the tunnel is how both profiles reach ax and the router) or ``loopback`` (a process talking to itself). A
digest-pinned ``image:`` line of a committed Task template under ``plane/tasks/`` is the kind profile's own value,
written there by ``ousast plane`` repinning from ``ops/k8s/profiles/kind-images.json``; it is accepted only while it
names that profile's registry, so the templates and the profile cannot drift apart. The patterns are built at runtime
so this file does not match them (as ``test_removed_plane_references.py`` does).
"""
from __future__ import annotations
import re
from pathlib import Path
from openultrasast.plane.profile import load_profile
ROOT = Path(__file__).resolve().parents[1]
SCANNED = ("src", "plane", "ops", "docs")
EXEMPT = (Path("ops/ax/up.sh"), Path("ops/k8s/profiles"))
ROUTER = Path("src/openultrasast/plane/router.py")
PATTERNS = {
"kind context": re.compile("kind-" + "ousast"),
"kind registry": re.compile("localhost:" + "5001"),
"kind gateway": re.compile(r"172\.19\."),
"loopback with port": re.compile(r"127\.0\.0\.1" + ":"),
"port-forward": re.compile("port-" + "forward"),
}
ROUTER_ONLY = {"loopback with port", "port-forward"}
TAGS = re.compile(r"ax-tunnel|loopback")
_IMAGE_LINE = re.compile(r'^\s*image:\s*"?(?P<ref>\S+?)@sha256:[0-9a-f]{64}"?\s*(#.*)?$')
def _files() -> list[Path]:
out = []
for top in SCANNED:
for path in sorted((ROOT / top).rglob("*")):
relative = path.relative_to(ROOT)
if not path.is_file() or "__pycache__" in relative.parts or path.suffix in (".png", ".jpg", ".gz", ".zip"):
continue
if any(relative == e or e in relative.parents for e in EXEMPT):
continue
out.append(path)
return out
def _pinned_template_image(relative: Path, line: str, registry: str) -> bool:
"""A committed template's digest-pinned image in the kind profile's registry: configuration, not a literal."""
if relative.parts[:2] != ("plane", "tasks"):
return False
found = _IMAGE_LINE.match(line)
return bool(found) and found.group("ref").startswith(registry + "/")
def offenders(paths: list[Path], registry: str, root: Path = ROOT) -> list[str]:
found = []
for path in paths:
relative = path.relative_to(root)
try:
text = path.read_text(encoding="utf-8")
except (UnicodeDecodeError, OSError):
continue
for number, line in enumerate(text.splitlines(), 1):
for label, pattern in PATTERNS.items():
if not pattern.search(line):
continue
if label in ROUTER_ONLY and relative == ROUTER:
continue
if TAGS.search(line) or _pinned_template_image(relative, line, registry):
continue
found.append(f"{relative}:{number}: [{label}] {line.strip()[:120]}")
return found
def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None:
kind = load_profile("kind", environ={})
found = offenders(_files(), kind.registry)
assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found)
def test_the_scan_sees_a_reinserted_literal(tmp_path: Path) -> None:
"""The instrument is not vacuous: each pattern is caught, a tag or the router exempts only what it says."""
kind = load_profile("kind", environ={})
src = tmp_path / "src"
src.mkdir()
(src / "a.py").write_text(f'ctx = "{"kind-" + "ousast"}"\nreg = "{"localhost:" + "5001"}"\nip = "172.19.0.1"\n', encoding="utf-8")
(src / "b.py").write_text('url = "http://127.0.0.1:8080" # loopback, this process\nargv = ["port-' + 'forward"] # ax-tunnel\n')
(src / "c.py").write_text('argv = ["port-' + 'forward"]\n', encoding="utf-8")
paths = sorted(src.glob("*.py"))
found = [o.split(": ", 1)[1] for o in offenders(paths, kind.registry, root=tmp_path)]
assert found == [
"[kind context] ctx = " + '"kind-' + 'ousast"',
"[kind registry] reg = " + '"localhost:' + '5001"',
'[kind gateway] ip = "172.19.0.1"',
"[port-forward] argv = " + '["port-' + 'forward"]',
]
def test_the_committed_templates_name_the_kind_profiles_registry() -> None:
"""The one exemption is checked, not assumed: every template image is in the kind registry, by digest."""
kind = load_profile("kind", environ={})
images = []
for path in sorted((ROOT / "plane/tasks").glob("*.yaml")):
for line in path.read_text(encoding="utf-8").splitlines():
found = _IMAGE_LINE.match(line)
if found:
images.append(found.group("ref"))
assert found.group("ref").startswith(kind.registry + "/"), f"{path.name}: {line.strip()}"
assert images, "the templates carry digest-pinned images"
tests/test_plane_profile.py
+158
View File
@@ -0,0 +1,158 @@
"""plane-on-kubernetes task 1.1: the ``PlaneProfile`` (Req 1.1). Defaults, file-then-environment precedence, every
rejection naming its field, both committed profiles loading, and no committed value that looks like a credential."""
from __future__ import annotations
import json
import re
from pathlib import Path
import pytest
from openultrasast.plane import profile as profile_module
from openultrasast.plane.profile import PlaneProfile, Pool, ProfileError, load_images, load_profile, profiles_dir
ROOT = Path(__file__).resolve().parents[1]
DIGEST = "sha256:" + "ab" * 32
MINIMAL = 'exec = "local"\nkube_context = "kind-test"\nregistry = "registry.test:5000"\nimages = "images.json"\nmemory = "s3://m"\n'
def write_profile(tmp_path: Path, text: str = MINIMAL, name: str = "test") -> Path:
path = tmp_path / f"{name}.toml"
path.write_text(text, encoding="utf-8")
(tmp_path / "images.json").write_text(json.dumps({"runner": f"registry.test:5000/ousast-runner@{DIGEST}"}), encoding="utf-8")
return path
def test_defaults_and_the_images_path_resolve_against_the_profile_file(tmp_path: Path) -> None:
loaded = load_profile(str(write_profile(tmp_path)), environ={})
assert loaded.name == "test" and loaded.exec == "local" and loaded.kube_context == "kind-test"
assert loaded.images == tmp_path / "images.json" and loaded.load_images() == {"runner": f"registry.test:5000/ousast-runner@{DIGEST}"}
assert loaded.atespace == "default" and loaded.router_url == "" and loaded.pools == {}
assert "(tunnel through the kube context)" in loaded.addresses()["router_url"]
def test_the_environment_overrides_the_file_and_names_the_profile(tmp_path: Path) -> None:
path = write_profile(tmp_path, MINIMAL + '[pools.default]\nreplicas = 2\ncpu = "1"\nmemory = "1536Mi"\n')
env = {
"OUSAST_PLANE_PROFILE": str(path),
"OUSAST_KUBE_CONTEXT": "other-cluster",
"OUSAST_MEMORY": "s3://elsewhere/prefix",
"OUSAST_ROUTER_URL": "https://router.test",
"OUSAST_POOL_DEFAULT_REPLICAS": "5",
"OUSAST_POOL_DEFAULT_MEMORY": "2Gi",
"OUSAST_EXEC": "", # an empty variable is not an override
}
loaded = load_profile(environ=env)
assert (loaded.kube_context, loaded.memory, loaded.router_url, loaded.exec) == (
"other-cluster",
"s3://elsewhere/prefix",
"https://router.test",
"local",
)
assert loaded.pools == {"default": Pool(replicas=5, cpu="1", memory="2Gi")}
assert load_profile(str(path), environ={}).kube_context == "kind-test", "the file is unchanged"
@pytest.mark.parametrize(
("text", "field"),
[
(MINIMAL + 'colour = "blue"\n', "unknown keys: colour"),
(MINIMAL + 'aws_secret_key = "x"\n', "aws_secret_key looks like a credential"),
(MINIMAL + 'api_token = "x"\n', "api_token looks like a credential"),
(MINIMAL.replace('exec = "local"', 'exec = "cluster"'), "exec must be one of local, remote"),
(MINIMAL + '[pools.default]\nreplicas = "two"\ncpu = "1"\nmemory = "1Gi"\n', "pools.default.replicas must be an integer"),
(MINIMAL + '[pools.default]\nreplicas = 1\ncpu = "1"\n', "pools.default lacks memory"),
(MINIMAL + '[pools.default]\nreplicas = 1\ncpu = "1"\nmemory = "1Gi"\nnodes = 3\n', "pools.default has unknown keys: nodes"),
(MINIMAL.replace('kube_context = "kind-test"', "kube_context = 3"), "kube_context must be a string"),
],
)
def test_each_rejection_names_the_field(tmp_path: Path, text: str, field: str) -> None:
path = write_profile(tmp_path, text)
with pytest.raises(ProfileError, match=re.escape(field)) as exc:
load_profile(str(path), environ={})
assert str(path) in str(exc.value)
def test_a_credential_shaped_value_is_refused(tmp_path: Path) -> None:
path = write_profile(tmp_path, MINIMAL.replace("s3://m", "sk-" + "a1b2c3d4" * 3))
with pytest.raises(ProfileError, match="value of memory looks like a credential"):
load_profile(str(path), environ={})
def test_an_images_file_without_digest_pins_is_refused(tmp_path: Path) -> None:
path = write_profile(tmp_path)
(tmp_path / "images.json").write_text(json.dumps({"runner": "registry.test:5000/ousast-runner:dev"}), encoding="utf-8")
with pytest.raises(ProfileError, match="runner is not digest-pinned"):
load_profile(str(path), environ={})
(tmp_path / "images.json").write_text(f"registry.test:5000/ousast-runner@{DIGEST}\n", encoding="utf-8")
assert load_images(tmp_path / "images.json") == {"runner": f"registry.test:5000/ousast-runner@{DIGEST}"}, "the one-line form"
(tmp_path / "images.json").unlink()
loaded = load_profile(str(path), environ={}) # a file the release has not published yet: doctor's finding, not a load error
with pytest.raises(ProfileError, match="images file"):
loaded.load_images()
def test_an_environment_override_of_the_wrong_shape_is_refused(tmp_path: Path) -> None:
path = write_profile(tmp_path, MINIMAL + '[pools.default]\nreplicas = 1\ncpu = "1"\nmemory = "1Gi"\n')
with pytest.raises(ProfileError, match="OUSAST_POOL_DEFAULT_REPLICAS must be an integer"):
load_profile(str(path), environ={"OUSAST_POOL_DEFAULT_REPLICAS": "many"})
with pytest.raises(ProfileError, match="names a pool the profile does not declare"):
load_profile(str(path), environ={"OUSAST_POOL_ENGINE_REPLICAS": "1"})
with pytest.raises(ProfileError, match="exec must be one of"):
load_profile(str(path), environ={"OUSAST_EXEC": "both"})
def test_a_missing_profile_names_its_path(tmp_path: Path) -> None:
with pytest.raises(ProfileError, match="no-such.toml"):
load_profile(str(tmp_path / "no-such.toml"), environ={})
with pytest.raises(ProfileError, match="unknown"):
load_profile("unknown", environ={"OUSAST_PLANE_PROFILES": str(tmp_path)})
@pytest.mark.parametrize("name", ["kind", "k3s"])
def test_both_committed_profiles_load_with_the_designs_values(name: str) -> None:
loaded = load_profile(name, environ={})
assert loaded.path == profiles_dir() / f"{name}.toml" and profiles_dir() == ROOT / "ops/k8s/profiles"
assert loaded.exec == {"kind": "local", "k3s": "remote"}[name]
assert loaded.memory == "s3://sast-memory" and loaded.atespace == "default" and loaded.snapshots_bucket == "s3://ax-snapshots"
assert set(loaded.pools) == {"default", "engine"} and loaded.pools["engine"].memory == "4Gi"
assert loaded.router_url == "" and loaded.ax_server == "", "both profiles tunnel through the Kubernetes API"
if name == "kind":
assert loaded.image_pull_secret == "" and loaded.images.name == "kind-images.json" and loaded.images.is_file()
assert set(loaded.load_images()) == {"runner"} and loaded.load_images()["runner"].startswith(loaded.registry + "/")
else:
assert loaded.image_pull_secret == "ghcr-pull" and loaded.registry.startswith("ghcr.io/") and loaded.images.name == "images.json"
assert not loaded.images.is_file(), "the release asset is downloaded, never committed (design section 8)"
def test_no_committed_profile_value_matches_a_credential_pattern() -> None:
patterns = (
re.compile(r"(?i)(api_key|secret_key|access_key|token|password)\s*="), # image_pull_secret names a Secret, holds none
re.compile(r"sk-[A-Za-z0-9]{8,}"),
re.compile(r"AKIA[0-9A-Z]{12,}"),
)
for path in sorted(profiles_dir().glob("*.toml")):
for number, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
code = line.split("#", 1)[0]
for pattern in patterns:
assert not pattern.search(code), f"{path}:{number}: {line.strip()}"
def test_print_for_shell_scripts(capsys: pytest.CaptureFixture[str], tmp_path: Path) -> None:
path = write_profile(tmp_path, MINIMAL + '[pools.default]\nreplicas = 3\ncpu = "1"\nmemory = "1Gi"\n')
fields = ["registry", "kube_context", "images.runner", "pools.default.replicas"]
assert profile_module.main(["--profile", str(path), "--print", *fields]) == 0
assert capsys.readouterr().out.splitlines() == ["registry.test:5000", "kind-test", f"registry.test:5000/ousast-runner@{DIGEST}", "3"]
assert profile_module.main(["--profile", str(path), "--print", "images"]) == 0, "the images file's path itself (up.sh writes it)"
assert capsys.readouterr().out.strip() == str(tmp_path / "images.json")
assert profile_module.main(["--profile", str(path), "--print", "images.engine"]) == 2
assert "no image named 'engine'" in capsys.readouterr().err
assert profile_module.main(["--profile", str(path), "--print", "nothing"]) == 2
def test_profile_is_frozen_and_typed() -> None:
loaded = load_profile("kind", environ={})
assert isinstance(loaded, PlaneProfile)
with pytest.raises(AttributeError):
loaded.exec = "remote" # type: ignore[misc]
tests/test_plane_reconciler.py
+176 -27
View File
@@ -4,9 +4,10 @@ The fake is a Python script the tests write into ``tmp_path`` that follows ax's
internal/controller/reconciler.go, internal/server/server.go): ``apply`` creates the task ``Suspended``; ``resume``
fails with DeadlineExceeded the first time (ax then reports ``Failed``, as while Agent Substrate builds a new
image's golden snapshot) and succeeds after; ``get task`` answers ``Running`` forever -- ax has no Completed phase
-- and on the scripted poll it POSTs a scripted tar to the ``OUSAST_ARTIFACT_URL`` in the applied Task's env, or
turns ``Failed`` with a condition message, or vanishes. Every outcome comes from a scripted delivery, and a
delivery happens only after the reconciler's start request reached the fake router (Req 4.5).
-- and on the scripted poll it PUTs a scripted tar to the presigned ``delivery.put`` URL the start request carried
(plane-on-kubernetes 2.4; or POSTs it to ``OUSAST_ARTIFACT_URL`` in receiver mode), or turns ``Failed`` with a
condition message, or vanishes. Every outcome comes from a scripted delivery, and a delivery happens only after
the reconciler's start request reached the fake router (Req 4.5). The store is ``plane_fake_store.HttpObjectStore``.
"""
from __future__ import annotations
@@ -28,11 +29,13 @@ from pathlib import Path
import pytest
import yaml
from plane_fake_store import HttpObjectStore
from openultrasast.plane import reconciler
from openultrasast.plane.workspaces import workspaces
RECONCILER = Path("src/openultrasast/plane/reconciler.py")
PROFILE_CONTEXT = "kind-test" # conftest's plane_profile fixture: the context every fake must be handed
PIN = "0123456789abcdef0123456789abcdef01234567"
SECRET = "sk-reconciler-test-7a2b-only-in-the-start-request"
@@ -145,11 +148,15 @@ elif argv[:2] == ["get", "task"]:
started = (HERE / "started" / argv[2]).exists() # the runner runs its command only after the start request
if rec["phase"] == "Running" and rec["polls"] >= script.get("polls", 1) and started and not rec.get("sent"):
rec["sent"] = True
body = json.loads((HERE / "started" / argv[2]).read_text() or "{{}}") # the start body the router saw
delivery = body.get("delivery") or {{}}
if script.get("deliver", True) and "phase" not in script and not script.get("vanish"):
files = script.get("files", {{}})
base = env["OUSAST_ARTIFACT_URL"].rstrip("/") + "/inputs/"
base = env.get("OUSAST_ARTIFACT_URL", "").rstrip("/") + "/inputs/"
for name, ref in json.loads(env.get("OUSAST_INPUTS", "{{}}")).items():
fetch = urllib.request.Request(base + ref, headers={{"X-Ousast-Run": env["OUSAST_RUN"], "X-Ousast-Task": task}})
presigned = (delivery.get("inputs") or {{}}).get(name)
fetch = urllib.request.Request(presigned) if presigned else urllib.request.Request(
base + ref, headers={{"X-Ousast-Run": env["OUSAST_RUN"], "X-Ousast-Task": task}})
try:
with urllib.request.urlopen(fetch) as resp:
dest = HERE / "fetched" / task / ref
@@ -170,7 +177,11 @@ elif argv[:2] == ["get", "task"]:
tar.addfile(info, io.BytesIO(data))
headers = {{"Content-Type": "application/x-tar", "X-Ousast-Run": env["OUSAST_RUN"]}}
headers["X-Ousast-Task"] = script.get("as_task", task)
req = urllib.request.Request(env["OUSAST_ARTIFACT_URL"], data=buf.getvalue(), method="POST", headers=headers)
if delivery.get("put"): # the store path: PUT to the presigned URL, no receiver headers
tar_headers = {{"Content-Type": "application/x-tar"}}
req = urllib.request.Request(delivery["put"], data=buf.getvalue(), method="PUT", headers=tar_headers)
else:
req = urllib.request.Request(env["OUSAST_ARTIFACT_URL"], data=buf.getvalue(), method="POST", headers=headers)
try:
with urllib.request.urlopen(req) as resp:
(HERE / "ax.log").open("a").write(f"{{time.monotonic():.4f}} delivered {{task}} {{resp.status}}\\n")
@@ -282,6 +293,7 @@ SCRIPT = {
class Fake:
router: FakeRouter
store: HttpObjectStore
def __init__(self, root: Path) -> None:
self.root = root
@@ -342,7 +354,8 @@ class _RouterHandler(BaseHTTPRequestHandler):
self.server.starts.append({"at": time.monotonic(), "path": self.path, "target": target, "body": body, "code": code})
if code == 202:
(self.server.root / "started").mkdir(exist_ok=True)
(self.server.root / "started" / name).touch()
without_secrets = {k: v for k, v in body.items() if k != "credentials"} # the fake ax reads delivery from it
(self.server.root / "started" / name).write_text(json.dumps(without_secrets))
with (self.server.root / "ax.log").open("a") as log:
log.write(f"{time.monotonic():.4f} started {name} {code}\n")
self.send_response(code)
@@ -355,7 +368,7 @@ class _RouterHandler(BaseHTTPRequestHandler):
@pytest.fixture
def fake(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, kubectl_ate: Path) -> Iterator[Fake]:
def fake(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, kubectl_ate: Path, plane_profile: Path) -> Iterator[Fake]:
monkeypatch.setenv("OUSAST_RESULTS", str(tmp_path / "results"))
monkeypatch.setenv("OUSAST_ARTIFACT_HOST", "127.0.0.1")
monkeypatch.setenv("OUSAST_POLL_SECONDS", "0.02")
@@ -365,12 +378,15 @@ def fake(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, kubectl_ate: Path) ->
fake = Fake(tmp_path)
fake.script(SCRIPT)
fake.router = FakeRouter(tmp_path)
fake.store = HttpObjectStore() # the memory store the reconciler opens first: S3-shaped, presigned URLs served locally
monkeypatch.setattr(reconciler, "open_store", lambda spec=None, environ=None: fake.store)
monkeypatch.setenv("OUSAST_ROUTER_URL", f"http://127.0.0.1:{fake.router.server_address[1]}")
try:
yield fake
finally:
fake.router.shutdown()
fake.router.server_close()
fake.store.close()
def write_run(tmp_path: Path, name: str, tasks: str = CHAIN) -> Path:
@@ -404,13 +420,17 @@ def test_chain_runs_in_order_with_inputs_env_and_model(fake: Fake, tmp_path: Pat
assert env["OUSAST_BUDGET_USD"] == "2.5" and env["OUSAST_BUDGET_CALLS"] == "40"
assert env["OUSAST_INPUT_FACTS"] == "/workspace/.ousast-in/verify/repo-facts/facts.json"
assert json.loads(env["OUSAST_INPUTS"]) == {"FACTS": "repo-facts/facts.json"}
assert env["OUSAST_ARTIFACT_URL"].startswith("http://127.0.0.1:")
assert "OUSAST_ARTIFACT_URL" not in env, "the store path renders no receiver address into the Task"
assert env["OUSAST_MODEL"] == "deepseek-flash"
assert json.loads(env["OUSAST_MODEL_PARAMS"]) == {"cache_hit_per_m": 0.07, "input_per_m": 0.27, "output_per_m": 1.1}
kinds = [(d["kind"], d["metadata"]["name"]) for d in verify["docs"]]
assert kinds == [("Model", "deepseek-flash"), ("Workspace", "case-pin"), ("Task", "chain-verify")], "no inputs Workspace"
assert verify["docs"][-1]["spec"]["workspaces"] == [{"name": "case-pin", "path": "/workspace"}]
assert [n for _, n in fake.events("fetched")] == ["verify FACTS 200", "agree PASS 200"], "inputs come over the receiver's GET"
assert [n for _, n in fake.events("fetched")] == ["verify FACTS 200", "agree PASS 200"], "inputs come by their presigned GET"
assert ("GET", "runs/chain/repo-facts/facts.json") in fake.store.requests and (
"PUT",
"runs/chain/verify/output.tar",
) in fake.store.requests
assert json.loads((tmp_path / "fetched" / "verify" / "repo-facts" / "facts.json").read_text()) == {"a.py": {"functions": ["f"]}}
assert json.loads((tmp_path / "fetched" / "agree" / "verify" / "agreed.json").read_text()) == {"agreed": ["a.py:f"]}
assert "annotations" not in verify["docs"][-1]["metadata"], "ax's ObjectMeta has no annotations"
@@ -569,7 +589,15 @@ def test_start_is_sent_once_after_resume_with_the_models_credential(fake: Fake,
assert [s["target"] for s in starts] == ["default/sig-repo-facts", "default/sig-verify", "default/sig-agree"], "once each"
assert all(s["path"] == "/ousast/v1/start" and s["code"] == 202 for s in starts)
by_task = {s["body"]["task"]: s for s in starts}
assert by_task["sig-verify"]["body"] == {"run": "sig", "task": "sig-verify", "credentials": {"DEEPSEEK_API_KEY": SECRET}}
body = by_task["sig-verify"]["body"]
assert {k: body[k] for k in ("run", "task", "credentials")} == {
"run": "sig",
"task": "sig-verify",
"credentials": {"DEEPSEEK_API_KEY": SECRET},
}
assert set(body["delivery"]) == {"put", "inputs"} and "/runs/sig/verify/output.tar?" in body["delivery"]["put"]
assert set(body["delivery"]["inputs"]) == {"FACTS"} and "/runs/sig/repo-facts/facts.json?" in body["delivery"]["inputs"]["FACTS"]
assert all("X-Amz" not in line for line in (tmp_path / "ax.log").read_text().splitlines()), "no presigned URL in the fake's log"
assert by_task["sig-repo-facts"]["body"]["credentials"] == {} and by_task["sig-agree"]["body"]["credentials"] == {}, "no Model"
accepted = {}
for stamp, words in fake.log(): # the last resume of each task is the one ax accepted
@@ -583,8 +611,10 @@ def test_start_is_sent_once_after_resume_with_the_models_credential(fake: Fake,
assert verify_model["spec"]["secretKey"] == {"name": "deepseek", "key": "DEEPSEEK_API_KEY"}, "the manifest names the key"
assert "DEEPSEEK_API_KEY" not in fake.applied("sig", "verify")["env"], "the rendered Task carries no credential"
rendered = str(reconciler.run_dir("sig") / "verify" / "task.yaml")
assert rendered in files_containing(tmp_path, "OUSAST_ARTIFACT_URL"), "the scan reads the rendered files it vouches for"
assert rendered in files_containing(tmp_path, "OUSAST_INPUTS"), "the scan reads the rendered files it vouches for"
assert files_containing(tmp_path, SECRET) == [], "no rendered task.yaml, state, log or applied record holds the value"
leaked = [p for p in files_containing(tmp_path, "X-Amz-Signature") if "/started/" not in p] # the fake router's own records
assert leaked == [], "nor a presigned URL in any product file (the fake store signs with that name)"
def test_a_model_bound_task_without_its_secret_fails_before_ax_sees_it(fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
@@ -631,7 +661,8 @@ def tar_of(files: dict[str, bytes]) -> bytes:
return buf.getvalue()
def test_receiver_rejects_traversal_unknown_tasks_and_wrong_types(tmp_path: Path) -> None:
def test_receiver_rejects_traversal_unknown_tasks_and_wrong_types(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OUSAST_ARTIFACT_HOST", "127.0.0.1") # the direct test address: no Service lookup, no context needed
receiver = reconciler.Receiver("r", tmp_path / "r", 0)
receiver.running.add("t")
headers = {"X-Ousast-Run": "r", "X-Ousast-Task": "t", "Content-Type": "application/x-tar"}
@@ -647,6 +678,7 @@ def test_receiver_rejects_traversal_unknown_tasks_and_wrong_types(tmp_path: Path
def test_delivery_for_a_task_that_is_not_running_is_rejected(fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OUSAST_DELIVERY", "receiver") # the receiver path, kept for group 2 (task 2.6 removes this test)
monkeypatch.setenv("OUSAST_TASK_TIMEOUT", "1") # the rejected delivery is no delivery: the task times out
script = json.loads(json.dumps(SCRIPT))
script["repo-facts"]["as_task"] = "agree" # the runner claims to be a task that has not started
@@ -719,16 +751,31 @@ def test_status_of_an_unknown_run_says_so(fake: Fake) -> None:
# --- doctor, workspaces, size ---------------------------------------------------------------------------------------
def test_doctor_names_the_bring_up_script_when_nothing_is_reachable(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
monkeypatch.setenv("PATH", str(tmp_path)) # no kubectl, no curl
def test_doctor_checks_the_profiles_cluster_and_reports_every_address(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, plane_profile: Path
) -> None:
"""Req 1.2: the configured context (never kind by name), the registry's runner reference, the memory store; the
namespaces are not checked behind an unreachable context, so only that line fails for it."""
monkeypatch.setenv("PATH", str(tmp_path)) # no kubectl, no docker
from openultrasast.plane import doctor as doctor_module
monkeypatch.setattr(doctor_module, "_registry_has", lambda image: (False, f"registry.test:5000 unreachable for {image}"))
checks = reconciler.doctor()
assert [name for name, _, _ in checks] == [
"kind cluster",
"agent substrate (ate-system)",
"ax controller (ax-system)",
"runner image in kind registry",
names = [name for name, _, _ in checks]
assert names == [
"profile test",
"cluster kind-test",
"runner image in registry.test:5000",
f"memory store file://{tmp_path / 'memory'}",
]
assert all(not ok and "ops/ax/up.sh" in text for _, ok, text in checks)
by_name = {name: (ok, text) for name, ok, text in checks}
assert by_name["profile test"][0] and "kube_context=kind-test" in by_name["profile test"][1]
for address in ("registry=registry.test:5000", "exec=local", "atespace=default", "router_url=(tunnel"):
assert address in by_name["profile test"][1], address
assert not by_name["cluster kind-test"][0] and "kind-test unreachable" in by_name["cluster kind-test"][1]
assert not by_name["runner image in registry.test:5000"][0]
assert by_name[f"memory store file://{tmp_path / 'memory'}"][0], "a file store opens; an S3 store is verified"
assert not any("kind-" + "ousast" in text for _, _, text in checks), "nothing names the kind cluster"
def test_workspaces_one_manifest_per_case_pin(tmp_path: Path) -> None:
@@ -755,6 +802,23 @@ def test_workspaces_one_manifest_per_case_pin(tmp_path: Path) -> None:
assert loaded["demo-sqli-benign-tip"].pins == {"repo": "d" * 40}
def test_cli_doctor_takes_a_profile_and_fails_on_its_unreachable_context(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, plane_profile: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""plane-on-kubernetes 1.2: ``ousast plane doctor --profile`` drives the checks from that file, nothing built."""
from openultrasast.cli import main
from openultrasast.plane import doctor as doctor_module
monkeypatch.setenv("PATH", str(tmp_path))
monkeypatch.setattr(doctor_module, "_registry_has", lambda image: (True, "present"))
monkeypatch.delenv("OUSAST_PLANE_PROFILE")
assert main(["plane", "doctor", "--profile", str(plane_profile)]) == 1
out = capsys.readouterr().out
assert "[FAIL] cluster kind-test:" in out and "[ok] profile test:" in out and "[ok] runner image in registry.test:5000" in out
assert main(["plane", "doctor", "--profile", str(tmp_path / "missing.toml")]) == 2
assert "missing.toml" in capsys.readouterr().err
def test_rendered_documents_carry_only_ax_fields(tmp_path: Path) -> None:
"""Req 1.1/1.4: whatever ``ax apply`` receives names only fields ax documents; annotations never leave."""
(tmp_path / "repo-facts").mkdir()
@@ -798,9 +862,13 @@ def test_mark_done_sets_the_state_under_the_run_lock(tmp_path: Path) -> None:
assert not (base / "lock").exists(), "the lock is released"
def test_reconciler_is_under_500_lines_and_holds_no_pipeline_logic() -> None:
def test_reconciler_is_under_550_lines_and_holds_no_pipeline_logic() -> None:
text = RECONCILER.read_text(encoding="utf-8")
assert len(text.splitlines()) < 500, "Req 3.4: the reconciler stays under 500 lines"
assert len(text.splitlines()) < 550, (
"Req 3.4: the reconciler stays under 500 lines; plane-on-kubernetes 1.6/2.1 moved the bound to 550 while the "
"store-delivery wiring (Delivery, state mirror and seed, the start-body delivery) sits beside the receiver path "
"that task 2.6 removes -- 2.6 records the count and returns the bound to 500 if it allows"
)
for word in ("prompt(", "openai", "ChatClient", "score(", "system_prompt"):
assert word not in text, f"Req 3.4: no pipeline logic in the reconciler ({word})"
assert "Popen" not in text and "openultrasast.plane.tasks" not in text, "ax is the only executor"
@@ -829,7 +897,7 @@ def test_doctor_treats_an_empty_namespace_as_not_ready(monkeypatch: pytest.Monke
from openultrasast.plane import doctor
monkeypatch.setattr(doctor, "_sh", lambda *a: (True, "No resources found in ax-system namespace."))
ok, text = doctor._pods_ready("ax-system", "kind-ousast")
ok, text = doctor._pods_ready("ax-system", "kind-test")
assert not ok and "no pods" in text
monkeypatch.setattr(doctor, "_sh", lambda *a: (True, "a-1 1/1 Running 0 1m\nb-2 0/1 Completed 0 1m"))
assert doctor._pods_ready("ns", "ctx") == (True, "2 pods ready")
@@ -854,11 +922,13 @@ def test_egress_policy_between_apply_and_resume_and_checked_after_delete(fake: F
order.append(first(prefix, order[-1] + 1))
assert order == sorted(order)
assert not (tmp_path / "policies" / "default_eg-verify.json").exists(), "the policy went with its actor"
contexts = {line.split()[-1] for line in lines if line.startswith("egress ")}
assert contexts == {PROFILE_CONTEXT}, "kubectl-ate received the profile's context, never a built one (1.2)"
written = json.loads((tmp_path / "policies" / "default_eg-verify.written.json").read_text())
tls = {"hostnames": ["api.deepseek.com", "example.invalid"], "ports": {"numbers": [443]}}
assert written["rules"] == [{"tlsPassthrough": tls}], "the Model host and the Git host, nothing else"
tls = {"hostnames": ["api.deepseek.com", "example.invalid", "localhost"], "ports": {"numbers": [443]}}
assert written["rules"] == [{"tlsPassthrough": tls}], "the Model host, the Git host and the store host, nothing else"
facts = json.loads((tmp_path / "policies" / "default_eg-repo-facts.written.json").read_text())
assert facts["rules"] == [{"tlsPassthrough": {"hostnames": ["example.invalid"], "ports": {"numbers": [443]}}}]
assert facts["rules"] == [{"tlsPassthrough": {"hostnames": ["example.invalid", "localhost"], "ports": {"numbers": [443]}}}]
def test_a_refused_egress_policy_fails_the_task_before_resume(fake: Fake, tmp_path: Path, kubectl_ate: Path) -> None:
@@ -943,3 +1013,82 @@ def test_ax_task_names_leave_room_for_the_template_suffix() -> None:
assert other != long, "no collision between tasks sharing a long prefix"
assert reconciler._ax_name("run", "facts", limit=reconciler.TASK_NAME_LIMIT) == "run-facts"
assert re.fullmatch(r"[a-z0-9]([a-z0-9-]*[a-z0-9])?", long)
# --- the store is the state of record (plane-on-kubernetes 2.1, design section 2) -----------------------------------
def test_state_is_mirrored_to_the_store_and_seeds_a_rerun_from_an_empty_results_root(
fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
assert reconciler.run(write_run(tmp_path, "mir"), ax=str(fake.ax)) == "done"
mirrored = json.loads(fake.store.objects["runs/mir/state.json"].decode())
assert mirrored == state_of("mir") and mirrored["status"] == "done", "mirrored after the last change"
applies = len(fake.events("apply"))
monkeypatch.setenv("OUSAST_RESULTS", str(tmp_path / "elsewhere")) # another machine: no local state at all
assert reconciler.run(write_run(tmp_path, "mir"), ax=str(fake.ax)) == "done"
assert statuses("mir") == {"repo-facts": "done", "verify": "done", "agree": "done"}, "seeded from the store"
assert len(fake.events("apply")) == applies, "nothing was submitted again"
def test_a_store_that_cannot_open_stops_the_run_before_ax_and_leaves_no_lock(
fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
from openultrasast.plane import memory
from openultrasast.plane.memory import MemoryStoreError
monkeypatch.setattr(reconciler, "open_store", memory.open_store) # the real one, not the fixture's fake
monkeypatch.setenv("OUSAST_MEMORY", "s3://nowhere") # the profile's override; no S3_* in the test environment
for key in ("S3_ENDPOINT", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"):
monkeypatch.delenv(key, raising=False)
with pytest.raises(MemoryStoreError, match="S3_ENDPOINT"):
reconciler.run(write_run(tmp_path, "nostore"), ax=str(fake.ax))
assert fake.events("apply") == [] and not (reconciler.run_dir("nostore") / "lock").exists()
# --- the store path end to end, and the receiver path kept for group 2 (plane-on-kubernetes 2.4) --------------------
def test_the_store_holds_the_tar_the_declared_outputs_and_done_json(fake: Fake, tmp_path: Path) -> None:
"""Completion is the tar in the store: downloaded, extracted under the run dir, declared outputs re-put as single
objects (the consumer's GET names one artifact), done.json with the summary's status and the tar's size."""
assert reconciler.run(write_run(tmp_path, "st"), ax=str(fake.ax)) == "done"
keys = sorted(k for k in fake.store.objects if k.startswith("runs/st/"))
assert keys == [
"runs/st/agree/done.json",
"runs/st/agree/output.tar",
"runs/st/repo-facts/done.json",
"runs/st/repo-facts/facts.json",
"runs/st/repo-facts/output.tar",
"runs/st/state.json",
"runs/st/verify/agreed.json",
"runs/st/verify/done.json",
"runs/st/verify/output.tar",
], "declared outputs only (summary.json is extracted locally, not declared by this Run)"
done = json.loads(fake.store.objects["runs/st/repo-facts/done.json"])
assert done["status"] == "done" and done["bytes"] == len(fake.store.objects["runs/st/repo-facts/output.tar"]) and done["files"] == 2
assert json.loads((reconciler.run_dir("st") / "repo-facts" / "facts.json").read_text()) == {"a.py": {"functions": ["f"]}}
assert fake.store.objects["runs/st/repo-facts/facts.json"] == (reconciler.run_dir("st") / "repo-facts" / "facts.json").read_bytes()
def test_a_tar_the_store_path_cannot_accept_fails_the_task(fake: Fake, tmp_path: Path) -> None:
script = json.loads(json.dumps(SCRIPT))
script["repo-facts"]["files"] = {"../escape.json": "{}", "summary.json": FACTS_SUMMARY}
fake.script(script)
assert reconciler.run(write_run(tmp_path, "esc"), ax=str(fake.ax)) == "failed"
assert "rejected member ../escape.json" in state_of("esc")["tasks"]["repo-facts"]["reason"]
assert not (reconciler.run_dir("esc").parent / "escape.json").exists()
def test_receiver_mode_still_runs_through_the_host_receiver(fake: Fake, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""OUSAST_DELIVERY=receiver (group 2 only): the old path, with its http rule and OUSAST_ARTIFACT_URL, until 2.6."""
monkeypatch.setenv("OUSAST_DELIVERY", "receiver")
assert reconciler.run(write_run(tmp_path, "rcv"), ax=str(fake.ax)) == "done"
assert fake.applied("rcv", "verify")["env"]["OUSAST_ARTIFACT_URL"].startswith("http://127.0.0.1:")
assert [s["body"].get("delivery") for s in fake.router.starts] == [None, None, None], "no presigned URLs in receiver mode"
assert not any(k.startswith("runs/rcv/") and k.endswith("output.tar") for k in fake.store.objects), "nothing PUT to the store"
assert "runs/rcv/state.json" in fake.store.objects, "the state is still mirrored"
written = json.loads((tmp_path / "policies" / "default_rcv-repo-facts.written.json").read_text())
assert written["rules"] == [{"tlsPassthrough": {"hostnames": ["example.invalid"], "ports": {"numbers": [443]}}}], (
"an IP receiver: no http rule"
)
tests/test_plane_router.py
+17 -9
View File
@@ -93,17 +93,17 @@ def alive(pid: int) -> bool:
@pytest.fixture(autouse=True)
def _clean_env(monkeypatch: pytest.MonkeyPatch) -> None:
for key in ("OUSAST_ROUTER_URL", "OUSAST_START_TIMEOUT", "KIND_CLUSTER_NAME", "FAKE_KUBECTL_MODE"):
for key in ("OUSAST_ROUTER_URL", "OUSAST_START_TIMEOUT", "FAKE_KUBECTL_MODE"):
monkeypatch.delenv(key, raising=False)
def test_port_forward_port_is_parsed_and_the_child_is_terminated_on_exit(tmp_path: Path, fake_router: FakeRouter) -> None:
port = int(fake_router.server_address[1])
kubectl = write_kubectl(tmp_path, port)
with open_router(kubectl=str(kubectl)) as router:
with open_router("kind-test", kubectl=str(kubectl)) as router: # the profile's context, as the reconciler passes it
assert router.url == f"http://127.0.0.1:{port}"
seen = json.loads((tmp_path / "kubectl.json").read_text())
assert seen["argv"] == ["--context", "kind-ousast", "-n", "ate-system", "port-forward", "svc/atenet-router", ":80"]
assert seen["argv"] == ["--context", "kind-test", "-n", "ate-system", "port-forward", "svc/atenet-router", ":80"]
assert alive(seen["pid"])
router.start_task("default", "run-t", "run", {})
deadline = time.monotonic() + 5
@@ -123,15 +123,23 @@ def test_a_failing_port_forward_raises_with_kubectls_output(tmp_path: Path, monk
def test_router_url_override_starts_no_port_forward(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None:
monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url)
with open_router(kubectl=str(tmp_path / "no-such-kubectl")) as router:
with open_router("kind-test", kubectl=str(tmp_path / "no-such-kubectl")) as router:
assert router.url == fake_router.url
assert not (tmp_path / "kubectl.json").exists()
monkeypatch.delenv("OUSAST_ROUTER_URL")
with open_router("kind-test", url=fake_router.url, kubectl=str(tmp_path / "no-such-kubectl")) as router:
assert router.url == fake_router.url, "the profile's router_url is the same override"
def test_open_router_refuses_to_build_a_context(tmp_path: Path) -> None:
with pytest.raises(StartError, match="kube context"), open_router("", kubectl=str(tmp_path / "no-such-kubectl")):
pass
@pytest.mark.parametrize("fake_router", [[(503, "no actor yet"), (502, "bad gateway"), (202, "started")]], indirect=True)
def test_start_retries_503_and_502_then_succeeds(monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None:
monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url)
with open_router() as router:
with open_router("kind-test") as router:
router.start_task("default", "chain-verify", "chain", {"DEEPSEEK_API_KEY": SECRET})
assert len(fake_router.requests) == 3
path, headers, body = fake_router.requests[-1]
@@ -142,7 +150,7 @@ def test_start_retries_503_and_502_then_succeeds(monkeypatch: pytest.MonkeyPatch
@pytest.mark.parametrize("fake_router", [[(409, f"this actor runs task 'x' (echo {SECRET})")]], indirect=True)
def test_a_409_is_raised_at_once_with_the_response_text_redacted(monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None:
monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url)
with open_router() as router, pytest.raises(StartError, match="HTTP 409.*runs task 'x'") as caught:
with open_router("kind-test") as router, pytest.raises(StartError, match="HTTP 409.*runs task 'x'") as caught:
router.start_task("default", "t", "r", {"DEEPSEEK_API_KEY": SECRET})
assert len(fake_router.requests) == 1 and SECRET not in str(caught.value)
assert router.starter("default", "t", "r", {})() is not None, "the starter returns the refusal instead of raising"
@@ -153,7 +161,7 @@ def test_a_restart_after_re_resume_accepts_a_running_command(monkeypatch: pytest
"""First start: 'already started' is a refusal. After a re-resume the actor may not have rebooted; then the
runner's 'already started on this boot' means the command still runs, not a failure."""
monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url)
with open_router() as router:
with open_router("kind-test") as router:
start = router.starter("default", "t", "r", {})
assert start() is not None, "a first start that finds the command running is a refusal"
assert start() is None, "a re-sent start after a re-resume is not"
@@ -165,7 +173,7 @@ def test_a_restart_after_re_resume_accepts_a_running_command(monkeypatch: pytest
def test_start_gives_up_after_the_start_timeout(monkeypatch: pytest.MonkeyPatch, fake_router: FakeRouter) -> None:
monkeypatch.setenv("OUSAST_ROUTER_URL", fake_router.url)
monkeypatch.setenv("OUSAST_START_TIMEOUT", "1")
with open_router() as router, pytest.raises(StartError, match="OUSAST_START_TIMEOUT.*workspaces not materialised"):
with open_router("kind-test") as router, pytest.raises(StartError, match="OUSAST_START_TIMEOUT.*workspaces not materialised"):
router.start_task("default", "t", "r", {})
assert len(fake_router.requests) >= 2
@@ -173,7 +181,7 @@ def test_start_gives_up_after_the_start_timeout(monkeypatch: pytest.MonkeyPatch,
def test_connection_errors_are_retried_until_the_timeout(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OUSAST_ROUTER_URL", "http://127.0.0.1:9")
monkeypatch.setenv("OUSAST_START_TIMEOUT", "1")
with open_router() as router, pytest.raises(StartError, match="within 1s"):
with open_router("kind-test") as router, pytest.raises(StartError, match="within 1s"):
router.start_task("default", "t", "r", {})
tests/test_plane_runner.py
+120
View File
@@ -23,6 +23,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
import pytest
from plane_fake_store import HttpObjectStore
from openultrasast.plane import egress, runner
from openultrasast.plane.manifests import GitSource
@@ -36,6 +37,7 @@ from openultrasast.plane.runner import (
clone_destination,
deliver,
find_cached_checkout,
parse_delivery,
start_health_server,
)
@@ -850,3 +852,121 @@ def test_fetch_dials_the_override_and_sends_the_url_host(tmp_path: Path) -> None
server.server_close()
assert size == 2 and (tmp_path / "x" / "a.json").read_bytes() == b"ok"
assert seen[0]["path"] == "/inputs/p/a.json" and seen[0]["Host"] == "receiver.example" and seen[0]["X-Ousast-Run"] == "r"
# --- store delivery through the start body (plane-on-kubernetes 2.2, design section 3) ---------------------------
@pytest.fixture
def object_store() -> Iterator[HttpObjectStore]:
fake = HttpObjectStore()
try:
yield fake
finally:
fake.close()
def test_the_start_body_may_carry_delivery_targets_and_refuses_unsafe_urls() -> None:
"""``delivery {put, inputs}``: https to a hostname, never an address or http; the answer echoes no URL."""
gate = StartGate()
gate.arm("r", "t")
bad = [
({"put": "http://store.example/b/k?sig=1"}, "delivery.put must be an https URL"),
({"put": "https://10.0.0.7/b/k"}, "delivery.put must name a host, not an address"),
({"put": "https://user:[email protected]/k"}, "delivery.put must be an https URL"),
({"inputs": {"FACTS": "ftp://store.example/k"}}, "delivery.inputs.FACTS must be an https URL"),
({"inputs": {"lower": "https://store.example/k"}}, "delivery.inputs must map"),
({"elsewhere": "x"}, "delivery must be an object"),
("https://store.example/k", "delivery must be an object"),
]
for delivery, message in bad:
status, text = gate.offer(json.dumps({"run": "r", "task": "t", "delivery": delivery}).encode())
assert status == 400 and message in text, (delivery, status, text)
assert "store.example" not in text and "10.0.0.7" not in text, "the answer never echoes a URL"
body = {
"run": "r",
"task": "t",
"delivery": {
"put": "https://store.example/b/t/output.tar?X-Amz-Signature=s",
"inputs": {"FACTS": "https://store.example/b/p/facts.json?X-Amz-Signature=s"},
},
}
assert gate.offer(json.dumps(body).encode()) == (202, "started")
start = gate.wait(threading.Event())
assert start is not None and start.delivery is not None
assert start.delivery.put == body["delivery"]["put"] and start.delivery.inputs == body["delivery"]["inputs"]
assert "X-Amz" not in repr(start) and "store.example" not in repr(start), "a Start never prints a URL"
assert parse_delivery(None) is None and parse_delivery({"put": "http://localhost:1/k"}, insecure=True) is not None
with pytest.raises(ValueError, match="must be an https URL"):
parse_delivery({"put": "http://localhost:1/k"})
def test_output_is_put_to_the_presigned_url_and_inputs_are_fetched_from_theirs(
env: dict[str, str], object_store: HttpObjectStore, tmp_path: Path, caplog: pytest.LogCaptureFixture
) -> None:
"""End to end against the S3-shaped fake: the start carries the URLs, the runner PUTs the tar and GETs the input;
no receiver, no OUSAST_ARTIFACT_URL, no URL in any log line."""
caplog.set_level(logging.DEBUG)
object_store._put("runs/run-1/repo-facts/facts.json", b'{"a.py": ["f"]}')
put_url = object_store.presign_put("runs/run-1/stub/output.tar")
facts_url = object_store.presign_get("runs/run-1/repo-facts/facts.json")
thread, environ, codes = main_in_thread(
tmp_path,
AX_RUNNER_INSECURE_DELIVERY="1",
OUSAST_TASK="stub",
OUSAST_INPUTS=json.dumps({"FACTS": "repo-facts/facts.json"}),
OUSAST_INPUT_FACTS=str(tmp_path / "ws" / "in" / "repo-facts" / "facts.json"),
)
body = {"run": "run-1", "task": "stub", "delivery": {"put": put_url, "inputs": {"FACTS": facts_url}}}
assert post_start(environ["AX_RUNNER_BOUND_PORT"], body) == (202, "started")
thread.join(timeout=30)
assert codes == [0]
assert (tmp_path / "ws" / "in" / "repo-facts" / "facts.json").read_bytes() == b'{"a.py": ["f"]}', "the input came by its URL"
assert ("GET", "runs/run-1/repo-facts/facts.json") in object_store.requests and (
"PUT",
"runs/run-1/stub/output.tar",
) in object_store.requests
with tarfile.open(fileobj=io.BytesIO(object_store.objects["runs/run-1/stub/output.tar"]), mode="r:") as tar:
names = sorted(tar.getnames())
assert "facts.json" in names and "summary.json" in names, "the whole output directory, as a tar, in the store"
marker = json.loads((tmp_path / "state" / "stub.done.json").read_text())
assert marker["delivered"] is True and marker["status"] == "done"
assert "X-Amz" not in caplog.text and "fake-bucket" not in caplog.text, "no presigned URL in any log line"
assert files_containing(tmp_path, "X-Amz-Signature") == [], "nor in any file"
def test_a_failed_put_is_retried_alone_on_the_next_start_with_a_fresh_url(
env: dict[str, str], object_store: HttpObjectStore, tmp_path: Path
) -> None:
"""The marker and retry-only boot are unchanged: a PUT that fails (wrong bucket: 404) ends the task failed with
the URL's host and path only; the next boot's start brings a working URL and only the delivery runs again."""
counter = tmp_path / "runs.txt"
wrong = object_store.presign_put("runs/run-1/stub/output.tar").replace("/fake-bucket/", "/no-such-bucket/")
thread, environ, codes = main_in_thread(
tmp_path, AX_RUNNER_INSECURE_DELIVERY="1", STUB_COUNTER=str(counter), OUSAST_DELIVERY_BACKOFF="0"
)
assert post_start(environ["AX_RUNNER_BOUND_PORT"], {"run": "run-1", "task": "stub", "delivery": {"put": wrong}})[0] == 202
thread.join(timeout=30)
assert codes == [EXIT_BY_STATUS["failed"]]
marker = json.loads((tmp_path / "state" / "stub.done.json").read_text())
assert marker["delivered"] is False and marker["status"] == "failed" and marker["exit"] == 0
assert "HTTP 404" in marker["reason"] and "X-Amz" not in marker["reason"], "the reason names the status, not the signature"
thread, environ, codes = main_in_thread(
tmp_path, AX_RUNNER_INSECURE_DELIVERY="1", STUB_COUNTER=str(counter), OUSAST_DELIVERY_BACKOFF="0"
)
good = object_store.presign_put("runs/run-1/stub/output.tar")
assert post_start(environ["AX_RUNNER_BOUND_PORT"], {"run": "run-1", "task": "stub", "delivery": {"put": good}})[0] == 202
thread.join(timeout=30)
assert codes == [0] and len(counter.read_text().splitlines()) == 1, "only the delivery is retried"
assert "runs/run-1/stub/output.tar" in object_store.objects
assert json.loads((tmp_path / "state" / "stub.done.json").read_text())["delivered"] is True
def test_deliver_puts_and_names_only_the_safe_url_on_failure(object_store: HttpObjectStore) -> None:
url = object_store.presign_put("k/output.tar")
assert runner.deliver(url, b"tar", {"Content-Type": "application/x-tar"}, backoff=0, method="PUT") == 200
assert object_store.objects["k/output.tar"] == b"tar"
with pytest.raises(runner.DeliveryError) as caught:
runner.deliver(url.replace("/fake-bucket/", "/other/"), b"x", {}, attempts=2, backoff=0, method="PUT")
assert "HTTP 404" in str(caught.value) and "X-Amz" not in str(caught.value) and "/other/k/output.tar" in str(caught.value)
assert runner.safe_url("https://h:1/p/q?X-Amz-Signature=s") == "https://h:1/p/q"