plane(1.3): host-literal scan test over src, plane, ops and docs

tests/test_plane_literals.py scans src/, plane/, ops/ (minus ops/ax/up.sh and
ops/k8s/profiles/) and docs/ for the kind context, the kind registry, the kind
network's gateway, a loopback address with a port and port-forward; router.py
keeps the last two, a line tagged ax-tunnel or loopback is allowed, and a
committed template's digest-pinned image is accepted only while it names the
kind profile's registry (checked by a third test). A self-check reinserts each
literal into a fixture and sees it caught. The runner's health URL and the Joern
session's query URL are tagged loopback; docs/deployment.md and ops/ax/README.md
stop naming this host and describe the kube context and the registry as the
profile's (what 1.2 made configurable).

Gates: pytest 2009 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0; mkdocs build --strict 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 20:32:51 +02:00
co-authored by Claude Fable 5.1
parent 77a03bad61
commit 36f5927999
6 changed files with 153 additions and 40 deletions
+1 -1
View File
@@ -29,7 +29,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
profile's context, never a built one. profile's context, never a built one.
- Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile. - Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile.
- _Requirements: 1.1, 1.2_ - _Requirements: 1.1, 1.2_
- [ ] 1.3 Host-literal scan test - [x] 1.3 Host-literal scan test
- `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does) - `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does)
over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`, over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`,
`localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged `localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged
+31 -32
View File
@@ -27,11 +27,11 @@ flowchart LR
sub["Agent Substrate (ate-system): atenet-router, atenet-egress"] sub["Agent Substrate (ate-system): atenet-router, atenet-egress"]
act["Task actors in gVisor workers (WorkerPool ousast-pool)"] act["Task actors in gVisor workers (WorkerPool ousast-pool)"]
svc["Service ousast-receiver + EndpointSlice to the host"] svc["Service ousast-receiver + EndpointSlice to the host"]
reg["Registry localhost:5001: runner image by digest"] reg["Registry named by the profile: runner image by digest"]
end end
mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")] mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")]
rec -- "ax apply, resume, get, delete" --> ax rec -- "ax apply, resume, get, delete" --> ax
rec -- "EgressPolicy per actor; start request via port-forward" --> sub rec -- "EgressPolicy per actor; start request through the kube-context tunnel" --> sub
sub --> act sub --> act
act -- "HTTP by Service name through the gateway" --> svc act -- "HTTP by Service name through the gateway" --> svc
svc --> rcv svc --> rcv
@@ -46,8 +46,8 @@ flowchart LR
| How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` | | How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` |
| ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` | | ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` |
| Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` | | Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` |
| Runner image | the kind-local registry `localhost:5001`, pinned by digest (`~/.cache/ousast/ax-src/runner-image`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` | | Runner image | the registry the profile names (`registry` in `ops/k8s/profiles/kind.toml`: the kind-local one), pinned by digest in the profile's images file (`ops/k8s/profiles/kind-images.json`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through a `kubectl port-forward` to `svc/atenet-router` | `src/openultrasast/plane/router.py` | | Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through the tunnel `router.py` opens to `svc/atenet-router` in the profile's kube context | `src/openultrasast/plane/router.py` |
| Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://<bucket>[/<prefix>]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) | | Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://<bucket>[/<prefix>]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) |
| Run state and artifacts | `$OUSAST_RESULTS/plane/<run>/` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) | | Run state and artifacts | `$OUSAST_RESULTS/plane/<run>/` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) |
@@ -67,8 +67,8 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
``` ```
Idempotent: each step is skipped when already done. It creates the kind cluster Idempotent: each step is skipped when already done. It creates the kind cluster
(`KIND_CLUSTER_NAME`, default `ousast`) with its registry (`KO_DOCKER_REPO`, default (`KIND_CLUSTER_NAME`, default `ousast`) with its registry (`KO_DOCKER_REPO`, default: the
`localhost:5001`), installs Agent Substrate, applies the egress gateway (the agentgateway `registry` of `ops/k8s/profiles/kind.toml`), installs Agent Substrate, applies the egress gateway (the agentgateway
variant, one prebuilt image) when the install left it out, deploys the ax control plane, applies variant, one prebuilt image) when the install left it out, deploys the ax control plane, applies
the receiver Service with the host's kind-gateway address as its endpoint, applies the gVisor the receiver Service with the host's kind-gateway address as its endpoint, applies the gVisor
worker pool, builds and pushes the runner image and writes its digest pin to worker pool, builds and pushes the runner image and writes its digest pin to
@@ -79,12 +79,15 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
2. **Check it.** 2. **Check it.**
```bash ```bash
uv run ousast plane doctor uv run ousast plane doctor --profile kind
``` ```
Four checks: the kind context is reachable, every pod in `ate-system` and in `ax-system` is ready, The checks of the profile (`--profile`, default `kind`; `ops/k8s/profiles/`): its configured addresses
and the registry at `localhost:5001` holds `ousast-runner`. Each failing line names on the first line, the kube context reachable, every pod in `ate-system` and in `ax-system` ready, the
`ops/ax/up.sh` as the fix. `doctor` is informational: `ousast plane run` does not call it. runner image of the profile's images file present in its registry (a local registry over its v2 API, a
remote one through `docker manifest inspect`), and the memory store opening (an S3 bucket is verified).
Each failing line names the fix (`ops/ax/up.sh` for the kind profile). `doctor` is informational:
`ousast plane run` does not call it.
3. **Run the end-to-end smoke Run.** 3. **Run the end-to-end smoke Run.**
@@ -130,7 +133,7 @@ produced, and what each one taught, are listed in
The target: a cluster that runs Agent Substrate with its egress gateway and the ax control plane, The target: a cluster that runs Agent Substrate with its egress gateway and the ax control plane,
pulls the runner image from a registry of yours, receives task artifacts without a developer pulls the runner image from a registry of yours, receives task artifacts without a developer
laptop in the path, and keeps its memory in your S3 bucket. The provider key already travels only laptop in the path, and keeps its memory in your S3 bucket. The provider key already travels only
in the start request through `atenet-router`, which works the same through a port-forward to any in the start request through `atenet-router`, which works the same through the kube-context tunnel to any
cluster. Everything else below is either a step you take or a code change that is not made yet. cluster. Everything else below is either a step you take or a code change that is not made yet.
### B.1 Cluster with Agent Substrate, its egress gateway and ax ### B.1 Cluster with Agent Substrate, its egress gateway and ax
@@ -172,26 +175,22 @@ owns.
reference. Write it into a file (on this host, `~/.cache/ousast/ax-src/runner-image`) and pass reference. Write it into a file (on this host, `~/.cache/ousast/ax-src/runner-image`) and pass
`--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`, which re-pin the `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`, which re-pin the
Tasks they generate. The committed templates under `plane/tasks/` and the generated Tasks they generate. The committed templates under `plane/tasks/` and the generated
`plane/tasks/validation-46.yaml` carry `localhost:5001/ousast-runner@sha256:...` and need the same `plane/tasks/validation-46.yaml` carry the kind profile's registry and digest and need the same
re-pin (regenerate, or replace the reference) before they run elsewhere. re-pin (regenerate, or replace the reference) before they run elsewhere.
- Workers must be able to pull from the registry (credentials, if any, are the cluster's concern; - Workers must be able to pull from the registry (credentials, if any, are the cluster's concern;
ax's default runner image on `gcr.io` needs credentials and is not used here). ax's default runner image on `gcr.io` needs credentials and is not used here).
- **Not yet configurable:** `ousast plane doctor` checks the registry at `localhost:5001` by name - `ousast plane doctor --profile <name>` resolves the runner reference of the profile's images file in the
(`src/openultrasast/plane/doctor.py:45`), so its fourth check fails on another cluster even when profile's `registry` (implemented 2026-10-02, plane-on-kubernetes task 1.2). `doctor` is informational,
the image is in place. `doctor` is informational, `plane run` does not depend on it. `plane run` does not depend on it.
### B.4 One configurable kube context ### B.4 One configurable kube context (implemented 2026-10-02)
The reconciler, the egress client and the router port-forward all use the context The reconciler, the egress client and the router tunnel take the kube context from the profile
`kind-$KIND_CLUSTER_NAME` (`kind-ousast` by default). **Not yet configurable**: the prefix `kind-` (`kube_context` in `ops/k8s/profiles/<name>.toml`, `OUSAST_KUBE_CONTEXT` overrides it; plane-on-kubernetes
is hard-coded in `src/openultrasast/plane/doctor.py:39`, `src/openultrasast/plane/egress.py:115` task 1.2): `doctor.py`, `egress.py` and `router.py` are handed it and build none, and
and `:148`, and `src/openultrasast/plane/router.py:145`; `ops/ax/up.sh:24` builds the same name. `tests/test_plane_literals.py` fails on any module, manifest or page that names this host's cluster,
The planned change is one variable (for example `OUSAST_KUBE_CONTEXT`) read in those three registry or addresses. The egress client also needs `kubectl-ate` (or the executable
modules. Until then the only way to drive another cluster from the unchanged code is to give its `OUSAST_KUBECTL_ATE` names) able to use that context.
kubeconfig context a name of that form (`kubectl config rename-context <ctx> kind-<name>` and
`KIND_CLUSTER_NAME=<name>`), which is a workaround, not a supported configuration. The egress
client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names) able to use that
context.
### B.5 The receiver: in-cluster instead of on the host ### B.5 The receiver: in-cluster instead of on the host
@@ -255,10 +254,10 @@ in-cluster is itself planned: today it is a host process, and its state director
| Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented | | Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented |
| S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) | | S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) |
| Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented | | Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented |
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates still name `localhost:5001` | | Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates name the kind profile's registry (`tests/test_plane_literals.py` checks it) |
| Deployment to a cluster other than this host's kind cluster | **not done** | | Deployment to a cluster other than this host's kind cluster | **not done** |
| One configurable kube context | **planned**; `kind-` hard-coded in `doctor.py`, `egress.py`, `router.py` | | One configurable kube context | implemented (2026-10-02): `kube_context` in the profile, `OUSAST_KUBE_CONTEXT` overrides |
| `doctor` registry check against a configurable registry | **planned**; `localhost:5001` hard-coded in `doctor.py:45` | | `doctor` registry check against a configurable registry | implemented (2026-10-02): the profile's `registry` and images file |
| Receiver as an in-cluster Deployment | **planned** (code change) | | Receiver as an in-cluster Deployment | **planned** (code change) |
| Presigned-URL delivery to the S3 store | **follow-on**, not started | | Presigned-URL delivery to the S3 store | **follow-on**, not started |
| Reconciler running in-cluster with Secrets injected into its environment | **planned** | | Reconciler running in-cluster with Secrets injected into its environment | **planned** |
@@ -318,7 +317,7 @@ device and does not appear in production.
**ax is a Kubernetes application**: a Deployment `ax-server` with a Redis beside it in the **ax is a Kubernetes application**: a Deployment `ax-server` with a Redis beside it in the
namespace `ax-system` (`ax/deploy/ax-server.yaml` in the ax checkout; its controller runs with namespace `ax-system` (`ax/deploy/ax-server.yaml` in the ax checkout; its controller runs with
`--template=default-template --template-atespace=ax-system`), and the `ax` CLI reaches it through a `--template=default-template --template-atespace=ax-system`), and the `ax` CLI reaches it through a
`kubectl port-forward` to `svc/ax-server` (`ax/internal/tunnel/tunnel.go`), so `ousast plane run` a tunnel to `svc/ax-server` through the kube context (`ax/internal/tunnel/tunnel.go`), so `ousast plane run`
needs a kube context or an ingress, not a public endpoint. **ax does not run containers itself**: needs a kube context or an ingress, not a public endpoint. **ax does not run containers itself**:
it turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`, it turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
`CreateActorTemplate`), built from the Task's image and environment, and the image must be pinned `CreateActorTemplate`), built from the Task's image and environment, and the image must be pinned
@@ -341,7 +340,7 @@ instead of a kubeconfig) and local-path storage. Images are built by GitHub Acti
commits, published to GitHub Container Registry and digest-pinned in the manifests; a private commits, published to GitHub Container Registry and digest-pinned in the manifests; a private
package needs an `imagePullSecret` on the worker ServiceAccount. Substrate's and ax's own images package needs an `imagePullSecret` on the worker ServiceAccount. Substrate's and ax's own images
are built with `ko` from their checkouts and pushed to the same registry. The kind profile keeps are built with `ko` from their checkouts and pushed to the same registry. The kind profile keeps
`localhost:5001` and `ops/ax/up.sh`. its local registry (`ops/k8s/profiles/kind.toml`) and `ops/ax/up.sh`.
### One image, and what concurrency equals ### One image, and what concurrency equals
@@ -369,7 +368,7 @@ applied by `ops/ax/up.sh`; the production profile's pool manifests and autoscali
| Component | Today (the kind profile) | Production (the k3s profile, planned) | | Component | Today (the kind profile) | Production (the k3s profile, planned) |
| --- | --- | --- | | --- | --- | --- |
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against `kind-ousast`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step | | Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against the profile's `kube_context`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
| Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction | | Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction |
| Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path | | Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path |
+5 -5
View File
@@ -7,7 +7,7 @@ single-node kind cluster. ax is the only executor; `ousast plane run` submits, r
ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway, ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway,
# receiver Service, runner image (digest pinned), ax CLI, smoke Task # receiver Service, runner image (digest pinned), ax CLI, smoke Task
ousast plane doctor # kind, Substrate, ax controller, runner image ousast plane doctor # the profile's cluster (ops/k8s/profiles/kind.toml), Substrate, ax, runner image, memory store
ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table
ops/ax/down.sh # deletes the cluster and its registry ops/ax/down.sh # deletes the cluster and its registry
@@ -192,16 +192,16 @@ assume this laptop and must change first (checked 2026-10-02):
| Assumption | Where | Needed in a real cluster | | Assumption | Where | Needed in a real cluster |
|---|---|---| |---|---|---|
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind gateway `172.19.0.1:18090` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it | | Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind network's gateway address on `OUSAST_ARTIFACT_PORT` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
| Registry `localhost:5001`, rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), `doctor.py:45` (hard-coded check), the runner digest pin, the `image` of every template under `plane/tasks/` | a registry the workers can pull from; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks | | The kind-local registry (`registry` in `ops/k8s/profiles/kind.toml`), rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), the profile's images file, the `image` of every template under `plane/tasks/` | the `k3s` profile's registry; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
| kubectl context `kind-$KIND_CLUSTER_NAME` | `doctor.py:39`, `egress.py:115` and `:148`, `router.py:145`, `up.sh:24` | one configurable context (for example `OUSAST_KUBE_CONTEXT`); not yet configurable | | kubectl context | `kube_context` in the profile, read by `doctor.py`, `egress.py`, `router.py` (plane-on-kubernetes 1.2); `up.sh` reads the same profile | the `k3s` profile's context; nothing else changes |
| ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` in ax's `deploy/ax-server.yaml` points at the ax authors' GCS bucket | ax deploy manifest | your own bucket, set before deploying ax | | ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` in ax's `deploy/ax-server.yaml` points at the ax authors' GCS bucket | ax deploy manifest | your own bucket, set before deploying ax |
| Egress gateway applied by hand (agentgateway variant, no Rust build) | this README | the Substrate-installed gateway; per-task EgressPolicies work unchanged | | Egress gateway applied by hand (agentgateway variant, no Rust build) | this README | the Substrate-installed gateway; per-task EgressPolicies work unchanged |
| Worker pool of 2 x 1 CPU / 1.5 GiB for the 7 GB host | `workerpool.yaml.tmpl` | sized to the cluster; `--workers` to match | | Worker pool of 2 x 1 CPU / 1.5 GiB for the 7 GB host | `workerpool.yaml.tmpl` | sized to the cluster; `--workers` to match |
| Plane memory store and results under `~/ousast-results/` on the host | `reconciler.py` (`OUSAST_RESULTS`), `memory.py` (`OUSAST_MEMORY`) | a persistent volume or bucket shared by the reconciler; the `s3://` store already works against any reachable S3-compatible server | | Plane memory store and results under `~/ousast-results/` on the host | `reconciler.py` (`OUSAST_RESULTS`), `memory.py` (`OUSAST_MEMORY`) | a persistent volume or bucket shared by the reconciler; the `s3://` store already works against any reachable S3-compatible server |
The provider key already travels only in the start request through `atenet-router`, which works the same through The provider key already travels only in the start request through `atenet-router`, which works the same through
a port-forward to any cluster. the kube-context tunnel to any cluster.
## What a larger ax deployment needs ## What a larger ax deployment needs
+3 -1
View File
@@ -162,7 +162,9 @@ class EngineSession:
return None return None
payload = json.dumps({"query": code}).encode() payload = json.dumps({"query": code}).encode()
call = urllib.request.Request( call = urllib.request.Request(
f"http://127.0.0.1:{self._port}/query-sync", data=payload, headers={"Content-Type": "application/json"} f"http://127.0.0.1:{self._port}/query-sync", # loopback: the Joern server this process started
data=payload,
headers={"Content-Type": "application/json"},
) )
try: try:
with urllib.request.urlopen(call, timeout=timeout) as response: # noqa: S310 -- fixed loopback URL with urllib.request.urlopen(call, timeout=timeout) as response: # noqa: S310 -- fixed loopback URL
+1 -1
View File
@@ -925,7 +925,7 @@ def main(argv: Sequence[str] | None = None, environ: MutableMapping[str, str] |
metadata = {"task": env.get("AX_TASK_YAML", ""), "workspaces": env.get("AX_WORKSPACES_YAML", "")} metadata = {"task": env.get("AX_TASK_YAML", ""), "workspaces": env.get("AX_WORKSPACES_YAML", "")}
server = start_health_server(int(env.get("AX_RUNNER_PORT", str(DEFAULT_PORT))), ready, metadata, gate) server = start_health_server(int(env.get("AX_RUNNER_PORT", str(DEFAULT_PORT))), ready, metadata, gate)
env["AX_RUNNER_BOUND_PORT"] = str(server.port) env["AX_RUNNER_BOUND_PORT"] = str(server.port)
env["AX_METADATA_URL"] = f"http://127.0.0.1:{server.port}" env["AX_METADATA_URL"] = f"http://127.0.0.1:{server.port}" # loopback: this runner's own health server
log.info("health and metadata server on port %d", server.port) log.info("health and metadata server on port %d", server.port)
try: try:
if server is None and env.get("AX_RUNNER_AUTOSTART") != "1": if server is None and env.get("AX_RUNNER_AUTOSTART") != "1":
+112
View File
@@ -0,0 +1,112 @@
"""plane-on-kubernetes Req 1.3: no module, manifest or page assumes this host.
The scan covers ``src/``, ``plane/``, ``ops/`` (minus ``ops/ax/up.sh``, which creates the kind cluster and may name
it, and ``ops/k8s/profiles/``, which *are* the configuration) and ``docs/`` for the kind context, the kind registry,
the kind network's gateway, a loopback address with a port, and ``port-forward``. Allowed: ``port-forward`` and the
loopback address inside ``router.py`` (the one module that opens the tunnel), and any line tagged ``ax-tunnel``
(the tunnel is how both profiles reach ax and the router) or ``loopback`` (a process talking to itself). A
digest-pinned ``image:`` line of a committed Task template under ``plane/tasks/`` is the kind profile's own value,
written there by ``ousast plane`` repinning from ``ops/k8s/profiles/kind-images.json``; it is accepted only while it
names that profile's registry, so the templates and the profile cannot drift apart. The patterns are built at runtime
so this file does not match them (as ``test_removed_plane_references.py`` does).
"""
from __future__ import annotations
import re
from pathlib import Path
from openultrasast.plane.profile import load_profile
ROOT = Path(__file__).resolve().parents[1]
SCANNED = ("src", "plane", "ops", "docs")
EXEMPT = (Path("ops/ax/up.sh"), Path("ops/k8s/profiles"))
ROUTER = Path("src/openultrasast/plane/router.py")
PATTERNS = {
"kind context": re.compile("kind-" + "ousast"),
"kind registry": re.compile("localhost:" + "5001"),
"kind gateway": re.compile(r"172\.19\."),
"loopback with port": re.compile(r"127\.0\.0\.1" + ":"),
"port-forward": re.compile("port-" + "forward"),
}
ROUTER_ONLY = {"loopback with port", "port-forward"}
TAGS = re.compile(r"ax-tunnel|loopback")
_IMAGE_LINE = re.compile(r'^\s*image:\s*"?(?P<ref>\S+?)@sha256:[0-9a-f]{64}"?\s*(#.*)?$')
def _files() -> list[Path]:
out = []
for top in SCANNED:
for path in sorted((ROOT / top).rglob("*")):
relative = path.relative_to(ROOT)
if not path.is_file() or "__pycache__" in relative.parts or path.suffix in (".png", ".jpg", ".gz", ".zip"):
continue
if any(relative == e or e in relative.parents for e in EXEMPT):
continue
out.append(path)
return out
def _pinned_template_image(relative: Path, line: str, registry: str) -> bool:
"""A committed template's digest-pinned image in the kind profile's registry: configuration, not a literal."""
if relative.parts[:2] != ("plane", "tasks"):
return False
found = _IMAGE_LINE.match(line)
return bool(found) and found.group("ref").startswith(registry + "/")
def offenders(paths: list[Path], registry: str, root: Path = ROOT) -> list[str]:
found = []
for path in paths:
relative = path.relative_to(root)
try:
text = path.read_text(encoding="utf-8")
except (UnicodeDecodeError, OSError):
continue
for number, line in enumerate(text.splitlines(), 1):
for label, pattern in PATTERNS.items():
if not pattern.search(line):
continue
if label in ROUTER_ONLY and relative == ROUTER:
continue
if TAGS.search(line) or _pinned_template_image(relative, line, registry):
continue
found.append(f"{relative}:{number}: [{label}] {line.strip()[:120]}")
return found
def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None:
kind = load_profile("kind", environ={})
found = offenders(_files(), kind.registry)
assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found)
def test_the_scan_sees_a_reinserted_literal(tmp_path: Path) -> None:
"""The instrument is not vacuous: each pattern is caught, a tag or the router exempts only what it says."""
kind = load_profile("kind", environ={})
src = tmp_path / "src"
src.mkdir()
(src / "a.py").write_text(f'ctx = "{"kind-" + "ousast"}"\nreg = "{"localhost:" + "5001"}"\nip = "172.19.0.1"\n', encoding="utf-8")
(src / "b.py").write_text('url = "http://127.0.0.1:8080" # loopback, this process\nargv = ["port-' + 'forward"] # ax-tunnel\n')
(src / "c.py").write_text('argv = ["port-' + 'forward"]\n', encoding="utf-8")
paths = sorted(src.glob("*.py"))
found = [o.split(": ", 1)[1] for o in offenders(paths, kind.registry, root=tmp_path)]
assert found == [
"[kind context] ctx = " + '"kind-' + 'ousast"',
"[kind registry] reg = " + '"localhost:' + '5001"',
'[kind gateway] ip = "172.19.0.1"',
"[port-forward] argv = " + '["port-' + 'forward"]',
]
def test_the_committed_templates_name_the_kind_profiles_registry() -> None:
"""The one exemption is checked, not assumed: every template image is in the kind registry, by digest."""
kind = load_profile("kind", environ={})
images = []
for path in sorted((ROOT / "plane/tasks").glob("*.yaml")):
for line in path.read_text(encoding="utf-8").splitlines():
found = _IMAGE_LINE.match(line)
if found:
images.append(found.group("ref"))
assert found.group("ref").startswith(kind.registry + "/"), f"{path.name}: {line.strip()}"
assert images, "the templates carry digest-pinned images"