mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
plane(1.4): up.sh by profile -- context, registry, pool and images file from kind.toml
ops/ax/up.sh reads kube_context, registry, pools.default and the images path with `python -m openultrasast.plane.profile --print`, writes the runner digest to ops/k8s/profiles/kind-images.json instead of ~/.cache/.../runner-image, renders ops/ax/workerpool.yaml.tmpl from the profile's pool (POOL_NAME/REPLICAS/CPU/ MEMORY, WORKER_IMAGE), and keeps an existing pool and an ax CLI newer than its checkout instead of rebuilding them (ko and go install would refill several GB of Go cache on this disk). smoke-run.sh takes the runner from `--print images.runner` and runs the checkout's code with OUSAST_PYTHON. repin_templates reads the JSON images file (runner key; the one-line pin still reads). `--print images` returns the file path (the first live rerun fell over this). Templates and the generated validation-46 Tasks re-pinned to the rebuilt image (128 lines, one digest). Live: up.sh reran idempotently on kind in 38 s (every step kept except the runner build); smoke-run.sh done in 24 s on the new image, facts.json 451 B, summary.json 238 B delivered (record: benchmarks/measurements/2026-10-02-k8s-01-profile/). Gates: pytest 2010 passed, 42 skipped, exit 0; ruff check 0; ruff format 0; mypy 0; mkdocs build --strict 0. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
co-authored by
Claude Fable 5.1
Claude Fable 5.1
parent
36f5927999
commit
8099b29deb
+4
-1
@@ -12,7 +12,10 @@ single-node kind cluster. ax is the only executor; `ousast plane run` submits, r
|
||||
ops/ax/down.sh # deletes the cluster and its registry
|
||||
|
||||
Tools live in `~/go/bin` (kind, ax, ko, kubectl-ate) and `~/.local/bin` (kubectl); sources and rendered files
|
||||
in `~/.cache/ousast/ax-src/` (the runner digest pin is `runner-image` there).
|
||||
in `~/.cache/ousast/ax-src/`. The cluster's context, registry and pool sizes come from the kind profile
|
||||
(`ops/k8s/profiles/kind.toml`); `up.sh` writes the runner's digest pin to that profile's images file
|
||||
(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them.
|
||||
`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`).
|
||||
|
||||
## How a Run executes
|
||||
|
||||
|
||||
+9
-5
@@ -1,14 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# Render the end-to-end smoke Run with the current runner digest and execute it through the reconciler on ax.
|
||||
# Render the end-to-end smoke Run with the profile's runner digest and execute it through the reconciler on ax.
|
||||
# The profile (OUSAST_PLANE_PROFILE, default kind; ops/k8s/profiles/) names the images file up.sh wrote.
|
||||
set -euo pipefail
|
||||
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
|
||||
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
|
||||
RUNNER_IMAGE="$(cat "$SRC/runner-image")"; export RUNNER_IMAGE
|
||||
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
PY="${OUSAST_PYTHON:-$REPO/.venv/bin/python}"
|
||||
RUNNER_IMAGE="$(PYTHONPATH="$REPO/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print images.runner)"
|
||||
export RUNNER_IMAGE
|
||||
# Actors reach the receiver by its Service name through the egress gateway (ops/ax/receiver-service.yaml.tmpl,
|
||||
# applied by up.sh, endpoint this host at OUSAST_ARTIFACT_PORT): the runner dials the Service's ClusterIP on port
|
||||
# 80 with that name as Host; the reconciler looks the ClusterIP up unless OUSAST_ARTIFACT_DIAL is set.
|
||||
export OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}"
|
||||
envsubst < "$(dirname "$0")/smoke-run.yaml.tmpl" > "$SRC/smoke-run.yaml"
|
||||
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
"$REPO/.venv/bin/ousast" plane run "$SRC/smoke-run.yaml" "$@"
|
||||
"$REPO/.venv/bin/ousast" plane status ax-e2e-smoke
|
||||
ousast() { PYTHONPATH="$REPO/src" "$PY" -m openultrasast.cli "$@"; } # this checkout's code, whatever python runs it
|
||||
ousast plane run "$SRC/smoke-run.yaml" "$@"
|
||||
ousast plane status ax-e2e-smoke
|
||||
|
||||
+30
-12
@@ -1,13 +1,21 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bring up ax on this host: kind cluster + local registry, Agent Substrate, the ax control plane, one smoke
|
||||
# Task. Idempotent enough to rerun; `down.sh` removes everything. Requires docker, go, kubectl, kind, ko, ax.
|
||||
# Task. Idempotent: every step is skipped when already done; `down.sh` removes everything. Requires docker, go,
|
||||
# kubectl, kind, ko, ax. The cluster's context, the registry and the pool sizes come from the kind profile
|
||||
# (ops/k8s/profiles/kind.toml; OUSAST_PLANE_PROFILE selects another file with exec = "local"). This script may
|
||||
# name the kind cluster because it creates it; the code reads the profile (tests/test_plane_literals.py).
|
||||
set -euo pipefail
|
||||
trap 'rc=$?; echo "up.sh FAILED at line $LINENO (exit $rc)"' ERR
|
||||
trap 'echo "up.sh exit $?"' EXIT
|
||||
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
|
||||
export GOTOOLCHAIN=auto
|
||||
export KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-ousast}"
|
||||
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-localhost:5001}"
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
|
||||
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
|
||||
CTX="$(profile kube_context)" # kind names its context kind-<cluster>
|
||||
export KIND_CLUSTER_NAME="${CTX#kind-}"
|
||||
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}"
|
||||
IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read
|
||||
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
|
||||
|
||||
for t in docker go kubectl kind ko ax; do
|
||||
@@ -19,9 +27,8 @@ clone substrate https://github.com/agent-substrate/substrate.git
|
||||
clone ax https://github.com/google/ax.git
|
||||
step() { printf '\n== %s (%s)\n' "$1" "$(date +%H:%M:%S)"; }
|
||||
|
||||
step "kind cluster '$KIND_CLUSTER_NAME' + registry $KO_DOCKER_REPO"
|
||||
step "kind cluster '$KIND_CLUSTER_NAME' (context $CTX) + registry $KO_DOCKER_REPO"
|
||||
# Each step is skipped when already done: the substrate helper DELETES an existing cluster before creating one.
|
||||
CTX="kind-$KIND_CLUSTER_NAME"
|
||||
ready() { kubectl --context "$CTX" -n "$1" get pods --no-headers 2>/dev/null | awk '$2 ~ /^[0-9]+\/[0-9]+$/ {n++; split($2,a,"/"); if (a[1]!=a[2] && $3!="Completed") bad++} END {exit !(n>0 && bad==0)}'; }
|
||||
if kind get clusters 2>/dev/null | grep -qx "$KIND_CLUSTER_NAME"; then
|
||||
echo "cluster exists; kept"
|
||||
@@ -56,25 +63,36 @@ KIND_GATEWAY="$(docker network inspect kind --format '{{range .IPAM.Config}}{{if
|
||||
export KIND_GATEWAY OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}" # the reconciler's fixed receiver port
|
||||
envsubst < "$(dirname "$0")/receiver-service.yaml.tmpl" | kubectl --context "$CTX" apply -f -
|
||||
|
||||
step "worker pool (gVisor)"
|
||||
step "worker pool (gVisor): pools.default of the profile"
|
||||
# `ko apply` builds the worker image from the Substrate checkout (several GB of Go caches): only when the pool
|
||||
# is absent. A size change in the profile is applied with `kubectl patch` or by deleting the pool first.
|
||||
export POOL_NAME=ousast-pool POOL_REPLICAS="$(profile pools.default.replicas)" POOL_CPU="$(profile pools.default.cpu)" POOL_MEMORY="$(profile pools.default.memory)"
|
||||
export WORKER_IMAGE="ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor"
|
||||
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
|
||||
export SUBSTRATE_VERSION
|
||||
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" > "$SRC/workerpool.yaml"
|
||||
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
|
||||
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l ate.dev/worker-pool=ousast-pool --timeout=600s \
|
||||
if kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" >/dev/null 2>&1; then
|
||||
echo "pool $POOL_NAME exists ($(kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" -o jsonpath='{.status.readyReplicas}/{.spec.replicas}') ready); kept"
|
||||
else
|
||||
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
|
||||
fi
|
||||
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l "ate.dev/worker-pool=$POOL_NAME" --timeout=600s \
|
||||
|| kubectl --context "$CTX" -n ax-system get pods
|
||||
|
||||
step "runner image $KO_DOCKER_REPO/ousast-runner"
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
docker build -q -f "$REPO_ROOT/plane/Dockerfile.runner" -t "$KO_DOCKER_REPO/ousast-runner:dev" "$REPO_ROOT"
|
||||
docker push -q "$KO_DOCKER_REPO/ousast-runner:dev"
|
||||
RUNNER_IMAGE="$(docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$KO_DOCKER_REPO/ousast-runner:dev" | grep "^$KO_DOCKER_REPO/ousast-runner@" | head -1)"
|
||||
[ -n "$RUNNER_IMAGE" ] || { echo "no digest for the runner image" >&2; exit 2; }
|
||||
export RUNNER_IMAGE; echo "$RUNNER_IMAGE" > "$SRC/runner-image" # the reconciler reads this pin
|
||||
echo "runner image $RUNNER_IMAGE"
|
||||
export RUNNER_IMAGE
|
||||
printf '{\n "runner": "%s"\n}\n' "$RUNNER_IMAGE" > "$IMAGES_FILE" # the profile's images file: commit it with the templates
|
||||
echo "runner image $RUNNER_IMAGE -> $IMAGES_FILE"
|
||||
echo "re-pin the templates with: ousast plane workspaces ... --runner-image $IMAGES_FILE (or ousast plane repin, task 4.3)"
|
||||
|
||||
step "ax CLI from the deployed checkout (a release CLI skews from the server)"
|
||||
(cd "$SRC/ax" && go install ./cmd/ax)
|
||||
if [ -x "$HOME/go/bin/ax" ] && [ "$HOME/go/bin/ax" -nt "$SRC/ax/.git/HEAD" ]; then echo "ax CLI newer than the checkout; kept"; else
|
||||
(cd "$SRC/ax" && go install ./cmd/ax)
|
||||
fi
|
||||
|
||||
step "smoke task"
|
||||
ax delete task ousast-smoke >/dev/null 2>&1 || true
|
||||
|
||||
@@ -1,23 +1,26 @@
|
||||
# Warm gVisor workers for ax tasks on this host (Agent Substrate WorkerPool). ax's per-task ActorTemplates
|
||||
# set no workerSelector, so every pool of the matching sandbox class is eligible. Two workers of 1 CPU /
|
||||
# 1.5 GiB fit the 7 GB host beside the control planes; an actor occupies a whole worker.
|
||||
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
|
||||
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
|
||||
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
|
||||
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
|
||||
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
|
||||
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
|
||||
apiVersion: ate.dev/v1alpha1
|
||||
kind: WorkerPool
|
||||
metadata:
|
||||
name: ousast-pool
|
||||
name: ${POOL_NAME}
|
||||
namespace: ax-system
|
||||
labels:
|
||||
workload: ousast
|
||||
spec:
|
||||
replicas: 2
|
||||
workerImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor
|
||||
replicas: ${POOL_REPLICAS}
|
||||
workerImage: ${WORKER_IMAGE}
|
||||
template:
|
||||
nodeSelector:
|
||||
ate.dev/substrate-version: "${SUBSTRATE_VERSION}"
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 1536Mi
|
||||
cpu: "${POOL_CPU}"
|
||||
memory: ${POOL_MEMORY}
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 1536Mi
|
||||
memory: ${POOL_MEMORY}
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
"runner": "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
"runner": "localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user