marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

plane(1.5): placement probe -- Substrate places by any free worker, not by fit

Browse Source 
ops/ax/probe-placement.sh scales ousast-pool to one worker, applies probe-pool
(1 x 2 CPU / 3Gi, the already-built ateom-gvisor image, no ko build), runs one
model-free repo-facts Task requesting 2560Mi (limit 3Gi) and one requesting
256Mi, records `kubectl ate get workers -o json` after each resume, and removes
everything it created. Live on kind: both actors landed on the 1536Mi
ousast-pool worker (status.allocated.actors=1 there, 0 on probe-pool) ->
placement: any. design.md section 4 records the outcome and the fallback group
5 must take (an engine atespace with engine-pool as its only pool if Substrate
scopes pools by atespace; else a plain Kubernetes Job on k3s). Evidence under
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/.

Gates (tree identical to 1.4 in src/ and tests/; ops/ gains the probe script,
which the literal scan covers): pytest 2010 passed, 42 skipped, exit 0; ruff
check 0; ruff format 0; mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 20:49:03 +02:00
co-authored by Claude Fable 5.1
parent 8099b29deb
commit 86ec237483
9 changed files with 387 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/plane-on-kubernetes/design.md
+7
View File
@@ -254,6 +254,13 @@ containerd; `Dockerfile:1-7` already argues against it).
a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a
separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays
`alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such.
**Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.**
With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor
requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS
1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so
group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether
Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the
k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path.
- **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and
`docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose
`engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`.
.kiro/specs/plane-on-kubernetes/tasks.md
+1 -1
View File
@@ -42,7 +42,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
`runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3).
- Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes.
- _Requirements: 1.1, 2.4_
- [ ] 1.5 Placement probe (open question 2, decided before the engine work)
- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any`
- `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's
engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records
with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after.
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-big.yaml
+16
View File
@@ -0,0 +1,16 @@
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: ousast-probe-big-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: ousast-probe-big, atespace: default}
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "2560Mi"}, limits: {cpu: "1", memory: "3Gi"}}
workspaces: [{name: ousast-probe-big-ws, path: "/workspace"}]
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/ousast-probe-small.yaml
+16
View File
@@ -0,0 +1,16 @@
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: ousast-probe-small-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: ousast-probe-small, atespace: default}
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "256Mi"}, limits: {cpu: "1", memory: "512Mi"}}
workspaces: [{name: ousast-probe-small-ws, path: "/workspace"}]
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/probe-pool.yaml
+26
View File
@@ -0,0 +1,26 @@
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
name: probe-pool
namespace: ax-system
labels:
workload: ousast
spec:
replicas: 1
workerImage: localhost:5001/ateom-gvisor-715889664656de67e44382a8d6ab981d@sha256:220f1cf13ea485b73a2e4270fa544901a18ae826d8f22bd54d42761944a41c5c
template:
nodeSelector:
ate.dev/substrate-version: "4f9c293"
resources:
limits:
cpu: "2"
memory: 3Gi
requests:
cpu: 250m
memory: 3Gi
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-before.json
+78
View File
@@ -0,0 +1,78 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T05:24:59.069304031Z",
"version": "1758"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:29:30.551668391Z",
"version": "2"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-big.json
+81
View File
@@ -0,0 +1,81 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T18:29:45.281958347Z",
"version": "1761"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"allocated": {
"actors": 1
},
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:29:30.551668391Z",
"version": "2"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/workers-ousast-probe-small.json
+81
View File
@@ -0,0 +1,81 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T18:29:53.001457323Z",
"version": "1763"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"allocated": {
"actors": 1
},
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:30:04.825058843Z",
"version": "4"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
ops/ax/probe-placement.sh
Executable
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# Placement probe (plane-on-kubernetes task 1.5, design open question 2): does Agent Substrate place an actor on a
# worker that fits its memory request, or on any free worker of the sandbox class?
#
# Applies a second WorkerPool `probe-pool` (1 replica, 2 CPU / 3Gi) from ops/ax/workerpool.yaml.tmpl with the worker
# image the existing pool already built (no ko build), scales the default pool to one worker so the 7 GB host holds
# both, then runs one model-free repo-facts Task asking 2560Mi (limit 3Gi) and one asking 256Mi, recording after
# each resume which worker holds the actor (`kubectl ate get workers -o json`). Everything it created is removed
# on exit and the default pool is scaled back. Usage: ops/ax/probe-placement.sh OUT_DIR (the JSON evidence lands there).
set -euo pipefail
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
OUT="${1:?usage: probe-placement.sh OUT_DIR}"
mkdir -p "$OUT"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
CTX="$(profile kube_context)"
RUNNER_IMAGE="$(profile images.runner)"
DEFAULT_REPLICAS="$(profile pools.default.replicas)"
K() { kubectl --context "$CTX" -n ax-system "$@"; }
log() { printf '%s %s\n' "$(date +%H:%M:%S)" "$*" | tee -a "$OUT/probe.log"; }
cleanup() {
rc=$?
log "cleanup (exit $rc): delete the probe tasks and probe-pool, scale ousast-pool back to $DEFAULT_REPLICAS"
for t in ousast-probe-big ousast-probe-small; do ax delete task "$t" >/dev/null 2>&1 || true; done
K delete workerpool probe-pool --ignore-not-found >/dev/null 2>&1 || true
K patch workerpool ousast-pool --type merge -p "{\"spec\":{\"replicas\":$DEFAULT_REPLICAS}}" >/dev/null 2>&1 || true
echo "probe exit $rc"
}
trap cleanup EXIT
workers() { kubectl ate --context "$CTX" get workers -o json > "$OUT/$1.json"; kubectl ate --context "$CTX" get workers | tee -a "$OUT/probe.log"; }
export POOL_NAME=probe-pool POOL_REPLICAS=1 POOL_CPU=2 POOL_MEMORY=3Gi
WORKER_IMAGE="$(K get workerpool ousast-pool -o jsonpath='{.spec.workerImage}')" # already built and pulled by the node
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
export WORKER_IMAGE SUBSTRATE_VERSION
log "default pool to 1 replica (room for a 3Gi worker on this host)"
K patch workerpool ousast-pool --type merge -p '{"spec":{"replicas":1}}'
for i in $(seq 1 60); do [ "$(K get pods -l ate.dev/worker-pool=ousast-pool --no-headers 2>/dev/null | wc -l)" = 1 ] && break; sleep 2; done
log "apply probe-pool: $POOL_REPLICAS x $POOL_CPU CPU / $POOL_MEMORY, image $WORKER_IMAGE"
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" | tee "$OUT/probe-pool.yaml" | K apply -f -
K wait --for=condition=Ready pod -l ate.dev/worker-pool=probe-pool --timeout=600s
workers workers-before
task() { # name, request memory, limit memory
cat > "$OUT/$1.yaml" <<EOF
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: $1-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: $1, atespace: default}
spec:
image: "$RUNNER_IMAGE"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "$2"}, limits: {cpu: "1", memory: "$3"}}
workspaces: [{name: $1-ws, path: "/workspace"}]
EOF
ax delete task "$1" >/dev/null 2>&1 || true
log "apply $1 (requests $2, limits $3)"
ax apply -f "$OUT/$1.yaml"
for i in 1 2 3 4 5 6 7 8 9; do
if ax resume task "$1" 2>&1 | tee -a "$OUT/probe.log"; then break; fi # the first resume may wait for the golden snapshot
sleep 20
done
sleep 15
ax get task "$1" | tee "$OUT/$1.status.yaml" | grep -E "phase|message" || true
workers "workers-$1"
ax delete task "$1" >/dev/null 2>&1 || true
for i in $(seq 1 30); do kubectl ate --context "$CTX" get workers | grep -q " 1/1000 " || break; sleep 2; done
}
task ousast-probe-big 2560Mi 3Gi
task ousast-probe-small 256Mi 512Mi
log "done; evidence under $OUT"