mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
ops(kind): skip Substrate's observability stack by default (930 MiB of 2.3 GB); opt in with OUSAST_KIND_OBSERVABILITY=1
A lean overlay omits otel-collector, prometheus and jaeger and disables the exporters; on an existing cluster up.sh scales them to 0. Verified live: render has no otel-system resources, up.sh idempotent (54 s second run), smoke Run done, kind node 2.66 -> 1.82 GiB. Docs say server VM. Implemented with Codex. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
e555ba68dc
commit
0d78d33fd2
+18
-3
@@ -17,6 +17,21 @@ This page uses two words strictly:
|
|||||||
been made.** Section B is guidance, not a record. (kind runs a Kubernetes cluster inside Docker
|
been made.** Section B is guidance, not a record. (kind runs a Kubernetes cluster inside Docker
|
||||||
containers on one machine.)
|
containers on one machine.)
|
||||||
|
|
||||||
|
The kind profile runs on a 7.7 GB server VM. Observability is off by default
|
||||||
|
(`kind_observability = false` in `ops/k8s/profiles/kind.toml`). This saves the measured
|
||||||
|
930 MiB used by `otel-system`, about 40% of the cluster's 2.3 GB. The smoke Run still
|
||||||
|
completes with those deployments stopped. To enable the stack, run
|
||||||
|
`OUSAST_KIND_OBSERVABILITY=1 ops/ax/up.sh` or set `kind_observability = true` in the profile.
|
||||||
|
|
||||||
|
`up.sh` stages copied Substrate manifests in a temporary checkout view and installs through
|
||||||
|
`hack/install-ate-kind.sh`. The local `ops/ax/substrate-kind-lean/` overlay removes the collector,
|
||||||
|
Prometheus, Jaeger and their supporting resources before installation. This preserves the
|
||||||
|
installer's secret, CRD and image setup. The upstream checkout is unchanged. The shared
|
||||||
|
`ate-otel-config` disables trace, metric and log export and the OTEL SDK; it removes the old
|
||||||
|
collector endpoint and metric timing keys. On an existing cluster, `up.sh` scales the three
|
||||||
|
observability deployments to zero, replaces the ConfigMap data and rolls its consumers when
|
||||||
|
the configuration changes. Opting in restores the upstream ConfigMap and observability manifests.
|
||||||
|
|
||||||
## What runs where today (implemented)
|
## What runs where today (implemented)
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
@@ -357,7 +372,7 @@ This section answers the maintainer's questions from the code and the upstream c
|
|||||||
|
|
||||||
The decisions of 2026-10-02 set the target: **two profiles of one code base**.
|
The decisions of 2026-10-02 set the target: **two profiles of one code base**.
|
||||||
|
|
||||||
- `kind` is this laptop. It stays a first-class local execution mode. Every change is proven here first.
|
- `kind` is this server VM. It stays a first-class local execution mode. Every change is proven here first.
|
||||||
- `k3s` is production. (k3s is a small Kubernetes distribution.)
|
- `k3s` is production. (k3s is a small Kubernetes distribution.)
|
||||||
|
|
||||||
Everything marked **planned** is not built. The rest is what runs on this host today.
|
Everything marked **planned** is not built. The rest is what runs on this host today.
|
||||||
@@ -368,7 +383,7 @@ flowchart LR
|
|||||||
hook["ousast pre-push: local, 30 s deadline"]
|
hook["ousast pre-push: local, 30 s deadline"]
|
||||||
cli["ousast plane run, remote mode (planned)"]
|
cli["ousast plane run, remote mode (planned)"]
|
||||||
end
|
end
|
||||||
subgraph k3s ["k3s cluster (the kind profile is the same picture on one laptop)"]
|
subgraph k3s ["k3s cluster (the kind profile is the same picture on one server VM)"]
|
||||||
subgraph axns ["Namespace ax-system"]
|
subgraph axns ["Namespace ax-system"]
|
||||||
axs["ax-server Deployment + Redis"]
|
axs["ax-server Deployment + Redis"]
|
||||||
end
|
end
|
||||||
@@ -403,7 +418,7 @@ This picture has no inbound path:
|
|||||||
- the tasks push their artifacts to the store through the egress gateway;
|
- the tasks push their artifacts to the store through the egress gateway;
|
||||||
- the CLI reads the store.
|
- the CLI reads the store.
|
||||||
|
|
||||||
Today's EndpointSlice back to a laptop (`ops/ax/receiver-service.yaml.tmpl`) belongs to the kind
|
Today's EndpointSlice back to the server VM (`ops/ax/receiver-service.yaml.tmpl`) belongs to the kind
|
||||||
profile only. It does not appear in production.
|
profile only. It does not appear in production.
|
||||||
|
|
||||||
### ax, Kubernetes and Agent Substrate in three sentences
|
### ax, Kubernetes and Agent Substrate in three sentences
|
||||||
|
|||||||
+1
-1
@@ -159,7 +159,7 @@ flowchart LR
|
|||||||
ax is a Kubernetes application over Agent Substrate. Every Task runs the one runner image as an actor. Each actor
|
ax is a Kubernetes application over Agent Substrate. Every Task runs the one runner image as an actor. Each actor
|
||||||
occupies a whole worker. This code base targets two profiles:
|
occupies a whole worker. This code base targets two profiles:
|
||||||
|
|
||||||
- `kind` on a laptop (today).
|
- `kind` on a server VM (today).
|
||||||
- `k3s` in production (planned). There the same CLI submits Runs remotely, and the tasks deliver to the S3 store.
|
- `k3s` in production (planned). There the same CLI submits Runs remotely, and the tasks deliver to the S3 store.
|
||||||
|
|
||||||
[Production topology: ax on a Kubernetes cluster](deployment.md#production-topology-ax-on-a-kubernetes-cluster)
|
[Production topology: ax on a Kubernetes cluster](deployment.md#production-topology-ax-on-a-kubernetes-cluster)
|
||||||
|
|||||||
+16
-1
@@ -17,6 +17,21 @@ in `~/.cache/ousast/ax-src/`. The cluster's context, registry and pool sizes com
|
|||||||
(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them.
|
(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them.
|
||||||
`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`).
|
`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`).
|
||||||
|
|
||||||
|
The kind profile runs on a 7.7 GB server VM. Observability is off by default
|
||||||
|
(`kind_observability = false` in `ops/k8s/profiles/kind.toml`). This saves the measured
|
||||||
|
930 MiB used by `otel-system`, about 40% of the cluster's 2.3 GB. The smoke Run still
|
||||||
|
completes with those deployments stopped. To enable the stack, run
|
||||||
|
`OUSAST_KIND_OBSERVABILITY=1 ops/ax/up.sh` or set `kind_observability = true` in the profile.
|
||||||
|
|
||||||
|
`up.sh` stages copied Substrate manifests in a temporary checkout view and installs through
|
||||||
|
`hack/install-ate-kind.sh`. The local `ops/ax/substrate-kind-lean/` overlay removes the collector,
|
||||||
|
Prometheus, Jaeger and their supporting resources before installation. This preserves the
|
||||||
|
installer's secret, CRD and image setup. The upstream checkout is unchanged. The shared
|
||||||
|
`ate-otel-config` disables trace, metric and log export and the OTEL SDK; it removes the old
|
||||||
|
collector endpoint and metric timing keys. On an existing cluster, `up.sh` scales the three
|
||||||
|
observability deployments to zero, replaces the ConfigMap data and rolls its consumers when
|
||||||
|
the configuration changes. Opting in restores the upstream ConfigMap and observability manifests.
|
||||||
|
|
||||||
## How a Run executes
|
## How a Run executes
|
||||||
|
|
||||||
A Run (`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG of steps; each step names an ax Task
|
A Run (`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG of steps; each step names an ax Task
|
||||||
@@ -191,7 +206,7 @@ configure the bucket.
|
|||||||
No deployment outside this host's kind cluster has been made. The step-by-step guidance, with what is implemented
|
No deployment outside this host's kind cluster has been made. The step-by-step guidance, with what is implemented
|
||||||
and what is planned, is `docs/deployment.md` (the site page "Deployment"). The Model and Workspace manifests under
|
and what is planned, is `docs/deployment.md` (the site page "Deployment"). The Model and Workspace manifests under
|
||||||
`plane/` move as they are and the Task templates need their image reference re-pinned to your registry; these parts
|
`plane/` move as they are and the Task templates need their image reference re-pinned to your registry; these parts
|
||||||
assume this laptop and must change first (checked 2026-10-02):
|
assume this server VM and must change first (checked 2026-10-02):
|
||||||
|
|
||||||
| Assumption | Where | Needed in a real cluster |
|
| Assumption | Where | Needed in a real cluster |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Stage an installer checkout with lean kind manifests; never modify the source checkout.
|
||||||
|
|
||||||
|
Usage: python ops/ax/prepare-substrate.py SOURCE EMPTY_DESTINATION
|
||||||
|
The destination lives only for an install/render. Source/build inputs are symlinked;
|
||||||
|
manifests are copied because the installer uses fixed paths (including the OTEL ConfigMap).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
|
||||||
|
def prepare(source: Path, destination: Path) -> None:
|
||||||
|
source = source.resolve()
|
||||||
|
destination.mkdir(parents=True, exist_ok=True)
|
||||||
|
if any(destination.iterdir()):
|
||||||
|
raise ValueError(f"staging directory must be empty: {destination}")
|
||||||
|
for entry in source.iterdir():
|
||||||
|
if entry.name != "manifests":
|
||||||
|
(destination / entry.name).symlink_to(entry, target_is_directory=entry.is_dir())
|
||||||
|
shutil.copytree(source / "manifests", destination / "manifests")
|
||||||
|
install = destination / "manifests/ate-install"
|
||||||
|
shutil.copytree(install / "kind", install / "kind-upstream")
|
||||||
|
overlay = Path(__file__).resolve().parent / "substrate-kind-lean"
|
||||||
|
shutil.copytree(overlay, install / "kind", dirs_exist_ok=True)
|
||||||
|
# The installer applies this file directly before it renders the bundle.
|
||||||
|
# Keep the strategic-merge directive only in the overlay's patch file.
|
||||||
|
config = yaml.safe_load((overlay / "ate-otel-config.yaml").read_text())
|
||||||
|
config["data"].pop("$patch")
|
||||||
|
(install / "kind/disabled-otel-config.yaml").write_text(yaml.safe_dump(config))
|
||||||
|
(install / "kind/ate-otel-config.yaml").rename(install / "kind/otel-config-patch.yaml")
|
||||||
|
(install / "kind/disabled-otel-config.yaml").rename(install / "kind/ate-otel-config.yaml")
|
||||||
|
kustomization = install / "kind/kustomization.yaml"
|
||||||
|
kustomization.write_text(kustomization.read_text().replace("path: ate-otel-config.yaml", "path: otel-config-patch.yaml"))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
prepare(Path(sys.argv[1]), Path(sys.argv[2]))
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: ate-otel-config
|
||||||
|
namespace: ate-system
|
||||||
|
data:
|
||||||
|
$patch: replace
|
||||||
|
OTEL_TRACES_EXPORTER: none
|
||||||
|
OTEL_METRICS_EXPORTER: none
|
||||||
|
OTEL_LOGS_EXPORTER: none
|
||||||
|
OTEL_SDK_DISABLED: "true"
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
# prepare-substrate.py stages the upstream kind overlay beside this directory.
|
||||||
|
resources:
|
||||||
|
- ../kind-upstream
|
||||||
|
patches:
|
||||||
|
- target:
|
||||||
|
namespace: otel-system
|
||||||
|
patch: |-
|
||||||
|
$patch: delete
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: unused
|
||||||
|
- target:
|
||||||
|
kind: Namespace
|
||||||
|
name: otel-system
|
||||||
|
patch: |-
|
||||||
|
$patch: delete
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Namespace
|
||||||
|
metadata:
|
||||||
|
name: otel-system
|
||||||
|
- target:
|
||||||
|
kind: ClusterRole
|
||||||
|
name: ate-prometheus
|
||||||
|
patch: |-
|
||||||
|
$patch: delete
|
||||||
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
|
kind: ClusterRole
|
||||||
|
metadata:
|
||||||
|
name: ate-prometheus
|
||||||
|
- target:
|
||||||
|
kind: RoleBinding
|
||||||
|
name: ate-prometheus
|
||||||
|
namespace: ate-system
|
||||||
|
patch: |-
|
||||||
|
$patch: delete
|
||||||
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
|
kind: RoleBinding
|
||||||
|
metadata:
|
||||||
|
name: ate-prometheus
|
||||||
|
namespace: ate-system
|
||||||
|
- path: ate-otel-config.yaml
|
||||||
+61
-1
@@ -15,6 +15,7 @@ profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --p
|
|||||||
CTX="$(profile kube_context)" # kind names its context kind-<cluster>
|
CTX="$(profile kube_context)" # kind names its context kind-<cluster>
|
||||||
export KIND_CLUSTER_NAME="${CTX#kind-}"
|
export KIND_CLUSTER_NAME="${CTX#kind-}"
|
||||||
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}"
|
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}"
|
||||||
|
OBSERVABILITY="$(profile kind_observability)"
|
||||||
IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read
|
IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read
|
||||||
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
|
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
|
||||||
|
|
||||||
@@ -37,9 +38,68 @@ else
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
step "agent substrate (ate-system)"
|
step "agent substrate (ate-system)"
|
||||||
|
# The upstream installer has fixed manifest paths and also creates secrets/CRDs.
|
||||||
|
# Give it a temporary checkout view so it never installs the observability resources.
|
||||||
|
install_substrate() (
|
||||||
|
if [ "$OBSERVABILITY" = true ]; then
|
||||||
|
cd "$SRC/substrate"
|
||||||
|
else
|
||||||
|
# Symlinked build inputs must not make the staged view a different binary version.
|
||||||
|
export VERSION="${VERSION:-$(git -C "$SRC/substrate" describe --tags --always --dirty)}"
|
||||||
|
staged="$(mktemp -d)"
|
||||||
|
trap 'rm -rf "$staged"' EXIT
|
||||||
|
"$PY" "$REPO_ROOT/ops/ax/prepare-substrate.py" "$SRC/substrate" "$staged"
|
||||||
|
cd "$staged"
|
||||||
|
fi
|
||||||
|
KUBECTL_CONTEXT="$CTX" hack/install-ate-kind.sh --deploy-ate-system
|
||||||
|
)
|
||||||
if ready ate-system; then echo "substrate ready; kept"; else
|
if ready ate-system; then echo "substrate ready; kept"; else
|
||||||
(cd "$SRC/substrate" && hack/install-ate-kind.sh --deploy-ate-system)
|
install_substrate
|
||||||
fi
|
fi
|
||||||
|
# Reconcile the choice even when the existing control plane is already ready.
|
||||||
|
if [ "$OBSERVABILITY" = true ]; then
|
||||||
|
for manifest in otel-collector prometheus; do
|
||||||
|
kubectl --context "$CTX" apply -f "$SRC/substrate/manifests/ate-install/kind/$manifest.yaml"
|
||||||
|
done
|
||||||
|
otel_config="$SRC/substrate/manifests/ate-install/kind/ate-otel-config.yaml"
|
||||||
|
else
|
||||||
|
otel_config="$REPO_ROOT/ops/ax/substrate-kind-lean/ate-otel-config.yaml"
|
||||||
|
for deployment in opentelemetry-collector prometheus jaeger; do
|
||||||
|
existing="$(kubectl --context "$CTX" -n otel-system get deployment "$deployment" --ignore-not-found -o name)"
|
||||||
|
if [ -n "$existing" ]; then
|
||||||
|
kubectl --context "$CTX" -n otel-system scale "$existing" --replicas=0
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo "kind observability off: existing otel-system deployments scaled to 0; opt in with OUSAST_KIND_OBSERVABILITY=1 ops/ax/up.sh"
|
||||||
|
fi
|
||||||
|
# Replace data to remove stale exporter/endpoint keys in either direction. ConfigMap
|
||||||
|
# envFrom is read at pod start; a content annotation triggers only changed rollouts.
|
||||||
|
otel_data="$("$PY" - "$otel_config" <<'PYCONFIG'
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
import yaml
|
||||||
|
config = yaml.safe_load(open(sys.argv[1]))
|
||||||
|
config["data"].pop("$patch", None)
|
||||||
|
print(json.dumps(config["data"], sort_keys=True))
|
||||||
|
PYCONFIG
|
||||||
|
)"
|
||||||
|
kubectl --context "$CTX" -n ate-system patch configmap ate-otel-config --type=json \
|
||||||
|
-p "[{\"op\":\"replace\",\"path\":\"/data\",\"value\":$otel_data}]"
|
||||||
|
otel_hash="$(printf '%s' "$otel_data" | sha256sum | cut -d ' ' -f1)"
|
||||||
|
otel_workloads="$(kubectl --context "$CTX" -n ate-system get deployments,daemonsets -o json | "$PY" -c '
|
||||||
|
import json, sys
|
||||||
|
for item in json.load(sys.stdin)["items"]:
|
||||||
|
spec = item["spec"]["template"]["spec"]
|
||||||
|
containers = spec.get("containers", []) + spec.get("initContainers", [])
|
||||||
|
if any(env.get("configMapRef", {}).get("name") == "ate-otel-config"
|
||||||
|
for container in containers for env in container.get("envFrom", [])):
|
||||||
|
print(item["kind"].lower() + "/" + item["metadata"]["name"])
|
||||||
|
')"
|
||||||
|
for workload in $otel_workloads; do
|
||||||
|
kubectl --context "$CTX" -n ate-system patch "$workload" --type=merge \
|
||||||
|
-p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"ousast.io/otel-config\":\"$otel_hash\"}}}}}"
|
||||||
|
kubectl --context "$CTX" -n ate-system rollout status "$workload" --timeout=300s
|
||||||
|
done
|
||||||
kubectl --context "$CTX" -n ate-system get pods
|
kubectl --context "$CTX" -n ate-system get pods
|
||||||
|
|
||||||
step "egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built)"
|
step "egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built)"
|
||||||
|
|||||||
@@ -1,3 +1,3 @@
|
|||||||
{
|
{
|
||||||
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
"runner": "localhost:5001/ousast-runner@sha256:784bc6afc869c679c4966d04d3f9653a8d379726b899809590cc83327f4038e3"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
# The kind profile: the plane on this laptop's single-node kind cluster, which ops/ax/up.sh creates and which is
|
# The kind profile: the plane on this server VM's single-node kind cluster, which ops/ax/up.sh creates and which is
|
||||||
# the development default (plane-on-kubernetes design section 1). Values only; a credential never goes here
|
# the development default (plane-on-kubernetes design section 1). Values only; a credential never goes here
|
||||||
# (keys stay in the environment or .env). Every field is overridable by OUSAST_<FIELD> in the environment, a pool
|
# (keys stay in the environment or .env). Every field is overridable by OUSAST_<FIELD> in the environment, a pool
|
||||||
# by OUSAST_POOL_<NAME>_<FIELD>. This file may name the kind cluster; the code must not (tests/test_plane_literals.py).
|
# by OUSAST_POOL_<NAME>_<FIELD>. This file may name the kind cluster; the code must not (tests/test_plane_literals.py).
|
||||||
|
|
||||||
|
kind_observability = false # opt in with OUSAST_KIND_OBSERVABILITY=1
|
||||||
exec = "local" # the reconciler runs on this host against the cluster below
|
exec = "local" # the reconciler runs on this host against the cluster below
|
||||||
kube_context = "kind-ousast" # kind's context for KIND_CLUSTER_NAME=ousast
|
kube_context = "kind-ousast" # kind's context for KIND_CLUSTER_NAME=ousast
|
||||||
registry = "localhost:5001" # the kind-local registry up.sh creates (Substrate rewrites it for the node)
|
registry = "localhost:5001" # the kind-local registry up.sh creates (Substrate rewrites it for the node)
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""The plane's configuration surface: one ``PlaneProfile`` per cluster (plane-on-kubernetes design section 1).
|
"""The plane's configuration surface: one ``PlaneProfile`` per cluster (plane-on-kubernetes design section 1).
|
||||||
|
|
||||||
A profile is a TOML file under ``ops/k8s/profiles/`` (``kind.toml`` for the laptop's kind cluster, ``k3s.toml`` for
|
A profile is a TOML file under ``ops/k8s/profiles/`` (``kind.toml`` for the server VM's kind cluster, ``k3s.toml`` for
|
||||||
the production cluster) holding every value that differs between clusters: the execution mode, the kube context,
|
the production cluster) holding every value that differs between clusters: the execution mode, the kube context,
|
||||||
the registry and the digest-pinned images file, the router and ax-server addresses (empty: tunnel through the
|
the registry and the digest-pinned images file, the router and ax-server addresses (empty: tunnel through the
|
||||||
Kubernetes API), the memory store, the atespace, ax's snapshot bucket, the worker pools and the image pull secret.
|
Kubernetes API), the memory store, the atespace, ax's snapshot bucket, the worker pools and the image pull secret.
|
||||||
@@ -59,6 +59,7 @@ class PlaneProfile:
|
|||||||
name: str
|
name: str
|
||||||
path: Path
|
path: Path
|
||||||
exec: str = "local"
|
exec: str = "local"
|
||||||
|
kind_observability: bool = False
|
||||||
kube_context: str = ""
|
kube_context: str = ""
|
||||||
registry: str = ""
|
registry: str = ""
|
||||||
images: Path = Path("images.json")
|
images: Path = Path("images.json")
|
||||||
@@ -135,6 +136,10 @@ def _reject_secret(where: str, key: str, value: object, declared: bool = False)
|
|||||||
|
|
||||||
|
|
||||||
def _coerce(where: str, key: str, value: object, kind: str) -> Any:
|
def _coerce(where: str, key: str, value: object, kind: str) -> Any:
|
||||||
|
if kind == "bool":
|
||||||
|
if not isinstance(value, bool):
|
||||||
|
raise ProfileError(f"{where}: {key} must be a boolean, got {value!r}")
|
||||||
|
return value
|
||||||
if kind == "int":
|
if kind == "int":
|
||||||
if isinstance(value, bool) or not isinstance(value, int):
|
if isinstance(value, bool) or not isinstance(value, int):
|
||||||
raise ProfileError(f"{where}: {key} must be an integer, got {value!r}")
|
raise ProfileError(f"{where}: {key} must be an integer, got {value!r}")
|
||||||
@@ -172,6 +177,10 @@ def _env_overrides(environ: Mapping[str, str], where: str, values: dict[str, Any
|
|||||||
raw = environ.get(_ENV_PREFIX + name.upper())
|
raw = environ.get(_ENV_PREFIX + name.upper())
|
||||||
if raw is not None and raw != "":
|
if raw is not None and raw != "":
|
||||||
value: Any = raw
|
value: Any = raw
|
||||||
|
if kind == "bool":
|
||||||
|
if raw.lower() not in ("true", "false", "1", "0"):
|
||||||
|
raise ProfileError(f"{where}: {_ENV_PREFIX}{name.upper()} must be true/false or 1/0, got {raw!r}")
|
||||||
|
value = raw.lower() in ("true", "1")
|
||||||
if kind == "int":
|
if kind == "int":
|
||||||
try:
|
try:
|
||||||
value = int(raw)
|
value = int(raw)
|
||||||
@@ -241,7 +250,8 @@ def _lookup(profile: PlaneProfile, dotted: str) -> str:
|
|||||||
return str(getattr(profile.pools[pool_name], key))
|
return str(getattr(profile.pools[pool_name], key))
|
||||||
if head not in _SCALARS or rest:
|
if head not in _SCALARS or rest:
|
||||||
raise ProfileError(f"{profile.path}: no such field {dotted!r}")
|
raise ProfileError(f"{profile.path}: no such field {dotted!r}")
|
||||||
return str(getattr(profile, head))
|
value = getattr(profile, head)
|
||||||
|
return str(value).lower() if isinstance(value, bool) else str(value)
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
|||||||
@@ -0,0 +1,130 @@
|
|||||||
|
"""Offline overlay contract and installer staging; no Kubernetes executable needed."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import runpy
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
OVERLAY = ROOT / "ops/ax/substrate-kind-lean"
|
||||||
|
DISABLED = {
|
||||||
|
"OTEL_TRACES_EXPORTER": "none",
|
||||||
|
"OTEL_METRICS_EXPORTER": "none",
|
||||||
|
"OTEL_LOGS_EXPORTER": "none",
|
||||||
|
"OTEL_SDK_DISABLED": "true",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_ax_lean_overlay_removes_observability_and_replaces_export_config() -> None:
|
||||||
|
overlay = yaml.safe_load((OVERLAY / "kustomization.yaml").read_text())
|
||||||
|
assert overlay["resources"] == ["../kind-upstream"]
|
||||||
|
deletes = [p for p in overlay["patches"] if "target" in p]
|
||||||
|
assert [p["target"] for p in deletes] == [
|
||||||
|
{"namespace": "otel-system"},
|
||||||
|
{"kind": "Namespace", "name": "otel-system"},
|
||||||
|
{"kind": "ClusterRole", "name": "ate-prometheus"},
|
||||||
|
{"kind": "RoleBinding", "name": "ate-prometheus", "namespace": "ate-system"},
|
||||||
|
]
|
||||||
|
assert all(yaml.safe_load(p["patch"])["$patch"] == "delete" for p in deletes)
|
||||||
|
config = yaml.safe_load((OVERLAY / overlay["patches"][-1]["path"]).read_text())
|
||||||
|
assert config["metadata"] == {"name": "ate-otel-config", "namespace": "ate-system"}
|
||||||
|
# Whole-data replacement drops the upstream endpoint, interval and timeout,
|
||||||
|
# including any future keys that would otherwise survive a merge.
|
||||||
|
assert config["data"] == {"$patch": "replace", **DISABLED}
|
||||||
|
|
||||||
|
|
||||||
|
def test_ax_staging_preserves_upstream_and_installer_prerequisites(tmp_path: Path) -> None:
|
||||||
|
prepare = runpy.run_path(str(ROOT / "ops/ax/prepare-substrate.py"))["prepare"]
|
||||||
|
source = tmp_path / "source"
|
||||||
|
kind = source / "manifests/ate-install/kind"
|
||||||
|
kind.mkdir(parents=True)
|
||||||
|
original = "resources: [otel-collector.yaml, prometheus.yaml, ate-otel-config.yaml]\n"
|
||||||
|
(kind / "kustomization.yaml").write_text(original)
|
||||||
|
(kind / "ate-otel-config.yaml").write_text("upstream config")
|
||||||
|
(kind / "postgres").mkdir()
|
||||||
|
(kind / "postgres/kustomization.yaml").write_text("resources: [../../postgres]\n")
|
||||||
|
(source / "go.mod").write_text("module test\n")
|
||||||
|
(source / "hack").mkdir()
|
||||||
|
(source / "hack/install-ate-kind.sh").write_text("upstream installer")
|
||||||
|
stage = tmp_path / "stage"
|
||||||
|
prepare(source, stage)
|
||||||
|
install = stage / "manifests/ate-install"
|
||||||
|
assert (install / "kind-upstream/kustomization.yaml").read_text() == original
|
||||||
|
assert (install / "kind/postgres/kustomization.yaml").read_text() == "resources: [../../postgres]\n"
|
||||||
|
assert (kind / "kustomization.yaml").read_text() == original
|
||||||
|
assert (kind / "ate-otel-config.yaml").read_text() == "upstream config"
|
||||||
|
assert (stage / "hack/install-ate-kind.sh").read_text() == "upstream installer"
|
||||||
|
assert (stage / "go.mod").is_symlink()
|
||||||
|
config = yaml.safe_load((install / "kind/ate-otel-config.yaml").read_text())
|
||||||
|
assert config["data"] == DISABLED # direct installer apply must not contain $patch
|
||||||
|
overlay = yaml.safe_load((install / "kind/kustomization.yaml").read_text())
|
||||||
|
patch = yaml.safe_load((install / "kind" / overlay["patches"][-1]["path"]).read_text())
|
||||||
|
assert patch["data"] == {"$patch": "replace", **DISABLED}
|
||||||
|
with pytest.raises(ValueError, match="must be empty"):
|
||||||
|
prepare(source, stage)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("enabled", [False, True])
|
||||||
|
@pytest.mark.parametrize("existing", [False, True])
|
||||||
|
def test_ax_reconciles_observability_on_ready_clusters(tmp_path: Path, enabled: bool, existing: bool) -> None:
|
||||||
|
"""Execute the reconciliation shell with a function double, never real kubectl."""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
script = (ROOT / "ops/ax/up.sh").read_text()
|
||||||
|
section = script.split("# Reconcile the choice", 1)[1].split('kubectl --context "$CTX" -n ate-system get pods', 1)[0]
|
||||||
|
section = "# Reconcile the choice" + section
|
||||||
|
source = tmp_path / "substrate/manifests/ate-install/kind"
|
||||||
|
source.mkdir(parents=True)
|
||||||
|
upstream_data = {"OTEL_EXPORTER_OTLP_ENDPOINT": "http://collector:4317", "OTEL_LOGS_EXPORTER": "otlp"}
|
||||||
|
(source / "ate-otel-config.yaml").write_text(yaml.safe_dump({"data": upstream_data}))
|
||||||
|
log = tmp_path / "calls"
|
||||||
|
consumer = {
|
||||||
|
"kind": "Deployment",
|
||||||
|
"metadata": {"name": "ate-controller"},
|
||||||
|
"spec": {"template": {"spec": {"containers": [{"envFrom": [{"configMapRef": {"name": "ate-otel-config"}}]}]}}},
|
||||||
|
}
|
||||||
|
nonconsumer = {"kind": "Deployment", "metadata": {"name": "unrelated"}, "spec": {"template": {"spec": {"containers": []}}}}
|
||||||
|
mock = """
|
||||||
|
set -euo pipefail
|
||||||
|
kubectl() {
|
||||||
|
printf '%s\\t' "$@" >> "$CALL_LOG"
|
||||||
|
printf '\\n' >> "$CALL_LOG"
|
||||||
|
case " $* " in
|
||||||
|
*" get deployment "*)
|
||||||
|
if [ "$EXISTING" = 1 ]; then printf 'deployment.apps/%s\\n' "$7"; fi ;;
|
||||||
|
*" get deployments,daemonsets "*) printf '%s' "$WORKLOADS" ;;
|
||||||
|
esac
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
env = {
|
||||||
|
**os.environ,
|
||||||
|
"OBSERVABILITY": str(enabled).lower(),
|
||||||
|
"EXISTING": str(int(existing)),
|
||||||
|
"PY": sys.executable,
|
||||||
|
"CTX": "test-context",
|
||||||
|
"SRC": str(tmp_path),
|
||||||
|
"REPO_ROOT": str(ROOT),
|
||||||
|
"CALL_LOG": str(log),
|
||||||
|
"WORKLOADS": json.dumps({"items": [consumer, nonconsumer]}),
|
||||||
|
}
|
||||||
|
result = subprocess.run(["bash", "-c", mock + section], env=env, text=True, capture_output=True)
|
||||||
|
assert result.returncode == 0, result.stderr
|
||||||
|
calls = [line.rstrip("\t").split("\t") for line in log.read_text().splitlines()]
|
||||||
|
scales = [call for call in calls if "scale" in call]
|
||||||
|
assert len(scales) == (3 if existing and not enabled else 0)
|
||||||
|
assert all(call[-1] == "--replicas=0" for call in scales)
|
||||||
|
applies = [call for call in calls if "apply" in call]
|
||||||
|
assert len(applies) == (2 if enabled else 0)
|
||||||
|
config_patch = next(call[-1] for call in calls if "configmap" in call)
|
||||||
|
assert json.loads(config_patch) == [{"op": "replace", "path": "/data", "value": upstream_data if enabled else DISABLED}]
|
||||||
|
assert not any("deployment/unrelated" in call for call in calls)
|
||||||
|
assert any("deployment/ate-controller" in call for call in calls)
|
||||||
|
if not enabled:
|
||||||
|
assert "OUSAST_KIND_OBSERVABILITY=1" in result.stdout
|
||||||
@@ -156,3 +156,27 @@ def test_profile_is_frozen_and_typed() -> None:
|
|||||||
assert isinstance(loaded, PlaneProfile)
|
assert isinstance(loaded, PlaneProfile)
|
||||||
with pytest.raises(AttributeError):
|
with pytest.raises(AttributeError):
|
||||||
loaded.exec = "remote" # type: ignore[misc]
|
loaded.exec = "remote" # type: ignore[misc]
|
||||||
|
|
||||||
|
|
||||||
|
def test_kind_observability_defaults_off() -> None:
|
||||||
|
assert load_profile("kind", environ={}).kind_observability is False
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(("raw", "expected"), [("1", True), ("true", True), ("0", False), ("FALSE", False)])
|
||||||
|
def test_kind_observability_environment_override(tmp_path: Path, raw: str, expected: bool) -> None:
|
||||||
|
path = write_profile(tmp_path, MINIMAL + f"kind_observability = {str(not expected).lower()}\n")
|
||||||
|
assert load_profile(str(path), environ={"OUSAST_KIND_OBSERVABILITY": raw}).kind_observability is expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_kind_observability_validation_and_print(
|
||||||
|
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
||||||
|
) -> None:
|
||||||
|
path = write_profile(tmp_path, MINIMAL + 'kind_observability = "false"\n')
|
||||||
|
with pytest.raises(ProfileError, match="kind_observability must be a boolean"):
|
||||||
|
load_profile(str(path), environ={})
|
||||||
|
path = write_profile(tmp_path)
|
||||||
|
with pytest.raises(ProfileError, match="OUSAST_KIND_OBSERVABILITY must be"):
|
||||||
|
load_profile(str(path), environ={"OUSAST_KIND_OBSERVABILITY": "maybe"})
|
||||||
|
monkeypatch.setenv("OUSAST_KIND_OBSERVABILITY", "1")
|
||||||
|
assert profile_module.main(["--profile", str(path), "--print", "kind_observability"]) == 0
|
||||||
|
assert capsys.readouterr().out == "true\n"
|
||||||
|
|||||||
Reference in New Issue
Block a user