mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 04:29:27 +00:00
feat(plane): per-task egress policy; receiver by name through the gateway
Agent Substrate's egress gateway denies an actor without an EgressPolicy, so no task could deliver its artifacts or reach a model. egress.py grants each task exactly: http/80 to ousast-receiver.ax-system.svc.cluster.local, and TLS passthrough/443 to the bound Workspaces' https Git hosts and the bound Model's openultrasast.io/egress-hosts (api.deepseek.com for deepseek-flash). No IPs, no wildcards, no https (MITM) rules. The reconciler writes the policy with kubectl-ate between ax apply and resume, reads it back, waits OUSAST_EGRESS_SETTLE (11 s, the gateway's cached deny), and after ax delete task confirms the policy went with its actor (kubectl-ate has no delete verb; the store cascades). The receiver is a selector-less Service (ops/ax/receiver-service.yaml.tmpl, applied by up.sh together with the no-build agentgateway egress gateway) on port 18090 of the host. Measured live: the gateway checks the policy on Host but connects to the address the actor dialled, so the runner dials the Service's ClusterIP (OUSAST_ARTIFACT_DIAL) with the receiver's name as Host. The runner now labels its delivery with the Run's task name (OUSAST_TASK), which is what the receiver accepts; the ax Task name was refused with 403. The Receiver moved to egress.py to keep reconciler.py at 460 lines. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
0744a157bf
commit
137c6c8ed5
@@ -110,8 +110,8 @@ new image can fail with `DeadlineExceeded` while Agent Substrate builds its gold
|
||||
`OUSAST_RESUME_TIMEOUT` (900 s). ax has no Completed phase and never reports that a command exited: completion is
|
||||
the artifact delivery. `Failed` after a resume went through, or the task disappearing, before delivery fails the
|
||||
task with ax's condition message; no delivery within `OUSAST_TASK_TIMEOUT` (7200 s) fails it too; `ax delete task`
|
||||
follows either way. Artifacts: the reconciler serves a small HTTP receiver on the host (reachable from kind pods
|
||||
via the docker bridge); the runner posts the output directory there after its command exits. Tasks sharing a
|
||||
follows either way. Artifacts: the reconciler serves a small HTTP receiver on the host (reachable from actors
|
||||
by name through the egress gateway, see Egress below); the runner posts the output directory there after its command exits. Tasks sharing a
|
||||
`serialize` label never overlap; everything else may run in parallel up to `--workers`. A task's outcome is
|
||||
read from its delivered `summary.json`: `done` only when the task says every unit finished; `unfinished` on a
|
||||
budget stop; `failed` on an account or authentication error, a crash, or a missing delivery. `status` prints
|
||||
@@ -131,6 +131,30 @@ bound Model's `spec.secretKey.key` names the variable (`DEEPSEEK_API_KEY` for `d
|
||||
reconciler's own environment (`ousast` loads `.env` at startup); a Model-bound task whose variable is unset fails
|
||||
before `ax apply`, and a task without a Model gets none. The value is never rendered, logged or written.
|
||||
|
||||
#### Egress (`egress.py`)
|
||||
|
||||
Every actor connection except DNS goes through Agent Substrate's egress gateway (`atenet-egress`), which denies
|
||||
an actor without an `EgressPolicy`. Three pieces make the plane work under that rule:
|
||||
|
||||
- **Receiver by name.** `ops/ax/receiver-service.yaml.tmpl` (applied by `up.sh`) is a selector-less Service
|
||||
`ousast-receiver` in `ax-system`, port 80, with an EndpointSlice on the kind gateway at `OUSAST_ARTIFACT_PORT`
|
||||
(fixed, 18090), where the reconciler binds its receiver. Tasks get
|
||||
`OUSAST_ARTIFACT_URL=http://ousast-receiver.ax-system.svc.cluster.local/` plus `OUSAST_ARTIFACT_DIAL=<the
|
||||
Service's ClusterIP>:80` (looked up with kubectl unless set): the actor cannot resolve cluster names, so the
|
||||
runner connects to the dial address and sends the URL's host as `Host`. The gateway decides the policy on
|
||||
`Host` but connects to the address the actor dialled (measured: dialling the kind gateway on port 80 was allowed
|
||||
and then refused upstream), so the dial address is the ClusterIP, which kube-proxy maps to the host.
|
||||
`OUSAST_ARTIFACT_HOST` set keeps the old direct URL (tests).
|
||||
- **Per-task policy.** `policy_for` grants `http` on port 80 to the receiver name, and `tls_passthrough` on 443
|
||||
to the Git hosts of the bound Workspaces' `https` repos and to the bound Model's
|
||||
`openultrasast.io/egress-hosts` (`api.deepseek.com` for `deepseek-flash`, validated as a plain DNS name).
|
||||
Nothing else: no IP address or wildcard, and passthrough only -- an `https` rule intercepts TLS and the actor
|
||||
would need the gateway CA, which ax templates do not mount.
|
||||
- **Order.** `ax apply` creates the actor; the reconciler writes the policy with `kubectl ate create
|
||||
egress-policy` (update when one exists), reads it back, waits `OUSAST_EGRESS_SETTLE` (11 s: the gateway caches a
|
||||
missing policy as deny for 10 s), then resumes. kubectl-ate has no delete verb; Substrate's store deletes the
|
||||
policy with its actor, so after `ax delete task` the reconciler confirms it is gone.
|
||||
|
||||
### Budget (`budget.py`)
|
||||
|
||||
Wraps a chat client: sums the provider's usage fields per call (`prompt_tokens`, `prompt_cache_hit_tokens`,
|
||||
|
||||
@@ -11,6 +11,12 @@ mypy are clean, and the commit is gated on pytest's own exit code. Script-level
|
||||
and wait for it to finish; `ops/ax/down.sh` tears it down. Record the measured idle footprint (RSS, CPU)
|
||||
in `ops/ax/README.md` next to the 7 GB host constraint.
|
||||
- `ousast plane doctor` (in `reconciler.py` or a sibling `doctor.py`): the four checks of Requirement 3.5.
|
||||
- Egress: `up.sh` applies Substrate's agentgateway egress gateway (`atenet-egress`, prebuilt image, rendered
|
||||
with `kubectl kustomize`) when absent, and the receiver Service `ousast-receiver.ax-system`
|
||||
(`receiver-service.yaml.tmpl`, endpoint the kind gateway at port 18090). The reconciler writes each task's
|
||||
egress policy (`egress.py`: receiver over HTTP, Git and Model hosts as TLS passthrough) between `ax apply` and
|
||||
resume; the runner delivers to `OUSAST_ARTIFACT_DIAL` (the Service's ClusterIP, port 80) with the receiver's name as
|
||||
`Host`.
|
||||
- Stop rule: if the cluster does not come up within one working session, report the exact failure and stop;
|
||||
the decision to work around it is the maintainer's.
|
||||
- _Requirements: 3.5, 3.6_
|
||||
|
||||
Reference in New Issue
Block a user