spec(plane-on-kubernetes): brief and requirements for review

Maintainer: the whole system moves to a Kubernetes cluster soon. Seven host-bound
assumptions become cluster resources or configuration; the engine becomes a
plane task on a larger pool; a delta Run command serves the pre-push use; every
step is proven on kind first.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 19:36:59 +02:00
co-authored by Claude Fable 5.1
parent e5c338070d
commit 6187214be3
3 changed files with 160 additions and 0 deletions
+33
View File
@@ -0,0 +1,33 @@
# Brief: the plane on a Kubernetes cluster
Maintainer, 2026-10-02: "the plan is to move the whole to a kube system soon. what is in the way?" and earlier
"later i will deploy this in a separate kube env."
## Where we are
The plane already runs on google/ax over Agent Substrate, on a single-node kind cluster on the maintainer's
laptop (`ops/ax/`). Its manifests are ax's own kinds, every task is one digest-pinned runner image, the memory
store is external (S3-compatible RustFS), and provider credentials travel only in the start request. What still
assumes the laptop is small and listed in `docs/deployment.md` and `ops/ax/README.md`:
1. the reconciler is a host CLI process (`ousast plane run`);
2. the artifact receiver is a thread of that process, reached through a Service + EndpointSlice to the kind
gateway; the router is reached through a kubectl port-forward;
3. the kube context `kind-ousast` and the registry `localhost:5001` are hard-coded in `doctor.py`, `egress.py`,
`router.py`;
4. images come from the kind registry;
5. ax's deploy manifest points `AX_SNAPSHOTS_BUCKET` at the ax authors' bucket;
6. the Joern engine runs as a host Docker container (2.8 GB image, 3 GB memory), not as an ax task;
7. the worker pool is two 1 CPU / 1.5 GiB workers with no autoscaling.
## Direction
Make the plane deployable to any Kubernetes cluster that runs Agent Substrate and ax, prove every change on the
kind cluster first (it already behaves like the target), and leave nothing that only works on this host. The
production use (a pre-push safety net fed by CI) needs, in addition, a delta Run command and Workspaces from a
URL and commit; those are in scope as the last increment because they define what the cluster is for.
## Not in scope
Provisioning the maintainer's cluster, Substrate's and ax's own deployment procedures (followed, not changed),
and any detection change.
@@ -0,0 +1,114 @@
# Requirements Document
## Introduction
The plane moves from this laptop's kind cluster to a Kubernetes cluster of the maintainer's choosing without
changing what runs: the same ax manifests, runner image, memory store and credential path. Every host-bound
piece becomes a cluster resource or a configuration value, each change is proven on kind first, and the
production entry point (a delta Run for a push) is added. Context: `brief.md`, `docs/deployment.md`.
## Boundary Context
- **In scope**: the reconciler as an in-cluster workload; receiver and router addressed as Services; configurable
cluster context, registry and image pins; our own snapshot and memory buckets; the Joern engine as a plane task
on a larger worker pool; worker pool sizing and autoscaling; a delta Run command and Workspaces from URL +
commit; a deployment runbook with a proof on kind and a checklist for the target cluster.
- **Out of scope**: provisioning or operating the target cluster; modifying Agent Substrate or ax; any change to
detection, scoring, populations or the decision engine; replacing the `ax` CLI with a bespoke client beyond what
in-cluster addressing needs.
- **Adjacent expectations**: `ai-service-plane` requirements (one Model + budget per task, attribution,
credentials only in the start request, egress policies, verify-at-startup memory store) keep holding;
`harnessx-removal` Req 6 (memory on the S3 store) keeps holding; nothing is deleted before its replacement
passes its test (the `harnessx-removal` sequencing rule).
## Requirements
### Requirement 1: Nothing assumes this host
**User Story:** As the maintainer, I want every host-bound assumption replaced by a cluster resource or a
configuration value, so the plane runs the same on kind and on the target cluster.
#### Acceptance Criteria
1. The kube context, the registry, the ax-server address, the router address, the receiver address and the
snapshot and memory buckets are configuration (environment or one config file), with the kind values as the
documented defaults for development; no module names `kind-ousast`, `localhost:5001` or `172.19.0.1`.
2. `ousast plane doctor` checks the configured cluster, not kind by name, and reports each configured address.
3. A test enumerates the source tree for host-only literals and fails on any.
### Requirement 2: The reconciler runs in the cluster
**User Story:** As the maintainer, I want runs submitted to a reconciler that lives in the cluster, so that a
push from CI, not a laptop session, drives the plane.
#### Acceptance Criteria
1. The reconciler runs as a Kubernetes workload (a Deployment serving a submit API, or a Job per Run; the design
decides) from the runner image, with a ServiceAccount whose RBAC allows what it does and nothing more.
2. It talks to ax-server and `atenet-router` through their Services, not port-forwards; the `ax` CLI, if kept, is
in the image.
3. Provider and store credentials come from Kubernetes Secrets into the reconciler's environment, never into a
manifest, an ActorTemplate or a log; the start-request path is unchanged.
4. `ousast plane run` from a laptop still works against the kind cluster for development (it is the same code
with the development defaults).
### Requirement 3: Receiver and artifacts without a host
**User Story:** As the maintainer, I want task artifacts to land in the cluster or the store, not on a laptop.
#### Acceptance Criteria
1. The receiver is a Service of the in-cluster reconciler, or tasks deliver through presigned URLs to the S3
store (the design chooses one and states why); the kind gateway EndpointSlice is removed.
2. Egress policies grant each task exactly the hosts it needs (receiver or store, git hosts, the bound Model's
host) and nothing else; a test renders the policy for every committed Task.
### Requirement 4: The engine is a plane task
**User Story:** As the maintainer, I want Joern to run where the other tasks run, on a worker sized for it.
#### Acceptance Criteria
1. A second WorkerPool sized for the engine (memory and CPU from a measured run) exists alongside the default
pool; an `engine` task module wraps today's host engine run and emits the same `alerts.jsonl` rows and
coverage as `alerts-engine`.
2. The engine image is digest-pinned in the registry like the runner image; its size and pull time are recorded.
3. `alerts-engine` on the host remains available for development and is marked as such.
### Requirement 5: Sizing and autoscaling
**User Story:** As the maintainer, I want the pool to follow the load of pushes without manual resizing.
#### Acceptance Criteria
1. Worker pool replicas, limits and the autoscaling rule (Substrate's assigned-worker signal) are manifests under
`ops/k8s/` with the kind values as a small profile and a documented production profile.
2. The runbook records the measured footprint per task type and the queueing consequence of one actor per
worker.
### Requirement 6: A delta Run for a push
**User Story:** As a user with a repository the project has never seen, I want a push checked by the plane.
#### Acceptance Criteria
1. `ousast plane scan <repo-url> --base <commit> --head <commit>` generates Workspaces from the URL and commits,
derives candidates from the changed functions (quick rules, inferred roles, repository facts), and submits a
Run of facts, verify a/b, tie-break and agree with per-task budgets; a whole-repository mode is secondary.
2. Results are written to the memory store under the repository and head commit and reported with the
attribution table; the decision engine's verdict is reported only when a model is adopted, else the plane's
agreement (ai-service-plane Req 6 and learned-decision-engine Req 6 unchanged).
3. The latency split is explicit: a blocking pre-push carries quick rules and the delta engine within its
deadline; plane verdicts arrive asynchronously.
### Requirement 7: Proven on kind, then documented for the target
**User Story:** As the maintainer, I want each step shown working on kind before I touch the real cluster.
#### Acceptance Criteria
1. Every requirement above is exercised end to end on the kind cluster (the in-cluster reconciler included), with
a recorded run under `benchmarks/measurements/`.
2. `docs/deployment.md` becomes the runbook: prerequisites for the target cluster (Substrate, ax, registry,
buckets, Secrets, node requirements for gVisor workers), the profile switch, and a checklist whose every item
maps to a requirement here; planned items are labelled planned until their run is recorded.
+13
View File
@@ -0,0 +1,13 @@
{
"feature_name": "plane-on-kubernetes",
"created_at": "2026-10-02T20:00:00Z",
"updated_at": "2026-10-02T20:00:00Z",
"language": "en",
"phase": "requirements-generated",
"approvals": {
"requirements": {"generated": true, "approved": false},
"design": {"generated": false, "approved": false},
"tasks": {"generated": false, "approved": false}
},
"ready_for_implementation": false
}