From bdabb978f83d4c58849a3ed7f50470bb5242e137 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Fri, 2 Oct 2026 20:06:21 +0200 Subject: [PATCH] docs(readme): front door -- current commands, status, and links to every docs page Rewrite README.md as the front door of the docs site: a short "What it is", the Status (NO-GO against M4, figures quoted from docs/where-we-stand.md), a tiered install (quick scan without Docker; standard/deep with the openultrasast:dev image; the plane on ax for maintainers; k3s planned), then one section per docs page, each linked by relative path so it works on GitHub and on the site. Stale claims removed: `ousast index` (deleted), `OPENROUTER_EMBEDDING_MODEL` (no longer read), the duplicated scan-mode, rule and gate tables (now on docs/scanning.md and docs/evaluation.md), the HarnessX-era wording. Adds a Licence section stating that no licence file exists yet. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo --- README.md | 421 +++++++++++++++++++++++++----------------------------- 1 file changed, 197 insertions(+), 224 deletions(-) diff --git a/README.md b/README.md index 6a8ba5f..3049384 100644 --- a/README.md +++ b/README.md @@ -1,270 +1,221 @@ # OpenUltraSAST -**v2.0.0 (Python package `2.0.0`), state as of 2026-10-02.** An experimental static -security analyser. The pre-push safety net is **NO-GO** for rollout: no hook capability is -enabled and no independent population has passed the qualification gates (see -[docs/evaluation.md](docs/evaluation.md)). See the [release notes](RELEASE_NOTES.md). +OpenUltraSAST is an experimental static security analyser meant to work as a safety net on +repositories it has never seen: it scans a checkout, or the commits a `git push` would publish, +and separates what it can prove from what it merely suspects. A scan combines language-scoped +pattern rules, a tree-sitter overlay, a Joern code property graph and, optionally, a model that is +asked only the question the graph cannot settle; every finding carries the evidence that earned +its place. A scan that could not look at something records a degradation instead of reporting a +clean result. `ousast` is the interface; model work over whole repositories runs on a separate +agentic plane (google/ax), and a learned decision engine is in development. Neither is needed +for a scan. -OpenUltraSAST separates what it can prove from what it merely suspects, and says which is which. -A scan combines language-scoped pattern rules, a semantic overlay, a Joern-based taint engine -and optional model calls; every finding carries the evidence that earned its place, and a scan -that could not look at something records that as a degradation instead of reporting a clean -result. +This file is the front door; each section links to its page of the documentation site (`docs/`, +`mkdocs.yml`; `uv sync --extra docs && uv run mkdocs serve`). -`ousast` is the interface. Model-driven work over whole repositories runs on a separate -agentic plane (google/ax), and a learned decision engine that weighs all the signals is in -development; neither is needed for a scan. +## Status -## Install and quickstart +**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate: +no hook capability is enabled, and the two independent populations spent so far gave engine recall +1 of 11 and then 0 of 17. The learned decision engine ranks but may not block: BLOCK is offered for +no family. The project has stopped trying to reach the gates by hand; every detection change now +has to win a controlled measurement on held-out repositories. The one page that states all of this +with the record behind each figure is [docs/where-we-stand.md](docs/where-we-stand.md); the +release history is in [RELEASE_NOTES.md](RELEASE_NOTES.md). + +## Install and first scan + +Which tier you need: a deterministic scan of a checkout or a CI gate is tier 1; the Joern engine +(`standard`, `deep`) is tier 2; running model work over whole repositories on the plane is tier 3, +for maintainers; tier 4 does not exist yet. + +### Tier 1: quick scan, no Docker + +The core install depends only on PyYAML. From a checkout, `uv sync` (or `pip install .`) installs +the `ousast` entry point; `uv sync --group dev` adds the test and lint tools. ```bash -uv sync --group dev - -# Deterministic scan, no model calls +uv sync uv run ousast scan /path/to/repo --mode quick - -# CI: exit 1 on any evidence-verified finding -uv run ousast scan . --mode quick --fail-on verified +uv run ousast scan . --mode quick --fail-on verified # CI: exit 1 on any evidence-verified finding ``` -The core install depends only on PyYAML. Optional extras (`pyproject.toml`): `semantic` -(tree-sitter grammars for the overlay), `s3` (boto3, for the plane's memory store on an -S3-compatible bucket) and `docs` (MkDocs with the Material theme, for the documentation site); -install one with `uv sync --extra s3`. +This runs the language-scoped pattern rules, entry-point reachability hints, ranking, +verification and scoring: deterministic, reproducible, no network, no keys. What it cannot do: +there is no engine, so nothing follows a value across statements or files, and the PHP quick rules +are a low-precision, in-sample development set (per-rule precision lower bounds from 0.03 to 1.0 on +the real pins, `benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). The optional +`semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for the overlay. -For `standard` and `deep` scans, use the Docker image, which ships Joern (with `php-cli` for -the PHP frontend) next to the tool. Source the shell wrapper once and `ousast` runs in the -container with your directory mounted read-only: +### Tier 2: standard and deep scans (Docker) + +`standard` and `deep` need Joern. The Docker image `openultrasast:dev` ships the tool, Joern +v4.0.625 and `php-cli` (needed by the PHP frontend) in one image; it is built locally from the +`Dockerfile` (no published image yet), and it is big: a JRE plus about 2 GB of Joern. The shell +wrapper builds it on first use and then runs `ousast` in the container with your directory mounted +read-only and the container network off: ```bash source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1 -ousast scan . --mode standard +ousast scan . --mode standard # the graph decides; no model is asked +ousast-with-judge scan . --mode standard # network on, the model is asked (DEEPSEEK_API_KEY) ``` -Details, including a pinned host install of Joern, are in [ops/README.md](ops/README.md). +`docker compose build` and `TARGET=/path/to/repo docker compose run --rm ousast scan /target` do +the same without the wrapper. `deep` adds REGRESS, which runs each reproduction snippet with +`docker run` from wherever the scan runs; the engine container has no Docker socket, so run `deep` +from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum- +verified) and a working `docker`, or accept the recorded `sandbox_unavailable` degradation. +Details, including the host install, are in [ops/README.md](ops/README.md). -Every run writes auditable artifacts under `/.openultrasast/runs//`: -`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`, -`report.sarif` and `trace/events.jsonl`, plus the stage artifacts of the mode. +Keys and store settings live in `.env` in the working directory (gitignored; `.env.example` lists +the names). `DEEPSEEK_API_KEY` serves every LLM call; `OPENROUTER_API_KEY` is used for embeddings +only (`ousast learn memory embed`). `.env` never overrides a variable already exported in your +shell, so `unset` a stale export when a key change seems to have no effect. The plane's memory +store defaults to a local file store under `~/ousast-results/plane/memory`; an S3-compatible bucket +is optional (`uv sync --extra s3`, then `OUSAST_MEMORY=s3://` with `S3_ENDPOINT` and the +AWS credentials in `.env`). Nothing in a scan needs it. + +### Tier 3: the plane on ax (maintainers) + +None of this is needed for tier 1 or 2. Today's profile is a single-node kind cluster on the +maintainer's host. `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the +PATH (it refuses to start when one is missing) and builds Agent Substrate and ax from their +checkouts under `~/.cache/ousast/ax-src/`. Measured idle footprint with two workers: the kind node +takes 1.7 GiB of memory (27% of the 7.7 GiB host) and 4.9 GB of disk for images, with several GB +more of Go caches during the build; the worker pool is sized for that 7 GB host. The verify and +roles tasks need `DEEPSEEK_API_KEY`, and memory lives in the local file store or the S3 store +(RustFS is the tested server). Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md). + +### Tier 4: Kubernetes (k3s), planned + +No deployment to any cluster other than that kind cluster has been made. The target is two +profiles of one code base, `kind` locally and `k3s` in production, where the same `ousast plane run` +submits a Run in a remote execution mode: it needs a kubeconfig context or the cluster's ingress +(and an `imagePullSecret` on the worker ServiceAccount for a private package), runner and engine +images published to GHCR and pinned by digest, and the S3 store as the artifact path, with no +receiver on a laptop. The plan, and the code changes it needs with file paths, are in +[docs/deployment.md](docs/deployment.md#production-topology-ax-on-a-kubernetes-cluster). ## Scan modes -| Mode | What runs | Needs | -| --- | --- | --- | -| `quick` | Language-scoped pattern rules, entry-point reachability hints, ranking, verification, scoring. Deterministic and reproducible. | nothing | -| `standard` | `quick` plus the MAP stage: complexity map, semantic overlay, authorization obligations, and the model layer, which builds one Joern code property graph per repository and decides taint, guard-dominance and configuration questions on it. An LLM is asked only the residual question the graph cannot settle (a `suspicion`); `[models] hunter` adds the tool hunter. | Joern for the model layer (else a `cpg_unavailable` degradation); a provider key for the LLM parts (else they are skipped and only the graph's entailed findings are reported) | -| `deep` | The MAP stage of `standard` plus REGRESS: promoted candidates are loaded by a small snippet inside a Docker sandbox (no network, read-only source, non-root, memory/pid/time limits) and get a `triggerable` / `not_triggerable` / ... verdict. | a working `docker` (else a `sandbox_unavailable` degradation) | +`quick` runs the pattern rules, hints, ranking, verification and score and needs nothing. +`standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and the +Joern model layer, which builds one code property graph per repository and decides taint, +guard-dominance and configuration questions on it; a model is asked only the residual question +(without a key it is skipped and only the graph's entailed findings are reported). `deep` adds +REGRESS: promoted candidates are loaded by a small snippet in a Docker sandbox and get a +`triggerable` or `not_triggerable` verdict. A missing engine is a recorded degradation +(`cpg_unavailable`, `sandbox_unavailable`), never a clean result. -`--fail-on never|findings|verified|worth-fixing` sets the exit code: `findings` fails on any -reported finding, `verified` on any evidence-verified one, `worth-fixing` on a deep-mode -verdict that is worth fixing. `--config openultrasast.toml` loads settings. +`--fail-on never|findings|verified|worth-fixing` sets the exit code and `--config openultrasast.toml` +loads settings. Every run writes `manifest.json`, `findings.json`, `verification.json`, +`score.json`, `report.md`, `report.sarif` and `trace/events.jsonl` under +`/.openultrasast/runs//`. `ousast pre-push` (experimental) analyses the commits a +push would publish against their base so that only new or worsened defects are candidates; it is +advisory by default and its capability registry is empty, so no normal alert is emitted today. +Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md). -## Detection coverage (quick rules) +## How detection works -Rules run only against files of their language. The bundled ruleset lives in -`src/openultrasast/ruleset//rules.toml`: +Three representations, from cheap to expensive: text (regex quick rules, one statement at a time), +tree (the tree-sitter CST and an IR per function, for the overlay and the obligations) and graph +(the Joern CPG, the only one that follows a value across statements, calls and files). Taint runs +as fact tables, queries and verdicts on the graph; absence bugs (an operation that should have +been guarded) are found as obligations, dominance and configuration census; a model answers only +the residual question. [docs/detection-techniques.md](docs/detection-techniques.md) explains each +technique with the code it comes from and what it misses; +[docs/architecture.md](docs/architecture.md) gives the stage order, the evidence ladder (a state +machine, not a label a model may assign), the central CWE policy and the project score, and the +self-improving loops (`ousast improve`, with `--memory` for proposals from plane rows). -| Language | Classes (CWE) | -| --- | --- | -| C / C++ | stack buffer overflow (121), format string (134), command injection (78) | -| Python | command injection (78), code injection (95), SSTI (94), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), insecure default (489) | -| JavaScript / TS | command injection (78), code injection (95), XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) | -| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) | -| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules | +Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP, and +run only against files of their language. The detection gate (`uv run python -m openultrasast.gate`) +enforces at least 90% recall and under 10% false positives per language on the bundled cheat-sheet +corpora; on this tree it reports 93.62% recall (44/47) and 0.0% false positives. Those corpora show +that a rule change did not break a known detection; they are not a recall estimate on real code, +and PHP is not in the gate. Framework knowledge (WordPress, Flask, Django, Express, Spring) is +tagged as an optional prior in `src/openultrasast/ruleset/frameworks.toml`, so its contribution can +be measured and switched off. -The detection gate (`uv run python -m openultrasast.gate`, also `tests/test_detection_benchmarks.py`) -enforces the project goal of **at least 90% recall and under 10% false positives** per language -on the bundled corpora in `benchmarks/manifests/`. Its output on this tree: +## The plane on ax -| Language | Recall | False-positive rate | -| --- | --- | --- | -| C / C++ | 93.3% (14/15) | 0.0% | -| Python | 93.3% (14/15) | 0.0% | -| JavaScript | 92.3% (12/13) | 0.0% | -| Java | 100% (4/4) | 0.0% | -| **Overall** | **93.6% (44/47)** | **0.0%** | - -These corpora are cheat-sheet fixtures modelled on deliberately vulnerable applications. They -show that a rule change did not break a known detection; they are not a recall estimate on -real code. Real-code results are in [docs/evaluation.md](docs/evaluation.md). - -**PHP is not in the gate, and its numbers are in-sample only.** The PHP rules were written and -tightened while looking at the development corpus they were measured on -(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): 13 of 14 expected -findings on the `php-vulnerable` fixture with no false positive, 2 false positives on -`php-benign`, and per-rule precision lower bounds from 0.03 to 1.0 on the real pins. Treat -them as development numbers, not as a holdout result. - -**Framework knowledge is optional.** Rule and taint-fact entries that know a framework or -library (WordPress, Flask, Django, Express, Spring, ...) carry a `framework` or `library` tag -naming a row of `src/openultrasast/ruleset/frameworks.toml`. The loaders take `priors=` -(`"all"`, the default and today's behaviour; `"off"`, language-level knowledge only; or a set -of ids), so their contribution can be measured and switched off. - -## Pre-push checks (experimental) - -`ousast pre-push` analyses the commits a `git push` would publish, or an explicit -`--base`/`--head` pair, and compares the head against its base so that only new or worsened -defects are candidates. `--mode advisory` (the default) never blocks; `--mode blocking` and -`--incomplete-coverage block` opt into enforcement. The artifact must be written outside the -analysed repository. - -```bash -uv run ousast pre-push /path/to/repo --base BASE --head HEAD \ - --artifact /outside/the/repo/result.json -``` - -The default capability registry is empty, so no normal alert is emitted today; every -candidate stays diagnostic in the artifact. Installation into a repository's hooks -(`ops/install-pre-push`), caching and the optional witness model are documented in -[ops/README.md](ops/README.md). - -## Model providers and keys - -Keys and store settings live in `.env` in the working directory (gitignored, never committed; -`.env.example` lists the names): - -```bash -DEEPSEEK_API_KEY=... # every LLM call: model layer, tool hunter, plane tasks, decision engine -OPENROUTER_API_KEY=... # embeddings only (`ousast learn memory embed`) - -# Plane memory store (maintainers). Unset: a local file store under ~/ousast-results/plane/memory. -OUSAST_MEMORY=s3:// # or s3:///; needs the s3 extra -S3_ENDPOINT=https://host[:port] # the full URL -S3_REGION=us-east-1 # optional; spares a GetBucketLocation call -S3_BUCKET= # the bucket `OUSAST_MEMORY=s3://` alone names -AWS_ACCESS_KEY_ID=... -AWS_SECRET_ACCESS_KEY=... -``` - -The project uses DeepSeek (`deepseek-flash`) for LLM calls and OpenRouter only for embeddings. -Set `DEEPSEEK_API_KEY`: without it, chat calls fall back to OpenRouter when that key is set -(`tool_hunter.resolve_hunter_client`). The S3 bucket is set up once by an admin and verified -by the store at every start ([docs/rustfs.md](docs/rustfs.md)). `ousast pre-push` does not read -`.env`; it takes credentials from the Git process environment. - -**`.env` never overrides a variable already exported in your shell.** A stale -`export DEEPSEEK_API_KEY=...` in your profile wins over the `.env` file silently; `unset` it -when a key change in `.env` seems to have no effect. - -## Benchmarks and pair evaluation - -```bash -# Score the scan against a ground-truth manifest -uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick - -# Vulnerable-vs-fixed pairs: fire on the vulnerable side, stay silent on the fix -uv run ousast pairs -uv run ousast pairs --slice sast # OWASP Benchmark + Juliet -uv run ousast pairs --slice vibe-py --pointers # also fetch non-vendored pointer pairs (network) -uv run python -m openultrasast.pair_gate # CI: local pairs must all pass -``` - -Each benchmark run writes `.openultrasast/benchmarks//` (`benchmark_result.json`, -`calibration_records.json` with every miss, `external_baseline_deltas.json`). The pair -catalogs live in `benchmarks/pairs/`. - -## Self-improvement - -`ousast improve` runs a bounded loop over the governed ruleset: benchmark, propose edits to -rule status and score constants only, validate, re-benchmark, and accept a round only if -recall stays at least 90%, false positives under 10%, the project score does not regress and -no matched finding is lost. Accepted rounds land in -`/.openultrasast/calibration/rule_policy.json`, which the next scan of that target loads. - -```bash -uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run -uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory -``` - -With `--memory`, proposals also come from the rows plane runs stored (see below), through the -same validator and gate; rows from holdout pairs, from the gated manifest's own cases and from -any `--qualify-population` are dropped first. Details: -[docs/architecture.md](docs/architecture.md#self-improvement). - -## The agentic plane (maintainers) - -Model work over whole repositories runs as a Run manifest on [google/ax](ops/ax/README.md) -over Agent Substrate, one isolated actor per task. ax is the only agentic executor: there is -no local subprocess path, and a scan needs none of it. Each task binds its own Model and its -own `usd`/`calls` budget; egress is deny-by-default per task (one EgressPolicy per actor, -hostnames only: the artifact receiver, the Workspaces' Git hosts, the Model's declared host); -and the provider key travels only in the task's start request, sent through Substrate's -`atenet-router` to that task's actor, never in a manifest or an image. - -What runs where today: the reconciler (`ousast plane run`) and the artifact receiver run on -the operator's host; the tasks run in gVisor sandboxes on Agent Substrate in a single-node kind -cluster on the maintainer's host; the memory store is a local file store or an S3-compatible -bucket (`OUSAST_MEMORY=s3://`; RustFS is the tested server). +Model-driven work over whole repositories runs as a Run manifest, a DAG of ax Tasks, each in its +own gVisor actor on Agent Substrate. ax is the only agentic executor; there is no local +subprocess path. Each Task binds at most one Model and its own `usd`/`calls` budget, egress is +deny-by-default per Task (hostnames only), and the provider key travels only in the Task's start +request, never in a manifest or an image. Measured on the first increment: the plane agreed on 16 +of 20 declared sites at $0.0205 per candidate. ```bash uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image uv run ousast plane run plane/runs/validation-46.yaml # a rerun skips tasks already done; --workers bounds tasks in flight -uv run ousast plane status validation-46 # per-task status and the token attribution table (--units per unit) +uv run ousast plane status validation-46 # per-task status and the token attribution table uv run ousast plane remember validation-46 # ingest the run's rows into the memory store -uv run ousast plane memory-normalise --dry-run # count the stored rows a rewrite to the fixed schema would touch ``` -`ousast plane workspaces`, `harvest` and `alerts-engine` generate Runs and their inputs. -Bring-up, the task catalogue, the memory store (`OUSAST_MEMORY`) and the host's lessons are in -[ops/ax/README.md](ops/ax/README.md); what runs where, this host's setup and the guidance for a -separate Kubernetes cluster (planned, not done) are in [docs/deployment.md](docs/deployment.md). +`workspaces`, `harvest`, `alerts-engine` and `memory-normalise` generate Runs, their inputs and +the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md); what runs +where today and what a separate cluster would take: [docs/deployment.md](docs/deployment.md). -> HarnessX, the earlier optional agentic extra, was retired 2026-09-30, and a leftover -> `[harnessx]` section (retired 2026-09-30) is ignored with one warning. `[models] verifier`, -> `[fusion] panel_model` and `[fusion] decider_model` fail with a message naming the plane -> replacement. The LLM judge and the LLM fusion panels were retired with it; fusion is -> deterministic. See [RELEASE_NOTES.md](RELEASE_NOTES.md). +HarnessX, the earlier agentic extra, was retired on 2026-09-30: a `[harnessx]` section is ignored +with one warning, and `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail +naming the plane replacement, as do config keys nothing ever read. Fusion is deterministic. -## Decision engine (in development, not adopted) +## Memory -A learned decision engine is being built to replace hand-tuned detection edits: each -instrument (quick rules, the Joern engine, model sink classification, verify passes, repository -facts) produces signals, never verdicts, and a compiled AI classifier with local memory turns -them into a calibrated probability per candidate with BLOCK and ADVISORY operating points. -No scan, pre-push or report uses it yet, and it is not adopted. Measured so far out of -repository (slice 2, 2026-10-02, six families): pooled AUC 0.77 to 0.96 per family, within-pair -AUC 0.72 to 0.93, BLOCK offered for no family, re-run agreement below 0.9 for three of the six -(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Status and the -numbers: [docs/decision-engine.md](docs/decision-engine.md). +One store, keyed by repository and pin, holds what plane runs and the decision engine learned: +rows (facts, verdicts, unit costs, alerts, coverage, feature records) and content-addressed blobs +(excerpts, embeddings, cached responses, compiled programs). `OUSAST_MEMORY` selects the backend: +a local file store, or `s3://` on any S3-compatible server with versioning, lifecycle +rules, object tags and S3 Select; RustFS is the tested server. An admin sets the bucket up once; +the store verifies it at every start and refuses to run, naming each missing piece, rather than +fall back. Layout and the read and write paths: [docs/memory.md](docs/memory.md); the bucket +policy and admin steps: [docs/rustfs.md](docs/rustfs.md); how stored rows are meant to turn into +better decisions: [docs/memory-and-detection.md](docs/memory-and-detection.md). -## Agent integrations +## Decision engine and evaluation -An agent that can run shell commands needs nothing but the CLI. For -[OpenCode](https://opencode.ai) there are two optional project skills in `.agents/skills/`: -`openultrasast-scan` (modes and their prerequisites, `--fail-on`, the evidence ladder, recorded -degradations, run artifacts, `ousast mcp`) and `openultrasast-triage` (false-positive reasons, -what an adjudication may feed, fixing and re-scanning). +A learned decision engine is being built to replace hand-tuned detection edits: every instrument +produces signals, never verdicts; labels come from ground truth only; a compiled AI classifier with +local memory turns retrieved similar cases and the candidate's signals into a calibrated +probability with BLOCK and ADVISORY operating points. It is **in development, not adopted**: no +scan, pre-push check or report uses it. Measured on six families, on repositories the compiled +program was not shown, the pooled AUC of its score runs from 0.77 (injection) to 0.96 (access +control); the stricter within-pair AUC is 0.72 to 0.93; BLOCK is offered for no family +(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Maintainer surface: +`ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`. Design, slices and +the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md). -`ousast mcp` runs a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools: -`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, -`benchmark`, `explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a -shell, Docker or a free-form command. +How anything is measured on code the tool was not tuned on, the pair corpora, the independent +populations and the M4 gates, and the fold discipline that keeps learned decisions out of their +own training data: [docs/evaluation.md](docs/evaluation.md). Population v3 (PHP, 15 cases) is +frozen as the one-time final check and nothing in the tree may read it. -```jsonc -{ "command": "uv", "args": ["run", "ousast", "mcp"] } +```bash +uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick +uv run ousast pairs --slice sast # fire on the vulnerable side, stay silent on the fix +uv run python -m openultrasast.pair_gate # CI: local pairs must all pass +uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory ``` -> The `kiro-*` skills, `AGENTS.md` and `.kiro/` are the maintainer's development tooling and -> can be ignored by users. +## Token economics -## Further reading +Model tokens are the one running cost. At runtime the plane routes them: one Model and one budget +per Task, a metered client that refuses the next call at the ceiling, repository facts computed +model-free and reused, blobs replayed at $0, triage before the hunt and a tie-break only on +disputed candidates. In development, 85.5% of the maintainers' coding-session tokens were tool +input and output rather than reasoning (down from 90%), and the remedies in `benchmarks/dev/` +target that. Mechanisms, +the measured costs and the before-and-after table: [docs/token-ergonomics.md](docs/token-ergonomics.md). -Documentation site with flow diagrams (`docs/`, `mkdocs.yml`): `uv sync --extra docs && uv run mkdocs serve`. +## Examples, threat model -- [docs/architecture.md](docs/architecture.md): the scan pipeline, evidence ladder, CWE policy - and project score, and the self-improving loops. -- [docs/evaluation.md](docs/evaluation.md): independent populations, the qualification gates, - pair corpora and the plane increment results. -- [docs/deployment.md](docs/deployment.md): the plane on this host, and the guidance for a - separate Kubernetes cluster (planned) with what is implemented and what is not. -- [docs/decision-engine.md](docs/decision-engine.md): the learned decision engine. -- [docs/threat-model.md](docs/threat-model.md): trust boundaries, sandbox and hardening. -- [docs/token-ergonomics.md](docs/token-ergonomics.md): model spend at runtime (budgets, reuse, - measured costs) and in coding sessions. -- [docs/examples.md](docs/examples.md): end-to-end walkthroughs. +End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md). +What the tool trusts and what it does not (scanned code is untrusted input, secret redaction, +per-task egress, budgets, the `deep` sandbox): [docs/threat-model.md](docs/threat-model.md). -## Development +## Contributing and maintainer tooling ```bash uv run pytest # tests, incl. the 90/10 detection gate @@ -276,6 +227,28 @@ uv run python -m openultrasast.pair_gate # local vuln-vs-fix pairs uv run python dagger/ci.py # containerized CI pipeline ``` -Maintainer commands not covered above: `ousast repos` (pinned known-vulnerable checkouts), -`ousast model candidates` (what the candidate enumerator can reach) and `ousast learn` (the decision engine's data; see -[docs/decision-engine.md](docs/decision-engine.md)). +An agent that can run shell commands needs nothing but the CLI. For [OpenCode](https://opencode.ai) +there are two project skills in `.agents/skills/`: `openultrasast-scan` (modes and their +prerequisites, `--fail-on`, the evidence ladder, recorded degradations, run artifacts) and +`openultrasast-triage` (false-positive reasons, what an adjudication may feed, fixing and +re-scanning). `ousast mcp` runs a narrow MCP server over stdio with ten tools (`openultrasast.scan`, +`status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`, `explain`, +`propose_patch`, `export_report`); no tool runs a shell, Docker or a free-form command: + +```jsonc +{ "command": "uv", "args": ["run", "ousast", "mcp"] } +``` + +The `kiro-*` skills, `AGENTS.md` and the Kiro-style specifications under `.kiro/` are the +maintainer's development tooling and can be ignored by users. Other maintainer commands: +`ousast repos` (pinned known-vulnerable checkouts) and `ousast model candidates` (what the +candidate enumerator can reach). + +## Licence + +The repository carries no licence file yet (checked 2026-10-02: no `LICENSE` at the root and no +`license` field in `pyproject.toml`), so until one is added, treat the code as all rights reserved +and ask the maintainer before reusing it. Third-party material keeps its own terms: Joern is +downloaded at image build time under its own licence, the tree-sitter wheels of the `semantic` +extra are MIT, and the published benchmark cases were licence-checked one by one (v1's OpenCVE +case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md)).