marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(readme): front door -- current commands, status, and links to every docs page

Browse Source 
Rewrite README.md as the front door of the docs site: a short "What it is", the
Status (NO-GO against M4, figures quoted from docs/where-we-stand.md), a tiered
install (quick scan without Docker; standard/deep with the openultrasast:dev
image; the plane on ax for maintainers; k3s planned), then one section per
docs page, each linked by relative path so it works on GitHub and on the site.

Stale claims removed: `ousast index` (deleted), `OPENROUTER_EMBEDDING_MODEL`
(no longer read), the duplicated scan-mode, rule and gate tables (now on
docs/scanning.md and docs/evaluation.md), the HarnessX-era wording. Adds a
Licence section stating that no licence file exists yet.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 20:06:21 +02:00
co-authored by Claude Fable 5.1
parent c978de9ff8
commit bdabb978f8
1 changed files with 197 additions and 224 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+197 -224
View File
@@ -1,270 +1,221 @@
# OpenUltraSAST # OpenUltraSAST
**v2.0.0 (Python package `2.0.0`), state as of 2026-10-02.** An experimental static OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
security analyser. The pre-push safety net is **NO-GO** for rollout: no hook capability is repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
enabled and no independent population has passed the qualification gates (see and separates what it can prove from what it merely suspects. A scan combines language-scoped
[docs/evaluation.md](docs/evaluation.md)). See the [release notes](RELEASE_NOTES.md). pattern rules, a tree-sitter overlay, a Joern code property graph and, optionally, a model that is
asked only the question the graph cannot settle; every finding carries the evidence that earned
its place. A scan that could not look at something records a degradation instead of reporting a
clean result. `ousast` is the interface; model work over whole repositories runs on a separate
agentic plane (google/ax), and a learned decision engine is in development. Neither is needed
for a scan.
OpenUltraSAST separates what it can prove from what it merely suspects, and says which is which. This file is the front door; each section links to its page of the documentation site (`docs/`,
A scan combines language-scoped pattern rules, a semantic overlay, a Joern-based taint engine `mkdocs.yml`; `uv sync --extra docs && uv run mkdocs serve`).
and optional model calls; every finding carries the evidence that earned its place, and a scan
that could not look at something records that as a degradation instead of reporting a clean
result.
`ousast` is the interface. Model-driven work over whole repositories runs on a separate ## Status
agentic plane (google/ax), and a learned decision engine that weighs all the signals is in
development; neither is needed for a scan.
## Install and quickstart **v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate:
no hook capability is enabled, and the two independent populations spent so far gave engine recall
1 of 11 and then 0 of 17. The learned decision engine ranks but may not block: BLOCK is offered for
no family. The project has stopped trying to reach the gates by hand; every detection change now
has to win a controlled measurement on held-out repositories. The one page that states all of this
with the record behind each figure is [docs/where-we-stand.md](docs/where-we-stand.md); the
release history is in [RELEASE_NOTES.md](RELEASE_NOTES.md).
## Install and first scan
Which tier you need: a deterministic scan of a checkout or a CI gate is tier 1; the Joern engine
(`standard`, `deep`) is tier 2; running model work over whole repositories on the plane is tier 3,
for maintainers; tier 4 does not exist yet.
### Tier 1: quick scan, no Docker
The core install depends only on PyYAML. From a checkout, `uv sync` (or `pip install .`) installs
the `ousast` entry point; `uv sync --group dev` adds the test and lint tools.
```bash ```bash
uv sync --group dev uv sync
# Deterministic scan, no model calls
uv run ousast scan /path/to/repo --mode quick uv run ousast scan /path/to/repo --mode quick
uv run ousast scan . --mode quick --fail-on verified # CI: exit 1 on any evidence-verified finding
# CI: exit 1 on any evidence-verified finding
uv run ousast scan . --mode quick --fail-on verified
``` ```
The core install depends only on PyYAML. Optional extras (`pyproject.toml`): `semantic` This runs the language-scoped pattern rules, entry-point reachability hints, ranking,
(tree-sitter grammars for the overlay), `s3` (boto3, for the plane's memory store on an verification and scoring: deterministic, reproducible, no network, no keys. What it cannot do:
S3-compatible bucket) and `docs` (MkDocs with the Material theme, for the documentation site); there is no engine, so nothing follows a value across statements or files, and the PHP quick rules
install one with `uv sync --extra s3`. are a low-precision, in-sample development set (per-rule precision lower bounds from 0.03 to 1.0 on
the real pins, `benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). The optional
`semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for the overlay.
For `standard` and `deep` scans, use the Docker image, which ships Joern (with `php-cli` for ### Tier 2: standard and deep scans (Docker)
the PHP frontend) next to the tool. Source the shell wrapper once and `ousast` runs in the
container with your directory mounted read-only: `standard` and `deep` need Joern. The Docker image `openultrasast:dev` ships the tool, Joern
v4.0.625 and `php-cli` (needed by the PHP frontend) in one image; it is built locally from the
`Dockerfile` (no published image yet), and it is big: a JRE plus about 2 GB of Joern. The shell
wrapper builds it on first use and then runs `ousast` in the container with your directory mounted
read-only and the container network off:
```bash ```bash
source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1 source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1
ousast scan . --mode standard ousast scan . --mode standard # the graph decides; no model is asked
ousast-with-judge scan . --mode standard # network on, the model is asked (DEEPSEEK_API_KEY)
``` ```
Details, including a pinned host install of Joern, are in [ops/README.md](ops/README.md). `docker compose build` and `TARGET=/path/to/repo docker compose run --rm ousast scan /target` do
the same without the wrapper. `deep` adds REGRESS, which runs each reproduction snippet with
`docker run` from wherever the scan runs; the engine container has no Docker socket, so run `deep`
from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
verified) and a working `docker`, or accept the recorded `sandbox_unavailable` degradation.
Details, including the host install, are in [ops/README.md](ops/README.md).
Every run writes auditable artifacts under `<target>/.openultrasast/runs/<scan-id>/`: Keys and store settings live in `.env` in the working directory (gitignored; `.env.example` lists
`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`, the names). `DEEPSEEK_API_KEY` serves every LLM call; `OPENROUTER_API_KEY` is used for embeddings
`report.sarif` and `trace/events.jsonl`, plus the stage artifacts of the mode. only (`ousast learn memory embed`). `.env` never overrides a variable already exported in your
shell, so `unset` a stale export when a key change seems to have no effect. The plane's memory
store defaults to a local file store under `~/ousast-results/plane/memory`; an S3-compatible bucket
is optional (`uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT` and the
AWS credentials in `.env`). Nothing in a scan needs it.
### Tier 3: the plane on ax (maintainers)
None of this is needed for tier 1 or 2. Today's profile is a single-node kind cluster on the
maintainer's host. `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the
PATH (it refuses to start when one is missing) and builds Agent Substrate and ax from their
checkouts under `~/.cache/ousast/ax-src/`. Measured idle footprint with two workers: the kind node
takes 1.7 GiB of memory (27% of the 7.7 GiB host) and 4.9 GB of disk for images, with several GB
more of Go caches during the build; the worker pool is sized for that 7 GB host. The verify and
roles tasks need `DEEPSEEK_API_KEY`, and memory lives in the local file store or the S3 store
(RustFS is the tested server). Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
### Tier 4: Kubernetes (k3s), planned
No deployment to any cluster other than that kind cluster has been made. The target is two
profiles of one code base, `kind` locally and `k3s` in production, where the same `ousast plane run`
submits a Run in a remote execution mode: it needs a kubeconfig context or the cluster's ingress
(and an `imagePullSecret` on the worker ServiceAccount for a private package), runner and engine
images published to GHCR and pinned by digest, and the S3 store as the artifact path, with no
receiver on a laptop. The plan, and the code changes it needs with file paths, are in
[docs/deployment.md](docs/deployment.md#production-topology-ax-on-a-kubernetes-cluster).
## Scan modes ## Scan modes
| Mode | What runs | Needs | `quick` runs the pattern rules, hints, ranking, verification and score and needs nothing.
| --- | --- | --- | `standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and the
| `quick` | Language-scoped pattern rules, entry-point reachability hints, ranking, verification, scoring. Deterministic and reproducible. | nothing | Joern model layer, which builds one code property graph per repository and decides taint,
| `standard` | `quick` plus the MAP stage: complexity map, semantic overlay, authorization obligations, and the model layer, which builds one Joern code property graph per repository and decides taint, guard-dominance and configuration questions on it. An LLM is asked only the residual question the graph cannot settle (a `suspicion`); `[models] hunter` adds the tool hunter. | Joern for the model layer (else a `cpg_unavailable` degradation); a provider key for the LLM parts (else they are skipped and only the graph's entailed findings are reported) | guard-dominance and configuration questions on it; a model is asked only the residual question
| `deep` | The MAP stage of `standard` plus REGRESS: promoted candidates are loaded by a small snippet inside a Docker sandbox (no network, read-only source, non-root, memory/pid/time limits) and get a `triggerable` / `not_triggerable` / ... verdict. | a working `docker` (else a `sandbox_unavailable` degradation) | (without a key it is skipped and only the graph's entailed findings are reported). `deep` adds
REGRESS: promoted candidates are loaded by a small snippet in a Docker sandbox and get a
`triggerable` or `not_triggerable` verdict. A missing engine is a recorded degradation
(`cpg_unavailable`, `sandbox_unavailable`), never a clean result.
`--fail-on never|findings|verified|worth-fixing` sets the exit code: `findings` fails on any `--fail-on never|findings|verified|worth-fixing` sets the exit code and `--config openultrasast.toml`
reported finding, `verified` on any evidence-verified one, `worth-fixing` on a deep-mode loads settings. Every run writes `manifest.json`, `findings.json`, `verification.json`,
verdict that is worth fixing. `--config openultrasast.toml` loads settings. `score.json`, `report.md`, `report.sarif` and `trace/events.jsonl` under
`<target>/.openultrasast/runs/<scan-id>/`. `ousast pre-push` (experimental) analyses the commits a
push would publish against their base so that only new or worsened defects are candidates; it is
advisory by default and its capability registry is empty, so no normal alert is emitted today.
Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md).
## Detection coverage (quick rules) ## How detection works
Rules run only against files of their language. The bundled ruleset lives in Three representations, from cheap to expensive: text (regex quick rules, one statement at a time),
`src/openultrasast/ruleset/<language>/rules.toml`: tree (the tree-sitter CST and an IR per function, for the overlay and the obligations) and graph
(the Joern CPG, the only one that follows a value across statements, calls and files). Taint runs
as fact tables, queries and verdicts on the graph; absence bugs (an operation that should have
been guarded) are found as obligations, dominance and configuration census; a model answers only
the residual question. [docs/detection-techniques.md](docs/detection-techniques.md) explains each
technique with the code it comes from and what it misses;
[docs/architecture.md](docs/architecture.md) gives the stage order, the evidence ladder (a state
machine, not a label a model may assign), the central CWE policy and the project score, and the
self-improving loops (`ousast improve`, with `--memory` for proposals from plane rows).
| Language | Classes (CWE) | Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP, and
| --- | --- | run only against files of their language. The detection gate (`uv run python -m openultrasast.gate`)
| C / C++ | stack buffer overflow (121), format string (134), command injection (78) | enforces at least 90% recall and under 10% false positives per language on the bundled cheat-sheet
| Python | command injection (78), code injection (95), SSTI (94), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), insecure default (489) | corpora; on this tree it reports 93.62% recall (44/47) and 0.0% false positives. Those corpora show
| JavaScript / TS | command injection (78), code injection (95), XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) | that a rule change did not break a known detection; they are not a recall estimate on real code,
| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) | and PHP is not in the gate. Framework knowledge (WordPress, Flask, Django, Express, Spring) is
| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules | tagged as an optional prior in `src/openultrasast/ruleset/frameworks.toml`, so its contribution can
be measured and switched off.
The detection gate (`uv run python -m openultrasast.gate`, also `tests/test_detection_benchmarks.py`) ## The plane on ax
enforces the project goal of **at least 90% recall and under 10% false positives** per language
on the bundled corpora in `benchmarks/manifests/`. Its output on this tree:
| Language | Recall | False-positive rate | Model-driven work over whole repositories runs as a Run manifest, a DAG of ax Tasks, each in its
| --- | --- | --- | own gVisor actor on Agent Substrate. ax is the only agentic executor; there is no local
| C / C++ | 93.3% (14/15) | 0.0% | subprocess path. Each Task binds at most one Model and its own `usd`/`calls` budget, egress is
| Python | 93.3% (14/15) | 0.0% | deny-by-default per Task (hostnames only), and the provider key travels only in the Task's start
| JavaScript | 92.3% (12/13) | 0.0% | request, never in a manifest or an image. Measured on the first increment: the plane agreed on 16
| Java | 100% (4/4) | 0.0% | of 20 declared sites at $0.0205 per candidate.
| **Overall** | **93.6% (44/47)** | **0.0%** |
These corpora are cheat-sheet fixtures modelled on deliberately vulnerable applications. They
show that a rule change did not break a known detection; they are not a recall estimate on
real code. Real-code results are in [docs/evaluation.md](docs/evaluation.md).
**PHP is not in the gate, and its numbers are in-sample only.** The PHP rules were written and
tightened while looking at the development corpus they were measured on
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): 13 of 14 expected
findings on the `php-vulnerable` fixture with no false positive, 2 false positives on
`php-benign`, and per-rule precision lower bounds from 0.03 to 1.0 on the real pins. Treat
them as development numbers, not as a holdout result.
**Framework knowledge is optional.** Rule and taint-fact entries that know a framework or
library (WordPress, Flask, Django, Express, Spring, ...) carry a `framework` or `library` tag
naming a row of `src/openultrasast/ruleset/frameworks.toml`. The loaders take `priors=`
(`"all"`, the default and today's behaviour; `"off"`, language-level knowledge only; or a set
of ids), so their contribution can be measured and switched off.
## Pre-push checks (experimental)
`ousast pre-push` analyses the commits a `git push` would publish, or an explicit
`--base`/`--head` pair, and compares the head against its base so that only new or worsened
defects are candidates. `--mode advisory` (the default) never blocks; `--mode blocking` and
`--incomplete-coverage block` opt into enforcement. The artifact must be written outside the
analysed repository.
```bash
uv run ousast pre-push /path/to/repo --base BASE --head HEAD \
--artifact /outside/the/repo/result.json
```
The default capability registry is empty, so no normal alert is emitted today; every
candidate stays diagnostic in the artifact. Installation into a repository's hooks
(`ops/install-pre-push`), caching and the optional witness model are documented in
[ops/README.md](ops/README.md).
## Model providers and keys
Keys and store settings live in `.env` in the working directory (gitignored, never committed;
`.env.example` lists the names):
```bash
DEEPSEEK_API_KEY=... # every LLM call: model layer, tool hunter, plane tasks, decision engine
OPENROUTER_API_KEY=... # embeddings only (`ousast learn memory embed`)
# Plane memory store (maintainers). Unset: a local file store under ~/ousast-results/plane/memory.
OUSAST_MEMORY=s3://<bucket> # or s3://<bucket>/<prefix>; needs the s3 extra
S3_ENDPOINT=https://host[:port] # the full URL
S3_REGION=us-east-1 # optional; spares a GetBucketLocation call
S3_BUCKET=<bucket> # the bucket `OUSAST_MEMORY=s3://` alone names
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
```
The project uses DeepSeek (`deepseek-flash`) for LLM calls and OpenRouter only for embeddings.
Set `DEEPSEEK_API_KEY`: without it, chat calls fall back to OpenRouter when that key is set
(`tool_hunter.resolve_hunter_client`). The S3 bucket is set up once by an admin and verified
by the store at every start ([docs/rustfs.md](docs/rustfs.md)). `ousast pre-push` does not read
`.env`; it takes credentials from the Git process environment.
**`.env` never overrides a variable already exported in your shell.** A stale
`export DEEPSEEK_API_KEY=...` in your profile wins over the `.env` file silently; `unset` it
when a key change in `.env` seems to have no effect.
## Benchmarks and pair evaluation
```bash
# Score the scan against a ground-truth manifest
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
# Vulnerable-vs-fixed pairs: fire on the vulnerable side, stay silent on the fix
uv run ousast pairs
uv run ousast pairs --slice sast # OWASP Benchmark + Juliet
uv run ousast pairs --slice vibe-py --pointers # also fetch non-vendored pointer pairs (network)
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
```
Each benchmark run writes `.openultrasast/benchmarks/<id>/` (`benchmark_result.json`,
`calibration_records.json` with every miss, `external_baseline_deltas.json`). The pair
catalogs live in `benchmarks/pairs/`.
## Self-improvement
`ousast improve` runs a bounded loop over the governed ruleset: benchmark, propose edits to
rule status and score constants only, validate, re-benchmark, and accept a round only if
recall stays at least 90%, false positives under 10%, the project score does not regress and
no matched finding is lost. Accepted rounds land in
`<target>/.openultrasast/calibration/rule_policy.json`, which the next scan of that target loads.
```bash
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
```
With `--memory`, proposals also come from the rows plane runs stored (see below), through the
same validator and gate; rows from holdout pairs, from the gated manifest's own cases and from
any `--qualify-population` are dropped first. Details:
[docs/architecture.md](docs/architecture.md#self-improvement).
## The agentic plane (maintainers)
Model work over whole repositories runs as a Run manifest on [google/ax](ops/ax/README.md)
over Agent Substrate, one isolated actor per task. ax is the only agentic executor: there is
no local subprocess path, and a scan needs none of it. Each task binds its own Model and its
own `usd`/`calls` budget; egress is deny-by-default per task (one EgressPolicy per actor,
hostnames only: the artifact receiver, the Workspaces' Git hosts, the Model's declared host);
and the provider key travels only in the task's start request, sent through Substrate's
`atenet-router` to that task's actor, never in a manifest or an image.
What runs where today: the reconciler (`ousast plane run`) and the artifact receiver run on
the operator's host; the tasks run in gVisor sandboxes on Agent Substrate in a single-node kind
cluster on the maintainer's host; the memory store is a local file store or an S3-compatible
bucket (`OUSAST_MEMORY=s3://<bucket>`; RustFS is the tested server).
```bash ```bash
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
uv run ousast plane run plane/runs/validation-46.yaml # a rerun skips tasks already done; --workers bounds tasks in flight uv run ousast plane run plane/runs/validation-46.yaml # a rerun skips tasks already done; --workers bounds tasks in flight
uv run ousast plane status validation-46 # per-task status and the token attribution table (--units per unit) uv run ousast plane status validation-46 # per-task status and the token attribution table
uv run ousast plane remember validation-46 # ingest the run's rows into the memory store uv run ousast plane remember validation-46 # ingest the run's rows into the memory store
uv run ousast plane memory-normalise --dry-run # count the stored rows a rewrite to the fixed schema would touch
``` ```
`ousast plane workspaces`, `harvest` and `alerts-engine` generate Runs and their inputs. `workspaces`, `harvest`, `alerts-engine` and `memory-normalise` generate Runs, their inputs and
Bring-up, the task catalogue, the memory store (`OUSAST_MEMORY`) and the host's lessons are in the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md); what runs
[ops/ax/README.md](ops/ax/README.md); what runs where, this host's setup and the guidance for a where today and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
separate Kubernetes cluster (planned, not done) are in [docs/deployment.md](docs/deployment.md).
> HarnessX, the earlier optional agentic extra, was retired 2026-09-30, and a leftover HarnessX, the earlier agentic extra, was retired on 2026-09-30: a `[harnessx]` section is ignored
> `[harnessx]` section (retired 2026-09-30) is ignored with one warning. `[models] verifier`, with one warning, and `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail
> `[fusion] panel_model` and `[fusion] decider_model` fail with a message naming the plane naming the plane replacement, as do config keys nothing ever read. Fusion is deterministic.
> replacement. The LLM judge and the LLM fusion panels were retired with it; fusion is
> deterministic. See [RELEASE_NOTES.md](RELEASE_NOTES.md).
## Decision engine (in development, not adopted) ## Memory
A learned decision engine is being built to replace hand-tuned detection edits: each One store, keyed by repository and pin, holds what plane runs and the decision engine learned:
instrument (quick rules, the Joern engine, model sink classification, verify passes, repository rows (facts, verdicts, unit costs, alerts, coverage, feature records) and content-addressed blobs
facts) produces signals, never verdicts, and a compiled AI classifier with local memory turns (excerpts, embeddings, cached responses, compiled programs). `OUSAST_MEMORY` selects the backend:
them into a calibrated probability per candidate with BLOCK and ADVISORY operating points. a local file store, or `s3://<bucket>` on any S3-compatible server with versioning, lifecycle
No scan, pre-push or report uses it yet, and it is not adopted. Measured so far out of rules, object tags and S3 Select; RustFS is the tested server. An admin sets the bucket up once;
repository (slice 2, 2026-10-02, six families): pooled AUC 0.77 to 0.96 per family, within-pair the store verifies it at every start and refuses to run, naming each missing piece, rather than
AUC 0.72 to 0.93, BLOCK offered for no family, re-run agreement below 0.9 for three of the six fall back. Layout and the read and write paths: [docs/memory.md](docs/memory.md); the bucket
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Status and the policy and admin steps: [docs/rustfs.md](docs/rustfs.md); how stored rows are meant to turn into
numbers: [docs/decision-engine.md](docs/decision-engine.md). better decisions: [docs/memory-and-detection.md](docs/memory-and-detection.md).
## Agent integrations ## Decision engine and evaluation
An agent that can run shell commands needs nothing but the CLI. For A learned decision engine is being built to replace hand-tuned detection edits: every instrument
[OpenCode](https://opencode.ai) there are two optional project skills in `.agents/skills/`: produces signals, never verdicts; labels come from ground truth only; a compiled AI classifier with
`openultrasast-scan` (modes and their prerequisites, `--fail-on`, the evidence ladder, recorded local memory turns retrieved similar cases and the candidate's signals into a calibrated
degradations, run artifacts, `ousast mcp`) and `openultrasast-triage` (false-positive reasons, probability with BLOCK and ADVISORY operating points. It is **in development, not adopted**: no
what an adjudication may feed, fixing and re-scanning). scan, pre-push check or report uses it. Measured on six families, on repositories the compiled
program was not shown, the pooled AUC of its score runs from 0.77 (injection) to 0.96 (access
control); the stricter within-pair AUC is 0.72 to 0.93; BLOCK is offered for no family
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Maintainer surface:
`ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`. Design, slices and
the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
`ousast mcp` runs a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools: How anything is measured on code the tool was not tuned on, the pair corpora, the independent
`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, populations and the M4 gates, and the fold discipline that keeps learned decisions out of their
`benchmark`, `explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a own training data: [docs/evaluation.md](docs/evaluation.md). Population v3 (PHP, 15 cases) is
shell, Docker or a free-form command. frozen as the one-time final check and nothing in the tree may read it.
```jsonc ```bash
{ "command": "uv", "args": ["run", "ousast", "mcp"] } uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
uv run ousast pairs --slice sast # fire on the vulnerable side, stay silent on the fix
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
``` ```
> The `kiro-*` skills, `AGENTS.md` and `.kiro/` are the maintainer's development tooling and ## Token economics
> can be ignored by users.
## Further reading Model tokens are the one running cost. At runtime the plane routes them: one Model and one budget
per Task, a metered client that refuses the next call at the ceiling, repository facts computed
model-free and reused, blobs replayed at $0, triage before the hunt and a tie-break only on
disputed candidates. In development, 85.5% of the maintainers' coding-session tokens were tool
input and output rather than reasoning (down from 90%), and the remedies in `benchmarks/dev/`
target that. Mechanisms,
the measured costs and the before-and-after table: [docs/token-ergonomics.md](docs/token-ergonomics.md).
Documentation site with flow diagrams (`docs/`, `mkdocs.yml`): `uv sync --extra docs && uv run mkdocs serve`. ## Examples, threat model
- [docs/architecture.md](docs/architecture.md): the scan pipeline, evidence ladder, CWE policy End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
and project score, and the self-improving loops. What the tool trusts and what it does not (scanned code is untrusted input, secret redaction,
- [docs/evaluation.md](docs/evaluation.md): independent populations, the qualification gates, per-task egress, budgets, the `deep` sandbox): [docs/threat-model.md](docs/threat-model.md).
pair corpora and the plane increment results.
- [docs/deployment.md](docs/deployment.md): the plane on this host, and the guidance for a
separate Kubernetes cluster (planned) with what is implemented and what is not.
- [docs/decision-engine.md](docs/decision-engine.md): the learned decision engine.
- [docs/threat-model.md](docs/threat-model.md): trust boundaries, sandbox and hardening.
- [docs/token-ergonomics.md](docs/token-ergonomics.md): model spend at runtime (budgets, reuse,
measured costs) and in coding sessions.
- [docs/examples.md](docs/examples.md): end-to-end walkthroughs.
## Development ## Contributing and maintainer tooling
```bash ```bash
uv run pytest # tests, incl. the 90/10 detection gate uv run pytest # tests, incl. the 90/10 detection gate
@@ -276,6 +227,28 @@ uv run python -m openultrasast.pair_gate # local vuln-vs-fix pairs
uv run python dagger/ci.py # containerized CI pipeline uv run python dagger/ci.py # containerized CI pipeline
``` ```
Maintainer commands not covered above: `ousast repos` (pinned known-vulnerable checkouts), An agent that can run shell commands needs nothing but the CLI. For [OpenCode](https://opencode.ai)
`ousast model candidates` (what the candidate enumerator can reach) and `ousast learn` (the decision engine's data; see there are two project skills in `.agents/skills/`: `openultrasast-scan` (modes and their
[docs/decision-engine.md](docs/decision-engine.md)). prerequisites, `--fail-on`, the evidence ladder, recorded degradations, run artifacts) and
`openultrasast-triage` (false-positive reasons, what an adjudication may feed, fixing and
re-scanning). `ousast mcp` runs a narrow MCP server over stdio with ten tools (`openultrasast.scan`,
`status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`, `explain`,
`propose_patch`, `export_report`); no tool runs a shell, Docker or a free-form command:
```jsonc
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
```
The `kiro-*` skills, `AGENTS.md` and the Kiro-style specifications under `.kiro/` are the
maintainer's development tooling and can be ignored by users. Other maintainer commands:
`ousast repos` (pinned known-vulnerable checkouts) and `ousast model candidates` (what the
candidate enumerator can reach).
## Licence
The repository carries no licence file yet (checked 2026-10-02: no `LICENSE` at the root and no
`license` field in `pyproject.toml`), so until one is added, treat the code as all rights reserved
and ask the maintainer before reusing it. Third-party material keeps its own terms: Joern is
downloaded at image build time under its own licence, the tree-sitter wheels of the `semantic`
extra are MIT, and the published benchmark cases were licence-checked one by one (v1's OpenCVE
case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md)).