mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
docs: describe the system as of 2026-10-02 (ax plane, AI decision engine, PHP, v3)
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
7b17eacd44
commit
32a2cb8bcf
+24
-15
@@ -13,7 +13,18 @@ jq . "$run/score.json" # 0–100 project score + gate verdict
|
||||
"$run/report.sarif" # SARIF for code-scanning / IDEs
|
||||
```
|
||||
|
||||
## 2. CI gate
|
||||
## 2. Standard scan with the engine (Docker)
|
||||
|
||||
```bash
|
||||
source ops/shell/ousast.sh # once per shell; builds the image on first use
|
||||
ousast scan . --mode standard # Joern model layer, container network off: no model calls
|
||||
ousast-with-judge scan . --mode standard # network on, so the model is asked (needs DEEPSEEK_API_KEY)
|
||||
```
|
||||
|
||||
Without a key the graph still decides; only the `suspicion` band goes unasked, and the manifest
|
||||
records the degradation.
|
||||
|
||||
## 3. CI gate
|
||||
|
||||
```bash
|
||||
# Fail the build on any evidence-verified finding:
|
||||
@@ -23,23 +34,21 @@ uv run ousast scan . --mode quick --fail-on verified
|
||||
uv run python -m openultrasast.gate
|
||||
```
|
||||
|
||||
## 3. Benchmark a known-vulnerable corpus
|
||||
## 4. Benchmark a known-vulnerable corpus
|
||||
|
||||
```bash
|
||||
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
|
||||
# -> expected / matched / missed counts + a per-rule recommendation delta
|
||||
```
|
||||
|
||||
## 4. Vuln vs fixed pair eval (efficiency, not cheat-sheet recall)
|
||||
## 5. Vuln vs fixed pair eval (efficiency, not cheat-sheet recall)
|
||||
|
||||
```bash
|
||||
uv run ousast pairs # local fixtures + vendored GitHub VFCs
|
||||
uv run ousast pairs --slice github --json # honesty dashboard as JSON
|
||||
uv run ousast pairs --slice sast # OWASP Benchmark + Juliet Youden (TPR-FPR)
|
||||
uv run ousast pairs --slice vibe-py --pointers # nightly: pointer pairs via the local cache (network)
|
||||
uv run ousast mechanisms export --slice vibe-py # maintainer: seed mechanism candidates from trusted pairs (offline)
|
||||
uv run ousast improve bench.toml --pair-catalog benchmarks/pairs/catalog.toml # the mechanisms lever admits candidates the holdout pairs prove; the scan then searches them
|
||||
uv run ousast pairs --slice vibe-py --loo # leave-one-out: the corpus's own detection rate (reports/loo.json)
|
||||
uv run ousast pairs --split holdout --profile human # filter by declared split and provenance profile
|
||||
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
|
||||
```
|
||||
|
||||
@@ -48,7 +57,7 @@ Each case is two isolated trees (vuln file vs patched file, same relative path).
|
||||
fix. Misses and fix-side leaks become `miss`/`fp` signals for `ousast improve`.
|
||||
The catalog and public dataset pointers live in `benchmarks/pairs/`.
|
||||
|
||||
## 5. Self-improve the ruleset from benchmark feedback
|
||||
## 6. Self-improve the ruleset from benchmark feedback
|
||||
|
||||
```bash
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run # preview
|
||||
@@ -57,16 +66,16 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
|
||||
# which the next scan/benchmark of that target loads automatically.
|
||||
```
|
||||
|
||||
## 6. Running the agentic plane
|
||||
## 7. Running the agentic plane
|
||||
|
||||
Model work over whole repositories runs as a Run manifest on ax, one isolated actor
|
||||
per task (bring-up: [ops/ax/README.md](../ops/ax/README.md)). The provider key is
|
||||
read from the environment (or `.env`) by `ousast` and sent only in each task's start
|
||||
request.
|
||||
read from the environment (or `.env`, which never overrides an exported variable) by
|
||||
`ousast` and sent only in each task's start request.
|
||||
|
||||
```bash
|
||||
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
|
||||
uv run ousast plane run plane/runs/validation-46.yaml # repo-facts, verify a/b(/c), agree, remember per case
|
||||
uv run ousast plane run plane/runs/validation-46.yaml # per case: repo-facts, verify a/b/c, agree, final, features, remember
|
||||
uv run ousast plane status validation-46 # per-task status and token attribution
|
||||
uv run ousast plane remember validation-46 # re-ingest the run's memory rows (OUSAST_MEMORY)
|
||||
|
||||
@@ -76,14 +85,14 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dr
|
||||
|
||||
Each task in the Run carries its own Model and `budget: {usd, calls}`; a task that
|
||||
reaches its ceiling stops as `unfinished` and resumes on a rerun with a larger
|
||||
budget. A local `ousast scan --mode standard` stays deterministic apart from the
|
||||
optional `[models] hunter` tool hunter.
|
||||
budget. `ousast plane status` prints the token attribution table: calls, prompt,
|
||||
cache-hit and output tokens and USD per task.
|
||||
|
||||
## 7. Drive it from an MCP client (OpenCode / IDE)
|
||||
## 8. Drive it from an MCP client (OpenCode / IDE)
|
||||
|
||||
```jsonc
|
||||
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
|
||||
```
|
||||
|
||||
Then: `openultrasast.scan {path}` → `run_dir` → `openultrasast.findings {run_dir}` →
|
||||
`openultrasast.explain {run_dir, finding_id}`. See the README "MCP server" section.
|
||||
`openultrasast.explain {run_dir, finding_id}`. See the README section "Agent integrations".
|
||||
|
||||
Reference in New Issue
Block a user