mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs
Release-readiness hardening. - redaction.py (zero-dep): redact_secrets masks provider API keys, AWS/GitHub/Slack/ Google tokens, bearer tokens, URL-embedded credentials, PEM private keys, and sensitive key=value assignments. Wired into the trace writer and the markdown report (on by default), so scan artifacts never leak a credential quoted from source. - provider/openrouter.py: call_with_retry — exponential backoff on transient failures (HTTP 429/5xx, connection errors, timeouts); non-transient errors fail fast. Both the chat and embedding clients use it (max_attempts/retry_base_delay configurable). - config [hardening]: redact_secrets (default true) + max_findings (bounded-CI output cap, 0 = unlimited; truncation is severity-ordered and disclosed as a `budget` degradation). Agentic spend stays bounded by [harnessx] max_cost_usd/token_threshold. - docs/threat-model.md (trust boundaries, controls, sandbox limits) + docs/examples.md (six end-to-end walkthroughs); README "Security & hardening" section links both. - CI lint/typecheck/test/detection-gate/dagger gates already existed. tests/test_redaction.py, test_provider_retry.py, test_hardening_pipeline.py + config tests. 206 passed, 4 skipped; ruff + mypy clean; detection gate PASS. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
3fd0057e5f
commit
b5e40318db
@@ -0,0 +1,73 @@
|
||||
# End-to-end examples
|
||||
|
||||
Each example is self-contained. Artifacts land under
|
||||
`<target>/.openultrasast/runs/<scan-id>/` and `.openultrasast/` is gitignored.
|
||||
|
||||
## 1. Quick scan a local repo (deterministic, no keys)
|
||||
|
||||
```bash
|
||||
uv run ousast scan /path/to/code --mode quick
|
||||
run=/path/to/code/.openultrasast/runs/<scan-id>
|
||||
cat "$run/report.md" # findings + evidence (secrets redacted)
|
||||
jq . "$run/score.json" # 0–100 project score + gate verdict
|
||||
"$run/report.sarif" # SARIF for code-scanning / IDEs
|
||||
```
|
||||
|
||||
## 2. CI gate
|
||||
|
||||
```bash
|
||||
# Fail the build on any evidence-verified finding:
|
||||
uv run ousast scan . --mode quick --fail-on verified
|
||||
|
||||
# The hard detection gate (>=90% recall, <10% FP), score-independent:
|
||||
uv run python -m openultrasast.gate
|
||||
```
|
||||
|
||||
## 3. Benchmark a known-vulnerable corpus
|
||||
|
||||
```bash
|
||||
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
|
||||
# -> expected / matched / missed counts + a per-rule recommendation delta
|
||||
```
|
||||
|
||||
## 4. Self-improve the ruleset from benchmark feedback
|
||||
|
||||
```bash
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run # preview
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml # apply
|
||||
# Accepted edits land in <target>/.openultrasast/calibration/rule_policy.json,
|
||||
# which the next scan/benchmark of that target loads automatically.
|
||||
```
|
||||
|
||||
## 5. Standard mode with the HarnessX agentic plane (OpenAI)
|
||||
|
||||
```bash
|
||||
uv sync --extra harnessx
|
||||
export OPENAI_API_KEY=sk-...
|
||||
cat > openultrasast.toml <<'TOML'
|
||||
[models]
|
||||
hunter = "gpt-4o"
|
||||
verifier = "gpt-4o"
|
||||
[harnessx]
|
||||
provider = "openai"
|
||||
max_cost_usd = 2.0
|
||||
token_threshold = 120000
|
||||
[fusion]
|
||||
panel_model = "gpt-4o" # two-panel adjudication on triggered findings
|
||||
[hardening]
|
||||
redact_secrets = true
|
||||
max_findings = 0
|
||||
TOML
|
||||
uv run ousast scan /path/to/code --mode standard --config openultrasast.toml
|
||||
jq '.degradations' "$run/manifest.json" # null = HarnessX ran; entries = it fell back
|
||||
jq '.fusion' "$run/manifest.json" # per-finding dispositions
|
||||
```
|
||||
|
||||
## 6. Drive it from an MCP client (OpenCode / IDE)
|
||||
|
||||
```jsonc
|
||||
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
|
||||
```
|
||||
|
||||
Then: `openultrasast.scan {path}` → `run_dir` → `openultrasast.findings {run_dir}` →
|
||||
`openultrasast.explain {run_dir, finding_id}`. See the README "MCP server" section.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Threat model & hardening
|
||||
|
||||
OpenUltraSAST analyzes **untrusted code**. This document states what it trusts, what
|
||||
it does not, and the controls that keep a scan safe to run in CI and safe to share.
|
||||
|
||||
## Trust boundaries
|
||||
|
||||
- **Scanned code is untrusted input, never executed.** Quick and standard modes read
|
||||
source as text (regex rules, evidence ladder, optional LLM review). They never import,
|
||||
build, or run the target. Only `deep` mode would execute target-derived code, and only
|
||||
inside the sandbox below — and deep mode is not yet implemented (it exits with a clear
|
||||
message rather than running anything).
|
||||
- **The ruleset and CWE policy are trusted, governed data.** They change only through
|
||||
reviewed commits or the bounded self-improvement loop (`ousast improve`), which can
|
||||
flip rule *status* and tune score constants but can never edit pattern text or the
|
||||
authoritative 0–5 severity. See the self-improving cycle in the README.
|
||||
- **Provider credentials are the operator's.** The zero-dependency core makes no network
|
||||
calls. The optional HarnessX agentic plane sends source excerpts to the configured LLM
|
||||
provider only when a model is set in `[models]`; keys are read from the provider's
|
||||
standard env var (`OPENAI_API_KEY` / `ANTHROPIC_API_KEY`), never written to artifacts.
|
||||
|
||||
## Controls
|
||||
|
||||
### Secret redaction
|
||||
Scan artifacts can quote source that contains live credentials. `redaction.py` masks
|
||||
recognizable secret shapes (provider API keys, AWS/GitHub/Slack/Google tokens, bearer
|
||||
tokens, URL-embedded credentials, PEM private keys, and `key = value` assignments for
|
||||
sensitive names) before traces (`trace/events.jsonl`) and the markdown report are
|
||||
written. On by default; disable with `[hardening] redact_secrets = false`.
|
||||
|
||||
### Cost & CI budgets
|
||||
- **Agentic spend** is bounded per task by `[harnessx] max_cost_usd` and
|
||||
`token_threshold` (enforced inside the HarnessX run loop) and by the per-tier hunter
|
||||
budgets.
|
||||
- **Output size** is bounded by `[hardening] max_findings` (0 = unlimited). Truncation is
|
||||
severity-ordered and disclosed as a `budget` degradation in the manifest.
|
||||
|
||||
### Provider reliability
|
||||
LLM/embedding calls retry transient failures (HTTP 429/5xx, connection errors, timeouts)
|
||||
with exponential backoff; non-transient errors (e.g. 4xx, malformed JSON) fail fast.
|
||||
|
||||
### Visible degradation & determinism
|
||||
When an optional capability is missing (the extra, a model, a provider key), the scan
|
||||
falls back to its deterministic equivalent and records a `degradations` entry in the
|
||||
manifest — it is never silently downgraded. Runs are reproducible: fixed config,
|
||||
artifact manifests, prompt hashes, and model identifiers are recorded.
|
||||
|
||||
### MCP surface
|
||||
`ousast mcp` exposes only the ten narrow project tools. No tool runs arbitrary shell,
|
||||
Docker, or internal hunter tools, and no tool accepts a free-form command argument.
|
||||
|
||||
## Sandbox limits (deep mode, when implemented)
|
||||
|
||||
Dynamic analysis runs under the bounds in `[sandbox]` (`config.SandboxConfig`):
|
||||
|
||||
| Control | Default |
|
||||
|---|---|
|
||||
| network | disabled (`network = false`) |
|
||||
| workspace | read-only (`workspace_readonly = true`) |
|
||||
| memory | 2048 MB (`memory_mb`) |
|
||||
| pids | 512 (`pids_limit`) |
|
||||
| timeout | 300 s (`timeout_seconds`) |
|
||||
|
||||
Dynamic probes additionally declare their network scope, log commands, and capture
|
||||
artifacts; a hardened runtime (e.g. gVisor) is used when available. Until deep mode
|
||||
ships, no target code is executed at all.
|
||||
Reference in New Issue
Block a user