marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: humanize README and code comments

Browse Source
This commit is contained in:
Marius Ciepluch
2026-06-21 21:02:18 +02:00
parent 0c0137e591
commit 0d24033f32
4 changed files with 40 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+29 -30
View File
@@ -3,8 +3,8 @@
OpenUltraSAST is an independent OpenCode security harness. It combines
HarnessX-style harness composition, OpenUltraCode verification discipline,
OpenRouter-hosted models and embeddings, Docker-isolated code analysis, and
Trail of Bits security skills into a SAST workflow whose primary product goal is
**false-positive elimination** — turning suspicion into evidence-backed findings
Trail of Bits security skills into a SAST workflow built around one goal:
eliminating false positives by turning suspicion into evidence-backed findings
and learning from rejected claims.
The full specification lives in `.kiro/specs/openrouter-sast-harness/`
@@ -12,7 +12,7 @@ The full specification lives in `.kiro/specs/openrouter-sast-harness/`
as an implementation oracle for proven source-hunting ideas, not as a dependency
or fork target.
> **Maturity legend** — ✅ implemented and tested · 🟡 primitive exists, not yet
> **Maturity legend:** ✅ implemented and tested · 🟡 primitive exists, not yet
> auto-wired into the scan loop · 🧭 designed, on the roadmap. The current
> verified gate is `usable_harness_mvp`.
@@ -52,13 +52,13 @@ repo ──▶ preprocess (language, LOC, tags, fuzz entry points)
──▶ report.md + report.sarif + manifest.json (shared finding IDs)
```
- **quick** ✅ — deterministic, language-scoped pattern rules + reachability +
- **quick** ✅: deterministic, language-scoped pattern rules + reachability +
verification. No model calls, fully reproducible.
- **standard** ✅ scheduling / 🧭 LLM hunters — runs the tiered hunter pool
- **standard** ✅ scheduling / 🧭 LLM hunters: runs the tiered hunter pool
(A/B/C budgets, retrieval context, selected Trail of Bits skill snippets,
JSONL trajectories) and verification; language-aware LLM hunters are the next
upgrade (today standard reuses the deterministic findings per target).
- **deep** 🧭 — sandboxed build/fuzz/dynamic reproduction (not yet implemented).
- **deep** 🧭: sandboxed build/fuzz/dynamic reproduction (not yet implemented).
## Detection coverage
@@ -114,16 +114,16 @@ SQLi, prototype pollution) so recall is not self-fulfilling.
| Java | 100% (4/4) | 0.0% |
| **Overall** | **93.6% (44/47)** | **0.0%** |
The project goal — **≥90% recall and <10% false positives** per language — is
enforced as a regression gate in `tests/test_detection_benchmarks.py`, so a rule
change that drops recall or raises false positives fails CI.
The project goal is **≥90% recall and <10% false positives** per language. The
gate lives in `tests/test_detection_benchmarks.py`, so a rule change that drops
recall or raises false positives fails CI.
## OpenCode commands, fusion, and ultra workflows
OpenUltraSAST is driven from the command line. You can call the `ousast` CLI
directly, or run [OpenCode](https://opencode.ai) in the repository and let the
agent run the harness for you — OpenCode is invoked from the command line, loads
the project skills below, and executes `ousast` plus the triage/fix workflow.
OpenUltraSAST runs from the command line. You can call the `ousast` CLI directly,
or run [OpenCode](https://opencode.ai) in the repository and let the agent run
the harness. OpenCode loads the project skills below, then executes `ousast` plus
the triage/fix workflow.
```bash
opencode run "scan this repo with ousast in quick mode and triage the findings"
@@ -139,13 +139,13 @@ Project skills that steer the harness from OpenCode (`.opencode/skills/`):
| `openultrasast-triage` ✅ | false-positive elimination, ranking calibration, verifier adjudication |
| `openultrasast-fix-audit` ✅ | OpenUltraCode fix lifecycle and adversarial fix review |
**Ultra workflows (OpenUltraCode discipline)** — fixes are never a one-shot
patch prompt. `openultrasast-fix-audit` runs the bounded lifecycle
**Ultra workflows (OpenUltraCode discipline):** fixes are never a one-shot patch
prompt. `openultrasast-fix-audit` runs the bounded lifecycle
`intake → plan → minimal patch → adversarial review → reconcile → fresh
verification → ready`, and a patch can only reach `patch_validated` after
sandboxed validation passes with no accepted blocking findings.
**Fusion (deepening) 🧭** — when a finding needs more reasoning than the normal
**Fusion (deepening) 🧭:** when a finding needs more reasoning than the normal
ranker → hunter → verifier → mapping loop provides (critical/high severity,
verifier disagreement, conflicting static vs semantic evidence, risky fixes, or
an explicit high-assurance request), fusion runs two independent panels that
@@ -184,9 +184,9 @@ When a finding is still rejected after review, `openultrasast-triage` records a
taxonomy (`unreachable_path`, `missing_attacker_control`, `sanitizer_disproved`,
`static_rule_mismatch`, `incorrect_model_assumption`, `duplicate`,
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`), evidence,
and a **scope**. Scoping is the safety property: a rejected `syscall_entry` claim
in `auth/` demotes only `auth/…`, never the whole vulnerability class — proven by
`tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`.
and a **scope**. Scoping keeps the learning narrow: a rejected `syscall_entry`
claim in `auth/` demotes only `auth/…`, never the whole vulnerability class. This
is covered by `tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`.
Demotion (with an audit trail) is preferred over deletion, so an analyst can
always see *why* something was downgraded.
@@ -209,7 +209,7 @@ overflow the regex engine cannot see:
```
The `next_improvement_candidate` routes the gap to the stage that should close
it — a static rule, SARIF source, entry-point mapping, retrieval package,
it: a static rule, SARIF source, entry-point mapping, retrieval package,
language-aware hunter prompt, dynamic reproducer, or skill route. Once a miss is
addressed, it stays closed because the recall/precision gate runs in CI.
`external_baseline_deltas.json` additionally shows where another tool found a
@@ -218,11 +218,10 @@ vulnerability OpenUltraSAST missed (or vice-versa).
## The HarnessX self-improving cycle
Each scan is a composed harness of typed processors with read/write **state
contracts** (strict mode fails the scan on a violation; warn mode records a
degradation) and a full event trace. The loop is now **closed automatically
inside the pipeline** — no agent required: every scan persists its verifier
rejections as scoped learnings and the next scan loads them and demotes those
scopes before findings are produced.
contracts**. Strict mode fails the scan on a violation; warn mode records a
degradation. The loop now runs inside the pipeline without an agent: every scan
persists verifier rejections as scoped learnings, and the next scan loads them
and demotes those scopes before findings are produced.
```
┌──────────── automatic ledger (no agent in the loop) ───────────┐
@@ -249,11 +248,11 @@ priority is demoted rather than the finding deleted, the audit trail
(`applied_calibrations.json`, the ledger) shows exactly why a surface lost
attention. Covered by `tests/test_pipeline_calibration.py`.
Beyond the automatic ranking loop, **OpenCode still steers the richer
adjustments** — prompt constraints, retrieval filters, skill routing, and
benchmark-miss triage — via `openultrasast-triage`; those `RankingCalibration`
fields are already produced and will be consumed directly once the language-aware
LLM hunters land (`standard_security_harness`).
Beyond the automatic ranking loop, `openultrasast-triage` still lets OpenCode
adjust prompt constraints, retrieval filters, skill routing, and benchmark-miss
triage. Those `RankingCalibration` fields are already produced and will be
consumed directly once the language-aware LLM hunters land
(`standard_security_harness`).
## Development
src/openultrasast/calibration.py
+1 -2
View File
@@ -9,8 +9,7 @@ from .findings import StaticFinding
from .rank import RankingScore
from .verification import VerificationResult, VerificationStatus
# Calibration may push a repeatedly-rejected scope below the natural ranking
# minimum (1.0) so accumulated false positives keep losing priority over runs.
# Repeated rejections can push a scope below the normal 1.0 ranking floor.
MIN_CALIBRATED_PRIORITY = 0.1
src/openultrasast/cli.py
+1 -2
View File
@@ -177,8 +177,7 @@ def _persist_calibration_feedback(
findings: list[StaticFinding],
verifications: list[VerificationResult],
) -> list[FalsePositiveLearning]:
# Turn this run's non-accepted verifier outcomes into scoped learnings and
# merge them into the persistent ledger so the next scan demotes those scopes.
# Store rejected or held findings as scoped learnings for the next scan.
new_learnings = learnings_from_verifications(findings, verifications)
merged = merge_false_positive_learnings(prior_learnings, new_learnings)
write_false_positive_learnings(merged, ledger_path)
src/openultrasast/findings.py
+9 -9
View File
@@ -69,8 +69,8 @@ PATTERN_RULES = (
title="Unsafe C format string needs review",
severity="high",
tags=("memory_unsafe",),
# Format function whose format argument is a bare identifier (not a string
# literal), i.e. the classic format-string sink. printf("...") is ignored.
# Flag format functions that receive a variable format string.
# Calls with string literals, such as printf("..."), are ignored.
pattern=re.compile(r"\b(printf|vprintf|vsprintf|vsnprintf|vfprintf|syslog)\s*\(\s*[A-Za-z_]"),
cwe="CWE-134",
languages=C_LANGUAGES,
@@ -144,8 +144,8 @@ PATTERN_RULES = (
title="Python SQL string composition needs review",
severity="high",
tags=("injection",),
# .execute(...) where the argument is built with %-formatting, .format(),
# string concatenation, or an f-string. Parameterised %s placeholders pass.
# Flag .execute(...) calls built with %-formatting, .format(), string
# concatenation, or an f-string. Parameterised %s placeholders pass.
pattern=re.compile(r"(\.execute)\s*\(\s*[^\n]*(?:[\"']\s*%\s*[(A-Za-z_]|\.format\s*\(|[\"']\s*\+|\bf[\"'])"),
cwe="CWE-89",
languages=PYTHON_LANGUAGES,
@@ -203,8 +203,8 @@ PATTERN_RULES = (
title="Node child_process command execution needs review",
severity="high",
tags=("syscall_entry",),
# exec/execSync invoke a shell; execFile/spawn with an argv array do not and
# are the safe alternatives, so they are intentionally excluded here.
# exec and execSync invoke a shell. execFile and spawn with an argv array
# do not, so this rule leaves them alone.
pattern=re.compile(r"\b(exec|execSync)\s*\("),
cwe="CWE-78",
languages=JS_LANGUAGES,
@@ -316,8 +316,8 @@ PATTERN_RULES = (
title="Java SQL string composition needs review",
severity="high",
tags=("injection",),
# A string literal that begins with a SQL keyword and is concatenated with
# untrusted data. Parameterised queries (placeholders, no '+') are ignored.
# Flag SQL strings that start with a statement keyword and then concatenate
# more data. Parameterised queries with placeholders are ignored.
pattern=re.compile(r"(?i)\"\s*(select|insert|update|delete)\b[^\"]*\"\s*\+"),
cwe="CWE-89",
languages=JAVA_LANGUAGES,
@@ -346,7 +346,7 @@ PATTERN_RULES = (
title="Groovy template unescaped output needs review",
severity="high",
tags=("injection",),
# Unescaped output of a bound variable; string literals are not flagged.
# Flag unescaped bound variables. String literals are ignored.
pattern=re.compile(r"(yieldUnescaped)\s+[A-Za-z_]"),
cwe="CWE-79",
languages=GROOVY_LANGUAGES,