mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
openultrasast-scan: evidence ladder as verification.py defines it, with the levels nothing produces today; --mode prerequisites, --fail-on, recorded degradations, run artifacts, ousast mcp; retired [evidence] threshold language removed. openultrasast-triage: reasons verified against calibration.py; deterministic fusion; adjudications as labels, plane verdicts as features, rule changes through improve or a registered experiment; fix-audit merged as a fix-and-re-scan section. kiro: spec-quick and spec-batch removed; validate-gap and validate-design merged into spec-design (--gap, --review; the review gate stays); steering-custom merged into steering; core steering names overview.md, roadmap.md, safety-net.md; instrument rule in debug and verify-completion; repo gates named in validate-impl. AGENTS.md, README skills paragraph and tests/test_agent_skills.py (.agents/skills, two product skills, drift tests) updated. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
3.7 KiB
3.7 KiB
name, description
| name | description |
|---|---|
| openultrasast-triage | OpenUltraSAST finding triage - false-positive adjudication with the fixed reason taxonomy, the evidence ladder, what a verdict may feed (labels, never rule edits), and fixing then re-scanning. Use when reviewing findings, reducing false positives, deciding whether evidence is enough, or auditing a fix for a finding. |
OpenUltraSAST Triage
Use this skill when triaging OpenUltraSAST findings or auditing a fix for one.
Triage Loop
- Identify the finding: id, path, rule, CWE and policy severity, confidence, evidence level and
verification status (
findings.json,verification.jsonin the run directory). - Separate deterministic evidence (rule match, graph entailment, attached reachability) from model assumptions.
- Check attacker control, reachability, sanitizer behaviour, duplicates, impact, and contradictory evidence.
- If rejected or unresolved, record the false-positive reason and its scope.
- Prefer scoped demotion over deleting a vulnerability class globally: a rejection in
auth/demotesauth/..., never the class (calibration.py). - Record the adjudication. Do not edit a rule because of one finding (see below).
False-Positive Reasons
The taxonomy is FalsePositiveReason in src/openultrasast/calibration.py; use these values exactly:
unreachable_pathmissing_attacker_controlsanitizer_disprovedstatic_rule_mismatchincorrect_model_assumptionduplicateinsufficient_impactunsupportedcontradictedunverified
Decision Standard
If evidence is uncertain, mark the finding needs_evidence and name the next evidence instead of
escalating severity. Fusion (fusion.json, standard mode) is deterministic: it adjudicates between
the quick-rule and MAP panels by fixed triggers and is not a second opinion from a model. Independent
model agreement exists only in the plane's verify passes a and b, with pass c on disputes, and the
agree step; those verdicts arrive as features of the decision engine, never as a verification
status.
What an Adjudication Feeds
- Recorded adjudications (a per-finding verdict with a reason) become decision-engine labels through
the fail-closed source list
src/openultrasast/learn/sources.toml([[source]] id = "adjudications"). Aggregate counts label nothing; only per-finding verdicts do. - Plane verdicts (
agreed,rejected,disputed) are features, never labels. "Rejected by the plane" is not a negative. - Rule changes go through the ledger:
ousast improve(the validator and the detection gate, holdout pairs per round) or a registered experiment (ousast learn experiment register|units|run|analyse). Never a hand edit ofrules.tomlper miss or per false positive.
Fix and Re-scan
- Intake: restate the finding id, evidence level, affected path, and the acceptance criterion.
- Plan: the smallest defensive change that addresses the root cause.
- Implement: modify only the files the finding requires.
- Adversarial review: look for bypasses, regressions, false assumptions, and unrelated edits. Use a differential review for changes in auth, crypto, config, parsers, and sandbox behaviour.
- Reconcile: fix each review finding or document why it does not apply.
- Verify: a re-scan of the checkout in the same mode, or a replay of the change with
ousast pre-push --base <before> --head <after> --artifact <file> <repo>, shows the finding gone and no new findings. That re-scan is the evidence. Nothing in the tool recordspatch_validated; do not claim it.
Guardrails:
- Do not rewrite modules when a local fix is enough.
- Do not change public APIs unless the vulnerability requires it.
- Do not call a finding fixed from model reasoning alone; the re-scan or replay decides.