mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-20 13:39:27 +00:00
docs: plain-English pass over the active specs
Ran the humanizer rules over 79 prose files in the active specs (brief, design, requirements, research and narrative docs). Removed em dashes, significance inflation, forced rule-of-three, bold-label list items and filler. Skipped tasks.md checklists and the retired specs (harnessx-*, openrouter-sast-harness, unseen-repo-evaluation). Every number, path, command, hash and identifier was preserved: checked each file against HEAD for drift in numbers, inline code and code fences, zero drift across all 79 files. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
b08f372022
commit
39a1873e5b
@@ -2,7 +2,7 @@
|
||||
|
||||
## Problem
|
||||
|
||||
OpenUltraSAST already has three scan stages (`quick` inventory, `standard` map, `deep` regression), but **only the first stage actually decides**. Pattern rules propose sinks; the map ranks files; the sandbox can run recipes. Nothing in the middle answers “does this sink receive attacker-controlled data, or is it a Juliet-style good twin / dead assignment / hardcoded command?”
|
||||
OpenUltraSAST already has three scan stages (`quick` inventory, `standard` map, `deep` regression), but **only the first stage actually decides**. Pattern rules propose sinks; the map ranks files; the sandbox can run recipes. Nothing in the middle answers "does this sink receive attacker-controlled data, or is it a Juliet-style good twin / dead assignment / hardcoded command?"
|
||||
|
||||
Calibration corpora (cheat-sheet fixtures, OWASP Benchmark true/false files, Juliet `bad`/`good`, CVE pairs) are **nice**: one CWE per file, named sinks, labeled counterparts. Real working trees are not: mixed languages, generated code, unparseable files, framework wrappers, comments, one-hop assignments (`sql = f"…{x}"; cur.execute(sql)`), and no ground-truth labels.
|
||||
|
||||
@@ -12,17 +12,17 @@ Measured honesty on the labeled SAST slice is about **+9% Youden**. Widening reg
|
||||
|
||||
Keep three roles, never collapse them:
|
||||
|
||||
1. **Propose** — pattern rules + optional SARIF. Stage 1. Does not decide worth-fixing.
|
||||
2. **Adjudicate** — tree-sitter IR (multi-language) + taint facts; optional Joern CPG sidecar; Python stdlib parse only as last-resort fallback. Promote, demote, or **unadjudicated**.
|
||||
3. **Budget** — ranker + **mechanism memory** (JSONL + OpenRouter embeddings, hard CWE/language/tag filters) **before** prove. Not a verdict. Not TF-IDF. Not raw file-slice embeddings.
|
||||
4. **Prove** — sandbox only on promoted, reachable, budgeted candidates.
|
||||
1. **Propose**, pattern rules + optional SARIF. Stage 1. Does not decide worth-fixing.
|
||||
2. **Adjudicate**, tree-sitter IR (multi-language) + taint facts; optional Joern CPG sidecar; Python stdlib parse only as last-resort fallback. Promote, demote, or **unadjudicated**.
|
||||
3. **Budget**, ranker + **mechanism memory** (JSONL + OpenRouter embeddings, hard CWE/language/tag filters) **before** prove. It is a budget rather than a verdict; it uses neither TF-IDF nor raw file-slice embeddings.
|
||||
4. **Prove**, sandbox only on promoted, reachable, budgeted candidates.
|
||||
|
||||
The primary in-scope target is a **messy working tree the operator is writing**, not a labeled benchmark. Known-vuln corpora remain calibration scoreboards (smoke 90/10, pair Youden, GitHub silent-on-fix). They do not define success on unlabeled code.
|
||||
|
||||
## Scope
|
||||
|
||||
- Semantic overlay as MAP-stage adjudication; **tree-sitter is the prio engine** (CLI/API, fail-closed). Joern optional. Embeddings tune prove order.
|
||||
- Explicit `unadjudicated` when parse or flow fails — do not treat silence as safe.
|
||||
- Explicit `unadjudicated` when parse or flow fails. Do not treat silence as safe.
|
||||
- Sandbox prove only after promote.
|
||||
- Pair eval scores the overlay on the sast slice; smoke gate stays inventory-only.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user