mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Verified root cause: the pilot repos are libraries whose untrusted input is public-API parameters, not HTTP requests. Python taint sources (ruleset/semantic/python.toml) are web/CLI only; the parameter-as-source mode (_parameters_are_input) fires only for recognised entry points, and the pilot vuln regions are entry:false -- so their params are deliberately not sources. Widening that statically would reintroduce the WordPress/PMPro false-SQL-injection flood the docstring documents. So the static engine cannot safely source library APIs on its own -- which is exactly the seam for the maintainer's '2 guided by 1' directive: the LLM adjudicates which library parameter is a trust boundary, the taint engine traces that source->sink as proof (not LLM say-so), and repeated source shapes graduate to guarded static facts. First increment (pending budget): LLM-guided-source mode on the 5 pilot regions, measuring source naming, traced finding, catch lift vs regex 1/5, and false-alarm on ordinary changes; staged. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
ae22136c08
commit
90e0098a94
@@ -267,3 +267,34 @@ catches over the regex floor.**
|
||||
engine 0/5. The static tiers (regex + Joern dataflow) do not detect these unseen vulns; the decisive untested lever
|
||||
is the **LLM adjudication tier**. Next: an **LLM-tier pilot (L1/L2)** on these 5 repos — does enabling the model
|
||||
produce vuln-family findings regex+dataflow miss, and at what $/latency.
|
||||
|
||||
## ROOT CAUSE of engine 0/5 + the "#2 guided by #1" workstream (2026-10-12)
|
||||
|
||||
**Why the taint tier traced no flow (verified in code + evidence):** the pilot repos are **libraries**, whose
|
||||
untrusted input arrives as **public-API function parameters**, not HTTP requests. The Python taint **source**
|
||||
vocabulary (`ruleset/semantic/python.toml`) is entirely web/CLI: Flask/Django `request.*`, `sys.argv`, `sys.stdin`,
|
||||
`input(`. There is no source for a library's own API parameter. The engine *does* have a parameter-as-source mode
|
||||
(`_parameters_are_input`, `scan.py`), but it fires **only for recognised entry points** (`_is_entry_point` and
|
||||
`entry_kind != "function"`) — the pilot vuln regions carry `entry:false`, so their parameters are **deliberately
|
||||
not** sources. That restriction is intentional: treating every function parameter as a source caused a flood of
|
||||
false SQL-injections on WordPress/PMPro (documented in the `_parameters_are_input` docstring). So **the static
|
||||
engine cannot safely widen its sources to library APIs without reintroducing that false-alarm explosion** —
|
||||
`banks` path: sink `path.open` found, `source_local/near=false`; `graphiti`/`bedrock` injection: no source.
|
||||
|
||||
**This is the precise seam for the maintainer's "#2 guided by #1" directive.** The static taint cannot *safely*
|
||||
decide which library parameters are trust boundaries; the **LLM can** reason about which public-API surface is
|
||||
attacker-reachable (intent, not pattern). So:
|
||||
- **#1 (LLM) guides:** on a changed library region the engine ranks but cannot source, the LLM adjudicates whether a
|
||||
parameter is an untrusted trust boundary and names the source→sink it believes exists (search-with-proof).
|
||||
- **#2 (taint) traces + proves:** the engine then runs the flow from the LLM-named source to the known sink,
|
||||
confirming (or refuting) with a traced path — a proof, not the LLM's say-so. Net: catch without the static FP
|
||||
blowup, and every catch carries a dataflow witness.
|
||||
- **Feedback:** where the LLM repeatedly names the same source shape (e.g. a registry-key → path family), promote it
|
||||
to a static fact in `python.toml` guarded by the entry/shape conditions that keep WordPress quiet.
|
||||
|
||||
**Proposed first increment (pending budget confirmation — LLM calls cost money):** wire the disabled LLM tier on the
|
||||
5 pilot regions in *LLM-guided-source* mode and measure: (a) does it name the right trust-boundary source per vuln,
|
||||
(b) does the taint engine then trace source→sink and emit a vuln-family finding, (c) catch lift over regex (1/5) and
|
||||
false-alarm cost on the ordinary changes. Stage it (5–10 sizing → ~10–20 → full on lift). This makes "#2 guided by
|
||||
#1" the concrete G2 detection workstream; budget/change-scoping and change-anchoring stay deprioritised until it
|
||||
yields findings.
|
||||
|
||||
@@ -123,9 +123,15 @@ against a realistic pre-push window (≤120 s target, 300 s hard):
|
||||
injection/deser) on all 5 — only config_secrets. The tier1/score2.0 "sink" was the RANKER's static candidate,
|
||||
not a taint finding. `ambiguous_line_correspondence` is a downstream red herring (nothing to anchor), so the
|
||||
region-level witness fix (old "Option A") was **refuted before implementation**. The **LLM tier was disabled**
|
||||
(model_calls=0). **Reprioritised levers:** #1 the **LLM adjudication tier** (disabled) — now primary: run the
|
||||
LLM pilot L1/L2, staged (costs money). #2 taint source/sink coverage + region ranking. #3 change-anchoring
|
||||
(only matters once tiers produce vuln-family findings — deprioritised).
|
||||
(model_calls=0). **ROOT CAUSE:** pilot repos are LIBRARIES whose untrusted input is public-API parameters;
|
||||
Python taint sources are web/CLI only, and the param-as-source mode fires only for entry points (vuln regions
|
||||
are `entry:false`) — widening it statically would reintroduce the WordPress/PMPro false-SQL-injection flood.
|
||||
**Workstream = maintainer's "#2 guided by #1":** the LLM adjudicates which library parameter is a trust boundary
|
||||
(static rules can't without FPs), the taint engine then traces that source→sink as proof, and repeated source
|
||||
shapes get promoted to guarded static facts. First increment (PENDING BUDGET): wire the disabled LLM tier on the
|
||||
5 pilot regions in LLM-guided-source mode; measure source naming, traced finding, catch lift vs regex 1/5, and
|
||||
false-alarm on ordinary changes; stage 5–10 → ~10–20 → full on lift. #3 change-anchoring deprioritised until
|
||||
tiers yield findings.
|
||||
- LLM: L1 model latency and calls per push; L2 the recall LIFT of the LLM tier over regex+dataflow on the pool.
|
||||
- Deliverable: a barrier table (budget × tier combo → fits? measured recall/FP) → the minimal releasable config.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user