mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 11:49:28 +00:00
feat(model-grounded-detection): parameter sources, so the measurement is like-for-like
Only 11 of the 50 injection pairs contain a framework source token at all. The
other 39 receive untrusted input as a FUNCTION PARAMETER, because SVEN-style
pairs are function-level extractions:
def add_user(username, password):
os.system("useradd ... -p " + encPass + " " + username)
A framework-anchored source model finds nothing there, which caps entailment on
this slice near 22% by construction -- a property of the source model, not of
the engine. More to the point, the flat-IR baseline this feature is measured
against already counted these (source_kinds: ["parameter"]), so a run without
them is not comparable to the 2.2% it is supposed to beat.
The flag is opt-in and off by default, because the two settings are right in
different places: in a function-level pair the function boundary IS the trust
boundary, while across a whole repository most parameters carry internal
values and treating them all as untrusted is not sound. The corpus harness
opts in; a repository scan does not.
On the pair that exposed this the model now resolves both flows, including one
through an intermediate binding:
username -> os.system(...) 2 steps
password -> crypt.crypt(...) -> encPass -> ... 7 steps
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
1765e51da1
commit
e127e430ed
@@ -182,3 +182,25 @@ def test_a_row_without_shape_fields_is_judged_on_the_flow_alone() -> None:
|
||||
"sanitized": False, "length": 3}]
|
||||
answer = verdict(_cpg(rows), _shape_spec(), function="run")
|
||||
assert answer is not None and answer.rung is Rung.ENTAILED
|
||||
|
||||
|
||||
def test_parameter_sources_are_opt_in_and_reach_the_query() -> None:
|
||||
"""A function-level pair's parameters are its trust boundary; a whole repository's are not.
|
||||
|
||||
39 of the 50 injection pairs carry no framework source token at all — their untrusted input arrives as a
|
||||
function parameter — and the flat-IR baseline this feature is measured against counted those as sources.
|
||||
So the flag must exist and must default to off.
|
||||
"""
|
||||
from openultrasast.cpg.backend import CpgResult
|
||||
from openultrasast.model.taint import verdict
|
||||
|
||||
seen: dict[str, object] = {}
|
||||
|
||||
def run(query: str, params: dict[str, object]) -> object:
|
||||
seen.update(params)
|
||||
return []
|
||||
|
||||
verdict(CpgResult(cpg_path=Path("c.bin"), run=run), _spec(), function="run")
|
||||
assert seen["parameterSources"] == "false", "treating every parameter as untrusted is not the default"
|
||||
verdict(CpgResult(cpg_path=Path("c.bin"), run=run), _spec(), function="run", parameter_sources=True)
|
||||
assert seen["parameterSources"] == "true"
|
||||
|
||||
Reference in New Issue
Block a user