marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 19:59:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(model-grounded-detection): the dominance question contradicted its own evidence

Browse Source 
vampi-users-update-password scored both_silent while the arbiter returned
model_corroborated on the vulnerable side and nothing on the fixed one -- so the
finding existed and the pipeline threw it away.

The residual question was written for the ENTAILED case, but entailed returns
before the judge is ever consulted, so the question is only ever asked for
CORROBORATED. It asserted "in a file where sibling handlers are guarded" and
asked about "the guard its siblings have", printed directly above a witness
reading "and no guarded sibling" -- because corroborated means precisely that
no sibling is guarded. The judge was handed a prompt contradicting its own
evidence line and reasonably answered no.

The question now describes what corroborated actually is: an obligated
operation no guard governs, with no sibling evidence either way, so the model
cannot tell from consistency alone whether the guard is missing or the module
is deliberately public. That is the real open question and the one worth asking.

Verified end to end against a live endpoint: vuln model_corroborated, fixed
silent, the pair splits.

The taint and config questions were checked for the same defect and are
correct -- each describes the corroborated case it is actually asked about.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-08 19:22:55 +02:00
co-authored by Claude Opus 5
parent c74e101d10
commit 3fe7b9ded7
2 changed files with 27 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/openultrasast/model/pipeline.py
+9 -4
View File
@@ -142,13 +142,18 @@ def residual_question(spec: TaintSpec | DominanceSpec | ConfigSpec, witness: str
was lost this way.
"""
if isinstance(spec, DominanceSpec):
# Only the CORROBORATED case reaches this question -- an entailed verdict returns before the judge is
# consulted. Corroborated means precisely that no sibling is guarded, so there is no asymmetry to
# appeal to and claiming one contradicts the witness printed directly beneath it.
return (
f"You are judging ONE finding a static model already made. Do not look for other issues.\n\n"
f"A dominance analysis of {spec.language} code found an obligated operation that NO discharging\n"
f"guard governs, in a file where sibling handlers are guarded:\n\n"
f"A dominance analysis of {spec.language} code found an obligated operation on a protected\n"
f"resource that NO discharging guard governs on any path:\n\n"
f" {witness}\n function: {function or '?'}\n\n"
f"The model has established the asymmetry. The open question is whether this operation genuinely\n"
f"needs the guard its siblings have -- or is legitimately public.\n\n"
f"No sibling handler in this file is guarded either, so the model cannot tell from consistency\n"
f"alone whether the guard is missing or the whole module is deliberately public. That is the\n"
f"open question: does this operation require an authorization check, or is it legitimately\n"
f"public?\n\n"
'Answer ONLY with JSON: {"vulnerable": true|false, "why": "<one sentence>"}'
)
if isinstance(spec, ConfigSpec):