mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 11:49:28 +00:00
fix(model-grounded-detection): the dominance question contradicted its own evidence
vampi-users-update-password scored both_silent while the arbiter returned model_corroborated on the vulnerable side and nothing on the fixed one -- so the finding existed and the pipeline threw it away. The residual question was written for the ENTAILED case, but entailed returns before the judge is ever consulted, so the question is only ever asked for CORROBORATED. It asserted "in a file where sibling handlers are guarded" and asked about "the guard its siblings have", printed directly above a witness reading "and no guarded sibling" -- because corroborated means precisely that no sibling is guarded. The judge was handed a prompt contradicting its own evidence line and reasonably answered no. The question now describes what corroborated actually is: an obligated operation no guard governs, with no sibling evidence either way, so the model cannot tell from consistency alone whether the guard is missing or the module is deliberately public. That is the real open question and the one worth asking. Verified end to end against a live endpoint: vuln model_corroborated, fixed silent, the pair splits. The taint and config questions were checked for the same defect and are correct -- each describes the corroborated case it is actually asked about. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
c74e101d10
commit
3fe7b9ded7
@@ -195,3 +195,21 @@ def test_a_corroborated_absence_finding_survives_a_judge_that_agrees() -> None:
|
||||
{"operation": "q.filter_by(id=n)", "opLine": "3", "opMethod": "safe", "dominatingGuards": ["current_user"]}]
|
||||
findings = scan_region(_cpg(rows), _dominance_spec(), function="leaky", client=_Client(), model="m", candidates=())
|
||||
assert len(findings) == 1 and findings[0].rung is Rung.ENTAILED
|
||||
|
||||
|
||||
def test_the_dominance_question_describes_the_evidence_the_model_actually_has() -> None:
|
||||
"""The residual question is only ever asked for CORROBORATED, never ENTAILED -- entailed returns early.
|
||||
|
||||
It was written for the entailed case, so it asserted "in a file where sibling handlers are guarded" and
|
||||
asked about "the guard its siblings have", directly above a witness reading "and no guarded sibling".
|
||||
The judge, handed a prompt contradicting its own evidence line, answered no and the finding was dropped:
|
||||
vampi-users-update-password scored both_silent while the arbiter returned corroborated on the vulnerable
|
||||
side and nothing on the fixed one.
|
||||
"""
|
||||
from openultrasast.model.pipeline import residual_question
|
||||
|
||||
witness = "update_password line 16: obligated operation with no discharging guard, and no guarded sibling"
|
||||
q = residual_question(_dominance_spec(), witness, "update_password")
|
||||
assert "siblings have" not in q, "the corroborated case has no guarded siblings to appeal to"
|
||||
assert "sibling handlers are guarded" not in q
|
||||
assert "guard" in q.lower() and "public" in q.lower(), "it must still ask whether the operation needs a guard"
|
||||
|
||||
Reference in New Issue
Block a user