feat(model-grounded-detection): group 3, the evidence ladder and the judge

3.1 adds StaticFinding.rung beside evidence_level rather than renaming it. The
existing evidence_level carries `static_corroboration`, emitted by the flat IR
overlay -- a path Req 4.4 demotes to the suspicion band. Restating that as
model_corroborated would claim a CPG arbitrated something it never saw, and
rewriting every legacy finding to suspicion would be a product-visible
regression nobody asked for. So `rung` defaults to suspicion and is raised only
by ladder.at_rung(finding, verdict), which is the structural form of Req 5.4:
nothing rises above suspicion without a Verdict behind it. The witness travels
with the rung, because a rung without its evidence is just a louder assertion.
It reaches markdown, JSON and SARIF.

3.2 is the judge, and it is defined as much by what it lacks. Three exhaustive
outcomes: the model entails (report it, never call the model), the model
contradicts (drop the claim, record why), the model cannot decide (report
suspicion). The LLM gets one bounded typed question with tools=[] so it cannot
search for the site, asked only about the residual the graph cannot settle --
whether a sanitizer on a REAL flow suffices. Its answer can only confirm a flow
the CPG already found; it can never conjure one.

Req 8.5 is asserted structurally: a test parses judge.py and fails on any AST
name like k_runs/majority/vote/average/acceptance/floor/budget, any for/while
loop, or an import from the deleted round machinery. That test first matched
the word "votes" in the module's own docstring, which is why it now inspects
code rather than text.

Verified live against a real CPG, not only scripted rows:
  vuln   rung=model_entailed   request.args["name"] -> db.execute(...)
  fixed  rung=suspicion        no source reaches this sink
  model calls made: 0

Both sides arbitrated by the graph with the LLM never consulted -- the thing
the prior architecture could not do at any price.

678 passing, mypy/ruff clean, gates byte-identical, zero orphaned modules.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-08 13:49:55 +02:00
co-authored by Claude Opus 5
parent 7387d2f190
commit b8f8f727c6
9 changed files with 471 additions and 7 deletions
+23 -1
View File
@@ -18,6 +18,12 @@ from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from typing import TYPE_CHECKING, TypeVar
if TYPE_CHECKING:
from ..findings import StaticFinding
_Finding = TypeVar("_Finding", bound="StaticFinding")
class Rung(StrEnum):
@@ -45,4 +51,20 @@ class Verdict:
contradiction: str = "" # why an LLM claim was dropped, when it was
__all__ = ["Rung", "Verdict", "at_or_above"]
def at_rung(finding: _Finding, verdict: Verdict | None) -> _Finding:
"""Return ``finding`` carrying ``verdict``'s rung and witness, or unchanged when there is no verdict.
This is the only way a finding rises above ``suspicion`` (Req 5.4). The witness travels with the rung
because a rung without the evidence that justifies it is just a louder assertion.
"""
from dataclasses import replace
if verdict is None or verdict.rung is Rung.SUSPICION:
return finding
rationale = finding.rationale
if verdict.witness and verdict.witness not in rationale:
rationale = f"{rationale} [{verdict.rung.value}: {verdict.witness}]".strip()
return replace(finding, rung=verdict.rung.value, rationale=rationale)
__all__ = ["Rung", "Verdict", "at_or_above", "at_rung"]