mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(model-grounded-detection): group 3, the evidence ladder and the judge
3.1 adds StaticFinding.rung beside evidence_level rather than renaming it. The existing evidence_level carries `static_corroboration`, emitted by the flat IR overlay -- a path Req 4.4 demotes to the suspicion band. Restating that as model_corroborated would claim a CPG arbitrated something it never saw, and rewriting every legacy finding to suspicion would be a product-visible regression nobody asked for. So `rung` defaults to suspicion and is raised only by ladder.at_rung(finding, verdict), which is the structural form of Req 5.4: nothing rises above suspicion without a Verdict behind it. The witness travels with the rung, because a rung without its evidence is just a louder assertion. It reaches markdown, JSON and SARIF. 3.2 is the judge, and it is defined as much by what it lacks. Three exhaustive outcomes: the model entails (report it, never call the model), the model contradicts (drop the claim, record why), the model cannot decide (report suspicion). The LLM gets one bounded typed question with tools=[] so it cannot search for the site, asked only about the residual the graph cannot settle -- whether a sanitizer on a REAL flow suffices. Its answer can only confirm a flow the CPG already found; it can never conjure one. Req 8.5 is asserted structurally: a test parses judge.py and fails on any AST name like k_runs/majority/vote/average/acceptance/floor/budget, any for/while loop, or an import from the deleted round machinery. That test first matched the word "votes" in the module's own docstring, which is why it now inspects code rather than text. Verified live against a real CPG, not only scripted rows: vuln rung=model_entailed request.args["name"] -> db.execute(...) fixed rung=suspicion no source reaches this sink model calls made: 0 Both sides arbitrated by the graph with the LLM never consulted -- the thing the prior architecture could not do at any price. 678 passing, mypy/ruff clean, gates byte-identical, zero orphaned modules. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
7387d2f190
commit
b8f8f727c6
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"counts": {
|
||||
"load_bearing": 79,
|
||||
"load_bearing": 80,
|
||||
"standalone_capability": 7
|
||||
},
|
||||
"generated": "2026-09-08 (regenerated after group 2 added the cpg/ seam)",
|
||||
"generated": "2026-09-08 (regenerated after group 3 added the ladder and the judge)",
|
||||
"kept_with_a_lever_stripped": {
|
||||
"improve/evolve.py + improve/validator.py": "the design's Removed list named these whole, but each is a mix: the mechanism lever (removed) and the deterministic rule-status improve loop that `ousast improve` still runs from benchmark evidence, which has no LLM in it and is not what this feature supersedes. Deleting the files would have taken a retained capability with them."
|
||||
},
|
||||
"modules": 86,
|
||||
"modules": 87,
|
||||
"note": "Classification of every source module after the group-1 subtraction. load_bearing = reachable from a gate, the CLI, or named by a surviving requirement; standalone_capability = its own entry point or protocol, deliberately off the scan path; orphaned = no importer, no entry point, no surviving requirement -- removed, never carried.",
|
||||
"orphaned": [],
|
||||
"removed_in_this_pass": {
|
||||
@@ -128,6 +128,7 @@
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"cpg.backend",
|
||||
"model.judge",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "cpg",
|
||||
@@ -136,6 +137,7 @@
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"model.judge",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "cpg.backend",
|
||||
@@ -165,6 +167,7 @@
|
||||
"hunter_tools",
|
||||
"improve.evolve",
|
||||
"map_gate",
|
||||
"model.ladder",
|
||||
"pairs",
|
||||
"rank",
|
||||
"regress.candidate",
|
||||
@@ -320,6 +323,8 @@
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"cli",
|
||||
"model.candidates",
|
||||
"model.judge",
|
||||
"model.specs",
|
||||
"model.taint"
|
||||
],
|
||||
@@ -346,9 +351,16 @@
|
||||
"module": "model.endpoint",
|
||||
"reason": "model-grounded-detection Req 8: the judge client, wired by model/judge.py in group 3"
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [],
|
||||
"module": "model.judge",
|
||||
"reason": "model-grounded-detection Req 8: one bounded question, checked against the model"
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"model.judge",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "model.ladder",
|
||||
@@ -357,6 +369,7 @@
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"model.judge",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "model.specs",
|
||||
@@ -364,7 +377,9 @@
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [],
|
||||
"importers": [
|
||||
"model.judge"
|
||||
],
|
||||
"module": "model.taint",
|
||||
"reason": "model-grounded-detection Req 6.1: taint reachability over the CPG, the flow-family arbiter"
|
||||
},
|
||||
@@ -372,6 +387,7 @@
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"cli",
|
||||
"model.candidates",
|
||||
"model.specs"
|
||||
],
|
||||
"module": "model.taxonomy",
|
||||
@@ -496,6 +512,7 @@
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"harness",
|
||||
"model.judge",
|
||||
"reports",
|
||||
"semantic.obligations.intent"
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user