mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 11:49:28 +00:00
feat(model-grounded-detection): wire the detector -- the enumerator proposes, the model disposes
Everything before this built an ARBITER: something that judges a flow taint
reachability had already found unaided. Nothing proposed to it. That is why its
recall was never comparable to a detector's, and why comparing 41.3% entailment
against the LLM hunter's 76.7% pair-correct was comparing two different kinds
of thing. model/judge.py and enumerate_candidates were called by no production
code at all; every number so far came from a measurement harness.
The wiring turns on a distinction judge() alone conflated:
the model CONTRADICTS it has coverage at this site and says no -- a
sanitizer on the path, or a declared-safe sink shape.
The claim is dropped (Req 8.2).
the model CANNOT DECIDE no modelled sink or source reaches the site at all.
The model is SILENT, not negative, so the claim is
reported at `suspicion` (Req 8.4).
Treating silence as refutation is the tempting mistake and would be wrong here:
23 of the 39 misses in the entailment ceiling were sinks simply absent from the
fact tables, so "no flow" usually means the model cannot see it rather than
that nothing is there. The rung carries that uncertainty on the finding's face
instead of a confidence score nobody can act on.
An entailed site still costs zero model calls. The candidate budget is bounded
so a dense region cannot become an unbounded spend.
Smoke-tested end to end against a live endpoint: 2 pairs, detection 100%,
$0.0024. The full vibe-py measurement -- the first like-for-like comparison
with the hunter's 76.7% pair-correct and 12.3% leak -- is running.
727 passing, mypy/ruff clean, zero orphaned modules.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
1af3db37f6
commit
cdedb372dd
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"counts": {
|
||||
"load_bearing": 85,
|
||||
"load_bearing": 86,
|
||||
"standalone_capability": 7
|
||||
},
|
||||
"generated": "2026-09-08 (regenerated after group 6: the deferred execution tier)",
|
||||
"generated": "2026-09-08 (regenerated after the detector pipeline was wired)",
|
||||
"kept_with_a_lever_stripped": {
|
||||
"improve/evolve.py + improve/validator.py": "the design's Removed list named these whole, but each is a mix: the mechanism lever (removed) and the deterministic rule-status improve loop that `ousast improve` still runs from benchmark evidence, which has no LLM in it and is not what this feature supersedes. Deleting the files would have taken a retained capability with them."
|
||||
},
|
||||
"modules": 92,
|
||||
"modules": 93,
|
||||
"note": "Classification of every source module after the group-1 subtraction. load_bearing = reachable from a gate, the CLI, or named by a surviving requirement; standalone_capability = its own entry point or protocol, deliberately off the scan path; orphaned = no importer, no entry point, no surviving requirement -- removed, never carried.",
|
||||
"orphaned": [],
|
||||
"removed_in_this_pass": {
|
||||
@@ -131,6 +131,7 @@
|
||||
"model.config_value",
|
||||
"model.dominance",
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "cpg",
|
||||
@@ -142,6 +143,7 @@
|
||||
"model.config_value",
|
||||
"model.dominance",
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "cpg.backend",
|
||||
@@ -333,6 +335,7 @@
|
||||
"model.dominance",
|
||||
"model.execution",
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"model.specs",
|
||||
"model.taint"
|
||||
],
|
||||
@@ -396,11 +399,18 @@
|
||||
"model.dominance",
|
||||
"model.execution",
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "model.ladder",
|
||||
"reason": "model-grounded-detection Req 5: the evidence ladder every finding carries"
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [],
|
||||
"module": "model.pipeline",
|
||||
"reason": "model-grounded-detection Req 8: the enumerator proposes, the LLM answers, the model disposes"
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [],
|
||||
@@ -413,6 +423,7 @@
|
||||
"model.config_value",
|
||||
"model.dominance",
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"model.taint"
|
||||
],
|
||||
"module": "model.specs",
|
||||
@@ -421,7 +432,8 @@
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"model.judge"
|
||||
"model.judge",
|
||||
"model.pipeline"
|
||||
],
|
||||
"module": "model.taint",
|
||||
"reason": "model-grounded-detection Req 6.1: taint reachability over the CPG, the flow-family arbiter"
|
||||
@@ -556,6 +568,7 @@
|
||||
"importers": [
|
||||
"harness",
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"reports",
|
||||
"semantic.obligations.intent"
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user