mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 20:49:28 +00:00
docs(model-grounded-detection): the arbiter is a program model, not a sandbox
Maintainer correction: execution isn't possible for all SAST targets, and the bugs this project owns -- IDOR, missing authz, injection -- don't crash anything. Execution is the easy arbiter only for memory-safety-in-C, which is not our world. What execution provided was an independent, deterministic arbiter of whether a claim is real -- not the same LLM that made it, not a noisy vote. That arbiter can be built statically, from the code, as a program model the claim is checked against. LLM proposes; the model disposes. Same architecture, same repair of the root cause, no runnable target required. The evidence ladder becomes static and deterministic: suspicion -> model_corroborated (claim consistent with resolved dataflow / a guard its siblings carry is absent) -> model_entailed (the model alone establishes it) -> execution_confirmed (deferred tier, C memory-safety only). model_corroborated and above are deterministic -- run the checker twice, same verdict -- which removes the run-to-run noise the same way a crash would, with no K-runs and no target to build. Our IR candidate enumerator (96.6% ceiling), binding chains and obligation checker stop being prompt context and become the arbiter itself. The course: promote those assets from inputs to arbiter, delete the noise layer, make the LLM a proposer the model audits, keep the corpus as the model's calibration set (the fix diff is what the model should see), and defer execution to Clearwing for the memory-safety classes only. Build the arbiter we can build from code; adopt the one we cannot, when a class demands it. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
81a63cf5ac
commit
091303c7c0
@@ -27,5 +27,5 @@
|
||||
"harnessx-self-improving-rulesets"
|
||||
],
|
||||
"approval_note": "2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase'); 2026-09-06 design approved by the maintainer in session ('approve the design and generate the tasks'); 2026-09-06 tasks approved by the maintainer in session ('approve the tasks and go on with the implementation'); 2026-09-06 requirements amended by the maintainer in session ('amend the requirements to untrusted_destination and note the flat taxonomy'): Req 1.1 names untrusted destination instead of server-side request forgery, and Req 2.7 and 3.7 require every agreement and family figure to state that hierarchical partial credit could not apply while the taxonomy is flat; 2026-09-06 Req 6.1 amended by the maintainer in session ('drop the skills set from the requirement'): a detector configuration has no skills set, because the skill router belongs to the tool's earlier shape and a family's prompt, checklist, tools and counterexamples already carry what a round evolves; 2026-09-06 design amended during task 4.1 (recorded in design.md): a recipe may name cross-file registration documents, because connexion registers by operationId in an OpenAPI file and express in a router module, so Req 10.2 is unreachable from a single-file excerpt for those rows; 2026-09-06 task 5.3 amended by the maintainer in session (asked and answered): the first evolve rounds target `injection` rather than `access_control`, because under the split and review-tier rules access_control has one train teacher and four holdout pairs and teaches zero obligation shapes, while injection has twelve teachers and ten holdout pairs; 2026-09-08 SUPERSEDED by execution-grounded-detection: this session's measurements showed the noise architecture (K-runs, per-family floor, regression budget, sign test, evolve loop) is a statistical treatment for a structural problem, and that the field (Big Sleep, AIxCC finals, Clearwing) grounds detection in execution rather than in scoring against a labeled corpus. The taxonomy, the IR candidate enumerator (96.6% ceiling) and the obligation checker survive as the web/logic detection path to contribute to a Clearwing-based substrate.",
|
||||
"superseded_by": "execution-grounded-detection"
|
||||
"superseded_by": "model-grounded-detection"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user