mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 11:49:28 +00:00
Marius CiepluchandClaude Fable 5.1
bfc3d806ba
Read against today's measurements the brief had two errors and one gap. They are corrected in place and kept on the record, because the corrections are themselves results. Phase 1's gate was wrong. Pruning removes 53% of requests, and 1,177 x 6.63s is still 2.2 hours -- it halves the problem and does not finish the scan. Saying otherwise would send someone looking for a bug in a phase that did exactly what it can. Ranking's job is concentration, not inclusion. CVE-2023-23488's region is at position 390, inside the budget, examined. It goes unreported because 500 regions are unaffordable at 6.63s per family per region, not because it was left out. The metric is therefore budget_at_recall -- the smallest K holding every pinned CVE -- and phase 2's gate is that K below 50. The gap: the heuristics were named, not defined, and an undefined heuristic cannot be checked for language leakage. Now defined as an evidence vector over (region, family) pairs whose every field is stated in terms of fact-table categories -- source, sink, sanitizer, dispatch, access -- so a heuristic that mentions `$wpdb` has by construction left the vector. The architectural decision that makes it agnostic rather than merely neutral-in-intent: the vector is a CPG query WITHOUT dataflow, a sibling of taint.sc that never calls reachableByFlows. Tiering is the CPG query without dataflow; arbitration is the query with it. Same graph, same facts, two costs. The graph normalises syntax across frontends, so "this argument is a literal" means the same thing in PHP, Python and C. Five tiers as predicates over the vector. Only tier 0 excludes, and its predicate has a proof: a family with no sink in reach cannot produce a flow in any language under any dataflow model. Its scope must match the arbiter's callDepth or it is not exact and must not be called tier 0. Everything above orders, where being wrong costs position, which regions_truncated reports. Where the model helps is narrower than "ranking": ambiguous cleansing (esc_sql two statements earlier into a variable), unfamiliar sink shapes as an ordering opinion never a fact, and within-tier ties. Where it must not be allowed: computing tier 0, seeing the question "is this vulnerable", or seeing the arbiter's own verdict on the same run. Six metrics so every phase can fail, and budget_at_recall is the one that decides the release question: below 50 and the model is a refinement, above and the model is the plan. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG