Open_Ultra_SAST

mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00

Files

T

Marius CiepluchandClaude Fable 5.1 bad8c7ada4 measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more

Declared vulnerable sites: 11 of the 15 that were candidates are confirmed (8 at 3
steps, 3 more at 6). Missed: winml (a CORS config; the reachability question does
not fit), FUXA, Budibase, Froxlor.

Restart-based escalation is strictly worse than a straight 6-step budget: 88% of
pgAdmin candidates used every step, so they paid 3 + 6 ($0.0201 per candidate vs
$0.0139 baseline). Defaults are now 6 steps, no escalation. The cache order change
did not lower the per-call cost ($0.002): the conversation prefix was already
cached; 56% of prompt tokens are served from cache.

The larger finding: on 180 pgAdmin candidates verified twice, 15 and 13 were
flagged with only 2 in common. Verdicts on borderline candidates flip run to run,
while the declared sites were found consistently. Precision needs agreement
across runs or a second judgement, not a single pass.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-09-29 10:10:33 +02:00

fixtures

feat(three-stage-scan): gate stage-2 maps against split-sink ranking

2026-09-04 11:53:06 +02:00

independent

measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more

2026-09-29 10:10:33 +02:00

manifests

feat(three-stage-scan): add aliased-sink mutation the regex scan misses

2026-09-04 11:50:44 +02:00

measurements

measure(sqli): PMPro 1-in-5 sample after the fifth fix -- 1/5, recall pins intact