mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
ablation.py registers the manifest (table digest, arms as masks, folds, metric, sample, budget, rule, seed, program id) before any run and refuses a run or an analysis that does not match it. Arms run over the same changes through runner.run, paired by change; effects on recall and on suspicions per ordinary push are reported pooled and per family with the repository-clustered bootstrap and exact McNemar, plus the recall-against-suspicions curve at every reported confidence (never a tuned cut). The importance table and decisions read held-out effects only; a neutral annotation survives only when a kept one needs it. 30 fixture tests with a scripted model. Implemented by Codex (gpt-6-astra), gated here. Co-Authored-By: Claude Fable 5.1 <[email protected]>