mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 11:49:28 +00:00
Round 5 re-ran round 1's content through the lever it should have used -- counterexamples, rendered to the detector as "Worked examples" rather than "Known non-issues, do not report these" -- and was rejected because one pair of eight moved, at p = 1.000. So the experiment I designed to give the demo lever a fair test could not give it one. Correcting my authoring error changed the reason for the verdict and not the verdict, because the rule cannot tell a lever that helps from a lever that does nothing. Five rounds, $2.17, zero decisions distinguishable from chance. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG