mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 5
b57b1a35d8
records whether its decision beat noise Two things the maintainer was right about. Thinking was disabled so `temperature: 0` would not be silently ignored -- a trade that only pays if it buys determinism. Round zero says it did not: 11 of 20 injection pairs disagreed with themselves across five runs, so the variance is in the agentic path, not the decoder. We paid the reasoning cost on a reasoning model and got nothing. It is now `[learning] thinking`, recorded in the artifact and part of its key so the two modes cannot be read as one number. The default stays off only because that is what round zero was measured under. And the deeper point: the spec built `sign_test` and `reliable_change` and the acceptance rule calls neither -- `decide` compares raw deltas. On a detector with that floor a raw delta is mostly noise, and a rule that brittle is exactly what makes a deterministic decoder look necessary. It is not; the alternative is an acceptance rule that models the score as noisy, the way reflective-evolution and Bayesian prompt optimizers do. Changing Req 9.5 is a maintainer's call, so this does not change the decision: every round now journals the two-sided sign test over the pairs that moved, and after four rounds the journal will say whether any decision could be told from a coin flip. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG