mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 5
bd3e2fe50f
The harness treats the detector as a black box emitting a noisy scalar, and K runs, majority voting, the measured floor, the regression budget and the sign test all follow from that. It is the stance you take when you cannot see inside what you are measuring, and we can. Temperature is already 0 and thinking is disabled, and 11 of 20 injection pairs still disagree with themselves. The freedom is in the trajectory -- which files get read, which greps get run -- and we built statistics to average over it instead of removing it. Three DSPy constructs attack the source: constraints enforced as predicates at generation time (we already compute every one of them, after the fact, at scoring time), typed decomposition of the single ReAct loop into steps whose deterministic parts we already answer without a model, and best-of-N against a checkable reward instead of a majority vote that discards the disagreement. The cheaper the detector model, the more of the return sits there and the less in prompt evolution -- the opposite of the order this brief first proposed. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG