Files
Open_Ultra_SAST/benchmarks/measurements
Marius CiepluchandClaude Opus 5.5 c2bfcef3df measure: hunt baseline on the 12 pilot positives
3 of 12 with the fixed answer check against the pilot hunter's 6 of 12, one
run each, not significant. The intended zero-cost replay was not a replay:
answers at temperature 0 differ between runs and the cache did not serve the
second run. Programs must be compared over repeated runs, paired by change.
About $1.10 for both runs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-13 13:15:22 +02:00
..