Files
Open_Ultra_SAST/benchmarks
Marius CiepluchandClaude Opus 5.5 66fa08cdd0 fix(independent): the sink verifier stops on an account error and retries unverified candidates
The rerun hit DeepSeek HTTP 402 ("Insufficient Balance") after 180 pgAdmin
candidates. Every later call failed at $0, and the runner wrote sixteen cases as
complete with nothing asked, because an unverified candidate counted toward
completion. That is a clean-looking zero from an instrument that could not call
the model.

Now:
- 401/402 or "Insufficient Balance" stops the run at once, exit 2, nothing
  recorded;
- only verified candidates count as done, so unverified ones are asked again on
  resume;
- a case is complete only when every candidate was verified.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 07:00:59 +02:00
..