Files
Open_Ultra_SAST/benchmarks
Marius CiepluchandClaude Opus 5.5 c64e37edbd spec(pre-push-safety-net): reserve the independent population v2 (17 cases, unscanned)
v1 was evaluated and then used to retune the analyzer. A retune can only be
qualified on a population that did not inform it.

The candidates came from web research alone (research-2026-09-26.md):
- advisories were taken newest-first per language under fixed criteria, never by
  how easy they look;
- 261 advisories were examined and 224 rejected, each with a reason;
- every SHA pair was verified through the GitHub API;
- no clone and no scan.
The maintainer selected the rule-based "balanced" set: 5 Python, 6 JS/TS, and 6 PHP
of which 4 are WordPress plugins; 10 injection, 6 destination and 1 config.
Each case declares the advisory's sink sites before any scan, for protocol v2.

freeze.py takes --population and writes freeze-vN.json. The reservation tests now
cover every population-v*.toml.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 17:53:51 +02:00
..