Files
Open_Ultra_SAST/benchmarks/independent
Marius CiepluchandClaude Opus 5.5 a75b9f62f9 spec(pre-push-safety-net): pre-register evaluation protocol v2 and its runner
Committed before any v2 case is scanned. It has two changes from v1, both chosen by
the maintainer from lessons of the v1 run:
- declared sites: a finding at a `path::function` the advisory names is credited,
  as well as one on the fix's changed lines;
- two runs of the vulnerable and fixed pins: detection needs the finding in both
  runs, a fixed-side alert in either counts, and the difference between runs is
  reported as stability.

evaluate.py takes --population. v2 scans six pins per case and scores with
matches_v2 / score_v2. v1's scoring is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 18:02:03 +02:00
..