Marius CiepluchandClaude Fable 5.1 6acd354311 feat(independent): fixed-pin recheck and an exploratory scorer for the batched verifier ...

- verify_batched.py --fixed asks the agreed candidates again at the fixed pin
  (no triage); a function the fix removed is recorded as removed.
- score_batched.py applies protocol v2's matching to the batched output
  (vulnerable pin, and the fixed pin where rechecked) and draws the precision
  sample.
- A finding's site is anchored to the candidate operation asked about; the
  model's own reported location is kept in `reported_at`. ReactPress's confirmed
  findAll was reported at the controller file and failed to match its declared
  site. The scorer anchors older records the same way.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-09-29 10:25:08 +02:00

..

evaluate.py

measure(independent): the unchanged tool hunter on population v2 -- recall 1/17, and why

2026-09-28 11:36:32 +02:00

freeze-v1.json

spec(pre-push-safety-net): freeze the independent population v1 (15.1)

2026-09-24 16:43:32 +02:00

freeze-v2.json

spec(pre-push-safety-net): freeze the independent population v2

2026-09-26 18:02:03 +02:00

freeze.py

spec(pre-push-safety-net): freeze the independent population v2

2026-09-26 18:02:03 +02:00

hunt.py

spec(pre-push-safety-net): pre-register the hunter experiment on population v2

2026-09-28 09:10:02 +02:00

hunter-protocol-v2.md

spec(pre-push-safety-net): pre-register the hunter experiment on population v2

2026-09-28 09:10:02 +02:00

model_sinks.py

measure(independent): model-classified sinks diagnostic (source only, no vocabulary)

2026-09-28 12:43:03 +02:00

population-v1.toml

spec(pre-push-safety-net): freeze the independent population v1 (15.1)

2026-09-24 16:43:32 +02:00

population-v2.toml

spec(pre-push-safety-net): freeze the independent population v2

2026-09-26 18:02:03 +02:00

protocol-v1.md

spec(pre-push-safety-net): pre-register the evaluation protocol for the independent population

2026-09-24 16:44:32 +02:00

protocol-v2.md

spec(pre-push-safety-net): pre-register evaluation protocol v2 and its runner

2026-09-26 18:02:03 +02:00

research-2026-09-24.md

spec(pre-push-safety-net): reserve the independent population v1 (15.1), unscanned

2026-09-24 16:12:22 +02:00

research-2026-09-26.md

spec(pre-push-safety-net): reserve the independent population v2 (17 cases, unscanned)

2026-09-26 17:53:51 +02:00

results-v1.json

measure(independent): first evaluation on the frozen population -- fails every M4 gate

2026-09-25 09:56:26 +02:00

results-v2-hunter.json

measure(independent): the unchanged tool hunter on population v2 -- recall 1/17, and why

2026-09-28 11:36:32 +02:00

results-v2.json

measure(independent): evaluation of population v2 under protocol v2 -- fails every M4 gate

2026-09-28 08:52:53 +02:00

score_batched.py

feat(independent): fixed-pin recheck and an exploratory scorer for the batched verifier

2026-09-29 10:25:08 +02:00

selection-v2-model-sinks.json

measure(independent): model-classified sinks put the vulnerable function in scope in 15/17

2026-09-28 13:21:41 +02:00

sink_candidates.py

measure(independent): sink-first selection from source -- the vocabulary misses most real sinks

2026-09-28 12:36:43 +02:00

verifier-batched-check-2026-09-29.json

measure(independent): triage + per-file batching + two-pass agreement -- 16/20 sites, flicker cut 26 -> 9, 34% cheaper

2026-09-29 10:22:11 +02:00

verifier-check-2026-09-29.json

measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more

2026-09-29 10:10:33 +02:00

verify_batched.py

feat(independent): fixed-pin recheck and an exploratory scorer for the batched verifier

2026-09-29 10:25:08 +02:00

verify_sinks.py

measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more

2026-09-29 10:10:33 +02:00