Marius CiepluchandClaude Opus 5.5 2f1b852966 bench(plane): correct the increment reading -- 17/20 flagged in either pass, not 19 ...

agree's declared_sites_matched counts candidate coverage, not detection. Recounted
from agreed.json rows, plane passes flagged 17 of 20 declared sites against the
reference's 19 (budibase read and fuxa getDaqValue missed by both passes). The
earlier "detection unchanged" reading was wrong.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-09-30 08:20:50 +02:00

..

evaluate.py

measure(independent): the unchanged tool hunter on population v2 -- recall 1/17, and why

2026-09-28 11:36:32 +02:00

freeze-v1.json

spec(pre-push-safety-net): freeze the independent population v1 (15.1)

2026-09-24 16:43:32 +02:00

freeze-v2.json

spec(pre-push-safety-net): freeze the independent population v2

2026-09-26 18:02:03 +02:00

freeze.py

spec(pre-push-safety-net): freeze the independent population v2

2026-09-26 18:02:03 +02:00

hunt.py

spec(pre-push-safety-net): pre-register the hunter experiment on population v2

2026-09-28 09:10:02 +02:00

hunter-protocol-v2.md

spec(pre-push-safety-net): pre-register the hunter experiment on population v2

2026-09-28 09:10:02 +02:00

model_sinks.py

measure(independent): model-classified sinks diagnostic (source only, no vocabulary)

2026-09-28 12:43:03 +02:00

plane-increment-1.json

bench(plane): correct the increment reading -- 17/20 flagged in either pass, not 19

2026-09-30 08:20:50 +02:00

population-v1.toml

spec(pre-push-safety-net): freeze the independent population v1 (15.1)

2026-09-24 16:43:32 +02:00

population-v2.toml

spec(pre-push-safety-net): freeze the independent population v2

2026-09-26 18:02:03 +02:00

protocol-v1.md

spec(pre-push-safety-net): pre-register the evaluation protocol for the independent population

2026-09-24 16:44:32 +02:00

protocol-v2.md

spec(pre-push-safety-net): pre-register evaluation protocol v2 and its runner

2026-09-26 18:02:03 +02:00

research-2026-09-24.md

spec(pre-push-safety-net): reserve the independent population v1 (15.1), unscanned

2026-09-24 16:12:22 +02:00

research-2026-09-26.md

spec(pre-push-safety-net): reserve the independent population v2 (17 cases, unscanned)

2026-09-26 17:53:51 +02:00

results-v1.json

measure(independent): first evaluation on the frozen population -- fails every M4 gate

2026-09-25 09:56:26 +02:00

results-v2-hunter.json

measure(independent): the unchanged tool hunter on population v2 -- recall 1/17, and why

2026-09-28 11:36:32 +02:00

results-v2-model-pipeline.json

docs(pre-push-safety-net): record the model-driven pipeline result on v2 and stop

2026-09-29 11:29:06 +02:00

results-v2.json

measure(independent): evaluation of population v2 under protocol v2 -- fails every M4 gate

2026-09-28 08:52:53 +02:00

score_batched.py

feat(independent): fixed-pin recheck and an exploratory scorer for the batched verifier

2026-09-29 10:25:08 +02:00

selection-v2-model-sinks.json

measure(independent): model-classified sinks put the vulnerable function in scope in 15/17

2026-09-28 13:21:41 +02:00

sink_candidates.py

measure(independent): sink-first selection from source -- the vocabulary misses most real sinks

2026-09-28 12:36:43 +02:00

verifier-batched-check-2026-09-29.json

measure(independent): triage + per-file batching + two-pass agreement -- 16/20 sites, flicker cut 26 -> 9, 34% cheaper

2026-09-29 10:22:11 +02:00

verifier-check-2026-09-29.json

measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more

2026-09-29 10:10:33 +02:00

verify_batched.py

feat(independent): fixed-pin recheck and an exploratory scorer for the batched verifier

2026-09-29 10:25:08 +02:00

verify_sinks.py

measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more

2026-09-29 10:10:33 +02:00