Marius CiepluchandClaude Opus 4.8 a43e6293f5 measure(g0/g2): LLM corpus-label audit -- only 47% of public pool labels are sound ...

Widened the banks label-mislocation finding across all 49 public pool vulns via DeepSeek
(advisory + head code + pool label -> sound/unsound), $0.038 total. Result: 23/49 (47%)
sound; 53% unsound (25 pre-exist in base, 16 sink outside the changed lines) -- a change-scoped
hook correctly stays quiet there. Systematic by family: injection 10/13 sound, but path 1/7,
output_encoding 1/8, config_secrets 1/4.

Implications: record-07's floor (catch 3/92) is scored against a corpus ~half unfair to a delta
hook; the honest catchable denominator is the sound subset (caveat added to record 07 NOTES).
Fair pilot re-score: only bedrock + graphiti (injection) of the 5 are sound and the engine
missed both = the real detection gaps for the LLM-guided hunter.

Also: GEPA-style reflective-retry (future spec) + GLM cyber model (future spec) noted; DeepSeek
is the working default. Artifact: benchmarks/measurements/2026-10-12-pool-p1-label-audit/.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

2026-10-12 10:09:32 +02:00

..

NOTES.md

measure(g0/g2): LLM corpus-label audit -- only 47% of public pool labels are sound

2026-10-12 10:09:32 +02:00

results.jsonl

measure(g0/g2): LLM corpus-label audit -- only 47% of public pool labels are sound

2026-10-12 10:09:32 +02:00

summary.txt

measure(g0/g2): LLM corpus-label audit -- only 47% of public pool labels are sound

2026-10-12 10:09:32 +02:00