Named the core problem with the maintainer: the tool spots dangerous ops but cannot prove
attacker-reachability -- the trust boundary lives outside the changed library code. Chosen
strategic bet (of three): FIX THE EVALUATION TARGET FIRST.
LLM shape classification of the 49 public pool vulns ($0.033): reachability_in_repo only
24% (12/49); 76% are pure libraries. A legitimate change-scoped catch needs sound label AND
reachability in-repo -> just 5/49 (~10%): graphiti, desktopcommandermcp, malla,
document-merge-service, mcp-remote. Shape predicted the hunter outcome (graphiti app-shaped ->
caught; bedrock library -> missed). So record-07's 3/92 is scored against a ~90%-illegitimate
denominator -- the near-zero catch is largely a target mismatch.
Next: deep scan on the 5 fair targets for a per-tier catch rate; then an app-shaped eval set
from known-vulnerable apps. Artifacts: 2026-10-12-pool-p1-shape-classification/ (+ earlier
label-audit, hunter-sound-targets). Memory: evaluation-target-mismatch.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
