measure(g2): banks LLM sizing -- '2 guided by 1' validated; corpus label mislocated

DeepSeek deepseek-flash reachable: ~$0.0002/call, ~1.1s (sub-cent/push). The LLM gives the
correct verdict WHEN aimed at the real sink + source context: asked about the pool's
SZZ-labelled site (_load) it correctly says not-vulnerable; asked about the real sink
(from_prompt_path line 44, prompt.name request-derived) it correctly says vulnerable. So '2
guided by 1' works as a mechanism.

Two blockers, both concrete: (1) it needs the agentic search-with-proof hunter to supply
caller/source context -- single-shot on the ranker's region is insufficient; (2) the pool's
banks label is mislocated (head_lines 30/104/138-141, function _load; real sink is line 44),
so even a correct finding wouldn't score as a catch and the sink pre-exists in base -- a
change-scoped hook correctly stays quiet. Part of the pilot engine 0/5 is SZZ mislocation, not
a detection failure. GLM/abliteration fallback errors HTTP422 on the thinking param (client
fix needed; DeepSeek works).

Next: validate the 5 pilot labels (change-located & introduced?) before detection runs; then
size the agentic hunter on the sound targets; feed confirmed source shapes back as guarded facts.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

.kiro/specs/engine-change-scoped-analysis/brief.md

@@ -298,3 +298,41 @@ attacker-reachable (intent, not pattern). So:
 false-alarm cost on the ordinary changes. Stage it (5–10 sizing → ~10–20 → full on lift). This makes "#2 guided by
 #1" the concrete G2 detection workstream; budget/change-scoping and change-anchoring stay deprioritised until it
 yields findings.
+
+## BANKS SIZING + "2 guided by 1" validated (2026-10-12, budget-approved, DeepSeek)
+
+Sized the LLM tier on banks (the clearest pilot miss) with the project's own DeepSeek client (`deepseek-flash`).
+
+**Infra + cost (sizing answer):** DeepSeek reachable, model id `deepseek-flash` valid. **~$0.00016–0.00025 per
+call, ~1.1 s latency.** Sub-cent per push even at 8 candidates × a few regions. The GLM/abliteration fallback
+(`ABLITERATION_*`) currently errors `HTTP 422` on the `thinking` request param — a client fix (don't send
+`thinking` to that endpoint) is needed before it can be used; DeepSeek works today so it is not blocking.
+
+**"2 guided by 1" validated as a mechanism — the LLM gives the correct verdict WHEN given the real sink + source
+context:**
+- Asked about `_load` (the pool's SZZ-labelled site, 135–141): `vulnerable=false` — *correct*, `_load` reconstructs
+  paths from its own index, not the sink.
+- Asked about the REAL sink `from_prompt_path`/`set()` line 44 (`path / f"{prompt.name}…"; write_text`) with the
+  context that `prompt.name` can be request-derived: **`vulnerable=true`** — *"prompt.name is joined into a
+  filesystem path and written without sanitization… path traversal or arbitrary file write."* A correct catch.
+
+**Two blockers to realising it in the live pipeline (both now concrete):**
+1. **Context, not single-shot.** The model only catches with the real sink + caller/source context. A single-shot
+   question on the ranker's region (which pointed at `_load`, and at the benign local-glob `_scan`) does not. This
+   requires the **agentic search-with-proof hunter** (explores callers/entry points to establish the trust
+   boundary), not the single CANDIDATE_QUESTION path. That is more calls → the real cost driver to size next.
+2. **Corpus-label soundness.** The pool's banks label is **mislocated**: `head_lines=[[30],[104],[138–141]]`,
+   `function=_load` — the real sink is line **44**, outside head_lines±10 and a different function. So (a) even a
+   correct finding at line 44 would NOT score as a catch under `location_match`, and (b) the sink pre-exists in base
+   (the change did not touch it), so a change-scoped pre-push hook **correctly stays quiet** — the pool even
+   annotates this bias ("delta hook can correctly stay quiet"). **Part of the pilot's engine 0/5 is this SZZ
+   mislocation, not a detection failure.**
+
+**Revised next steps (budget approved, stay staged):**
+- **Validate the 5 pilot labels first** (cheap, LLM-assisted corpus audit): for each, is the real sink inside the
+  changed lines, and is it change-introduced vs pre-existing-in-base? Only sound, change-located vulns are fair
+  targets. This separates true detection gaps from SZZ artifacts before spending on detection runs.
+- **Then run the AGENTIC hunter** (search-with-proof) on the sound targets and size its per-vuln call count/cost;
+  measure catch + false-alarm (on ordinary changes) vs regex 1/5. The single-shot path is insufficient; the hunter
+  is the real "#1".
+- Feed confirmed source shapes (e.g. library param → path join) back as guarded static facts (#2).