feat(ranking): phase 0 -- the baseline, the evidence instrument, and a 7x taint speedup found on the way

flow-aware-ranking phase 0: measure before changing anything.

THE BASELINE, committed as benchmarks/ranking/baselines/2026-09-10-access-rank.json
from benchmarks/ranking/position.py, which sorts regions exactly as the scan does:

    pmpro         4,463 regions  83.8% at 0.30   CVE at 390   budget_at_recall 391
    mwwpform        983 regions  81.7% at 0.30   CVE at 512   budget_at_recall 513
    wpstatistics  1,673 regions  75.3% at 0.30   CVE at 503   budget_at_recall 504
    vampi            25 regions                  BOLA at 15, 10; SQLI reached transitively

That corrects the brief's own cross-assessment: "concentration, not inclusion"
was measured on pmpro alone. On the other two plugins the CVE sits OUTSIDE a
500-region budget and is never examined; the slices found those CVEs only
because a slice has too few regions to exclude anything.

THE INSTRUMENT: `evidenceOnly` mode on the taint query -- what is in scope,
which sink calls, what shape, whether a sanitizer wraps THEIR argument, whether
a source is local or reachable -- returning before reachableByFlows is ever
called. model/evidence.py turns its rows into a vector per (region, family)
and a tier; scan.py records every pair's tier on the result and acts on none of
it. Every later phase is measured against a record made by scans that did not
know they were being measured.

    2,500 pairs in 98.9s incl. JVM load, ~40 ms per pair
    tier 0 -- no sink of that family in reach: 2,161 of 2,500 = 86%

Two earlier attempts ran past an hour each with no payload, and both were my
fault: repository-wide walks repeated per request -- the sink scan, the
reachable-method BFS, familyInRepo. Memoised across the batch.

AND THE SAME FIX REACHES THE DATAFLOW MODE. sinkCalls and reachableMethods do
not depend on the family, and were recomputed for all 2,500 requests. The
identical 250-request set with both joins on:

    2,026.3s -> 287.0s      8,105 -> 1,148 ms per request      same 5,779 rows

Most of what 5.13 measured as "the base taint query" was never dataflow. A 7x
speedup with nothing about the analysis changed; both corrections recorded in
the specs. Combined with 86% tier 0, the whole-repository taint query that
timed out at 40 minutes is ~339 real requests at ~1.1s: about six minutes.

Also: the module audit manifest is regenerated (91 load-bearing, 7 standalone,
0 orphaned) because evidence.py is now imported by the scan, and the
failed-query test is order-independent since the evidence pass records its own
failure before taint's.

869 tests pass; query compile test 4 passed; gates byte-identical.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

.kiro/specs/contributor-scan/tasks.md

@@ -821,6 +821,13 @@
     half is free (inside noise, and measured slightly faster), the hook half adds 7.5%, both add 22%.
     Removing them buys a fifth of the time and costs two of the three CVEs, which is not a trade worth
     making.
+  - **Corrected 2026-09-10, same day: most of that 6.63 s was never dataflow.** `sinkCalls` scanned every
+    call in the graph per request, and `reachableMethods` ran a call-graph BFS per request — both
+    independent of the region's family, both repeated 2,500 times. Memoised across the batch (found while
+    building flow-aware-ranking's evidence mode, which asks no dataflow and still took an hour), the same
+    250 requests with both joins on: **2,026.3 s → 287.0 s, 8,105 → 1,148 ms per request, identical 5,779
+    rows.** A 7× speedup with nothing about the analysis changed. The base cost of a taint request is about
+    1.1 s, and the levers above still apply to it.
   - **The estimate this replaces was wrong and worth recording as wrong.** ">0.94s per request" came from
     assuming taint COMPLETED within its 2,400s ceiling. It timed out, so that was a floor presented as a
     figure. The measured cost is seven times higher.

.kiro/specs/flow-aware-ranking/brief.md

@@ -55,6 +55,41 @@ A phase that cannot show that is not an improvement, it is a story.
 - **Phase 0 — the oracle and the baseline.** The evidence extractor (no model) and a scoring harness over the
   pinned CVEs, reporting each one's rank position. Gate: reproduces today's numbers — CVE-2023-23488 at
   position 390 of 4,463, 83.8% of regions in one tier. Without a baseline nothing later is measurable.
+
+  **Done 2026-09-10.** `benchmarks/ranking/position.py`, committed as
+  `benchmarks/ranking/baselines/2026-09-10-access-rank.json`:
+
+  | repository | regions | largest tier | CVE position | `budget_at_recall` |
+  |---|---|---|---|---|
+  | `pmpro` | 4,463 | 83.8% at 0.30 | 390 | 391 |
+  | `mwwpform` | 983 | 81.7% at 0.30 | **512** | **513** |
+  | `wpstatistics` | 1,673 | 75.3% at 0.30 | **503** | **504** |
+  | `vampi` | 25 | 36.0% at 1.00 | 15, 10 (SQLI reached transitively) | 16 |
+
+  **And this corrects the cross-assessment above.** "Ranking's job is concentration, not inclusion" was
+  measured on `pmpro` alone, where the CVE is inside the budget. On the other two plugins the CVE sits at
+  512 and 503 — **outside a 500-region budget** — so the current ranker never examines it at all, and the
+  slices found those CVEs only because a slice has too few regions to exclude anything. Both jobs are
+  real: inclusion on two repositories, concentration on all three. The single metric `budget_at_recall`
+  covers both, which is why it is the one that matters.
+
+  The instrument is `evidenceOnly` mode on the taint query, recorded on every scan as `ModelScanResult.tiers`
+  and `tier_counts`, and acted on by nothing — so every later phase is measured against a record made by
+  scans that did not know they were being measured.
+
+  **Measured cost of the instrument**, 637-file graph, the scan's own 500-region request set:
+
+      2,500 pairs in 98.9 s including JVM load — about 40 ms per pair
+      tier 0 (no sink of that family in reach): 2,161 of 2,500 = 86%
+
+  Two attempts before that ran past an hour each without producing a payload, and the cause was mine both
+  times: repository-wide walks — the sink scan, the reachable-method BFS, `familyInRepo` — repeated per
+  request. Memoising them across the batch is what made "milliseconds per pair" true rather than claimed.
+
+  **And the same memoisation reaches the dataflow mode.** The identical 250-request set that cost 2,026.3 s
+  (8,105 ms per request) with both joins on now costs **287.0 s — 1,148 ms per request — for the same 5,779
+  rows.** Most of what contributor-scan 5.13 measured as "the base taint query" was the per-request scans,
+  not the dataflow. That correction is recorded there.
 - **Phase 1 — tier 0 only, no model.** Exact pruning. Gate: **53% fewer taint requests issued, measured** —
   and nothing more. 1,177 requests at 6.63 s is still 2.2 hours, so this phase does NOT make the scan finish;
   it halves the problem. Claiming completion here would send someone looking for a bug where there is none.