measure(contributor-scan): 7.7 makes a large PHP population affordable, 0 findings to 125

On the 500-region population where every earlier run produced nothing: 153 of 406 questions completed
and 125 findings, inside the deadline. 380 requests carrying 5,807 sink visits split into 11 batches,
each answering in 80 to 197 s, none near the 300 s ceiling. Every family that can speak on this
subject now does, injection included, which is the one holding a true positive here.

Four defects, each hidden by the one before it, and three of them one rule.

  Batches sized by request count, when cost is per matched sink.
  Packing heaviest-first, which starved injection of all 114 of its questions.
  Any per-query timeout cancelling the whole scan, abandoning batches with 33 minutes left.
  No budget kept for arbitration, which removing that cancellation exposed: rows are not findings.

The reserve had to be checked per BATCH. Checked between query kinds it changed nothing at all,
because one kind is a single loop iteration that runs every one of its batches inside it.

What it does not claim: 253 of 406 questions still went unanswered and one batch still records a
timeout; the raw finding count is dominated by output encoding, the family already measured to be
imprecise on PHP; and the reserve share of a fifth is a first value, not a tuned one.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

.kiro/specs/contributor-scan/tasks.md

@@ -1268,6 +1268,10 @@ the ontology describes libraries, frameworks and language constructs, never repo
   - The driver currently splits a failed batch into eight parts by request count, which cannot help: each part carries the same everywhere-sinks.
   - The count needed is already computed. `taint.sc` memoises its candidate sink list per sink set, and the evidence pass reports the in-scope sink list per request before any dataflow runs. A driver that reads it can size a batch by expected work, and can report a family whose sinks are everywhere as EXPENSIVE rather than as failed.
   - Done when a batch is sized by the sink nodes it implies, the measured plugin's arbitration completes within its ceiling, and a family too expensive to arbitrate is reported as such rather than as a timeout.
+  - _Implemented and measured 2026-09-22._ On the 500-region population where every earlier run produced nothing: **153 of 406 questions completed and 125 findings, inside the deadline**, against 0 and 0. 380 requests carrying 5,807 sink visits split into 11 batches, each answering in 80 to 197 s, none near the 300 s ceiling. Every family that can speak on this subject now does: injection 49 completed and 24 findings, path 64 and 15, output encoding 32 and 70, untrusted destination 8 and 16. Eight batches were left unasked to keep time for arbitration, and that is recorded rather than silent.
+  - _Four defects, each hidden by the one before it, and three of them the same rule._ Batches sized by request count when cost is per matched sink. Packing heaviest-first, which starved injection — the one family with a true positive here — of all 114 of its questions. Any per-query timeout cancelling the whole scan, abandoning batches with 33 minutes of budget left. And no budget kept for arbitration, which removing that cancellation exposed: rows are not findings, so the query phase ran the full deadline and completed nothing, where the cascade had been an accidental guard.
+  - _The reserve had to be checked per BATCH._ Checked between query kinds it changed nothing at all, because one kind is a single loop iteration that runs every one of its batches inside it. The granularity of a budget check has to match the granularity of the work it guards. Evidence: `benchmarks/measurements/2026-09-22-sink-weighted-batching.json`.
+  - _What it does not claim:_ 253 of 406 questions still went unanswered and one batch still records a timeout; the raw finding count is dominated by output encoding, the family already measured to be imprecise on PHP; and the reserve share of a fifth is a first value, not a tuned one.
   - _Boundary: contributor-scan scan budget, cpg backend_
   - _Depends: 7.3_
   - _Requirements: 4.1, 6.3, 8.1_

benchmarks/measurements/2026-09-22-sink-weighted-batching.json

@@ -0,0 +1,74 @@
+{
+  "schema_version": 1,
+  "experiment": "sink-weighted-batching-v1",
+  "date": "2026-09-22",
+  "task": "contributor-scan 7.7",
+  "claim_scope": "Making a large PHP population affordable. Measured on Paid Memberships Pro at 500 regions, where every earlier run produced nothing.",
+  "four_defects_each_hidden_by_the_one_before": [
+    {
+      "defect": "batches sized by request count",
+      "why_wrong": "cost is per matched sink; reachableByFlows is asked once per sink call the family's names match",
+      "fix": "budget in sink visits, from the count the evidence pass already produced",
+      "measured": "380 requests carrying 5,807 sink visits split into 11 batches, each answering in 80-197 s, none near the 300 s ceiling"
+    },
+    {
+      "defect": "packing heaviest-first",
+      "why_wrong": "the expensive output-encoding batches ran first and spent the deadline, leaving injection with 114 questions asked and none answered -- the one family holding a true positive on this subject",
+      "fix": "keep the ranker's order; a deadline always cuts a tail, and the tail it should cut is the one the ranker put last",
+      "measured": "injection 0 completed and 0 findings became 11 and 12"
+    },
+    {
+      "defect": "any query timeout cancelled the whole scan",
+      "why_wrong": "a per-query ceiling and an exhausted scan deadline reached one handler and got one consequence; four batches returned in 0.0 s with 33 minutes of budget left",
+      "fix": "cancel only when the SCAN deadline is gone",
+      "measured": "batches behind a failed one now run: 27 in 111 s, 8 in 97 s, 71 in 137 s"
+    },
+    {
+      "defect": "no budget kept for arbitration",
+      "why_wrong": "rows are not findings. Removing the cascade was right and made the result WORSE, because the cascade had been an accidental guard: the query phase then ran the full deadline and completed nothing",
+      "fix": "reserve a fifth of what is left when querying starts, floor 60 s, never more than half -- and check it before each BATCH, because one query kind is a single loop iteration that runs all of its batches inside it",
+      "measured": "checked per kind it changed nothing at all, 0 of 406; checked per batch, 153 of 406"
+    }
+  ],
+  "measured": {
+    "with the cascade": {
+      "questions_completed": 37,
+      "questions_arbitrated": 406,
+      "findings": 78,
+      "query_seconds": 734.43,
+      "total_seconds": 957.7,
+      "deadline_exhausted": false
+    },
+    "cascade removed": {
+      "questions_completed": 0,
+      "questions_arbitrated": 406,
+      "findings": 0,
+      "query_seconds": 2444.42,
+      "total_seconds": 2700.8,
+      "deadline_exhausted": true
+    },
+    "reserve per kind": {
+      "questions_completed": 0,
+      "questions_arbitrated": 406,
+      "findings": 0,
+      "query_seconds": 2435.44,
+      "total_seconds": 2700.6,
+      "deadline_exhausted": true
+    },
+    "reserve per batch": {
+      "questions_completed": 153,
+      "questions_arbitrated": 406,
+      "findings": 125,
+      "query_seconds": 2078.94,
+      "total_seconds": 2338.2,
+      "deadline_exhausted": false
+    }
+  },
+  "the_result": "153 of 406 questions completed and 125 findings, inside the deadline with no exhaustion, where the same population previously completed nothing. Every family that can speak on this subject now does: injection 49 completed and 24 findings, path 64 and 15, output encoding 32 and 70, untrusted destination 8 and 16. Eight batches were left unasked to keep time for arbitration, and that is recorded rather than silent.",
+  "what_this_does_not_claim": [
+    "The population is still not fully arbitrated: 253 of 406 questions went unanswered and the scan still records process_timeout on one batch.",
+    "Raw finding counts are dominated by output encoding, which is the family already measured to be imprecise on PHP because the WordPress escapers are absent from a sanitizer list that cannot separate weakness classes. More findings there is not better detection.",
+    "One subject. The saving depends on how common each family's sinks are in a codebase, which is a property of the code and the ontology together.",
+    "The reserve share of a fifth is a first value, chosen because the phases after querying took roughly 220 s for 400 questions on this subject. It is not tuned."
+  ]
+}