feat(scan): size portions from measured per-answer cost and isolate the kill culprit

The 7.8 trace showed the completion loss was a sizer death spiral, not
expensive families: after two kills the budget fell to 3 sink visits and
never recovered, because portions finishing in 25-30 s sat under an assumed
70 s fixed start and read as "no information", so the budget could only
shrink.

- taint.sc streams each answer's own cost (`ms`); the backend carries it as a
  reserved `__timing__` list through shards, partitions and reuse.
- The sizer budgets predicted seconds from per-family seconds-per-visit rates
  learned from those costs, and measures the fixed start as the wall time the
  answers do not explain (median), instead of assuming it.
- A kill names its culprit: the first unanswered id in request order. The ids
  behind it never started and are re-queued at once; the culprit is asked
  alone after the rest. One that still fails alone after more than a
  portion's target is reported as `query_too_expensive`, apart from
  `query_failed`.
- Portions grow id by id from the queue head, and a single-portion scan now
  goes through the queue too, so a kill there re-asks its remainder.
- Fix: the reuse wrapper wrapped only `run_batch`, so since 7.8 every
  portioned query bypassed the graph-reuse cache and its source-root
  normalization. `run_batch_once` is wrapped too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

src/openultrasast/cpg/backend.py

@@ -194,6 +194,10 @@ Runner = Callable[..., subprocess.CompletedProcess[str]]
 STREAM_ID = "id"
 STREAM_ROWS = "rows"
 PARTIAL = "__partial__"
+# What each streamed answer cost the engine, as `[{"id": ..., "ms": ...}]` under a reserved key beside the
+# census. It is a list so every layer that merges answers per key -- shards concatenate, partitions collect --
+# carries it without knowing what it is, and every consumer that reads answers by request id ignores it.
+TIMING = "__timing__"
 
 
 def extract_payload(stdout: str) -> object | None:
@@ -254,6 +258,11 @@ def _streamed(body: str, *, complete: bool) -> dict[str, object] | None:
         elif STREAM_ID in document and STREAM_ROWS in document:
             answers[str(document[STREAM_ID])] = document[STREAM_ROWS]
             saw_stream = True
+            cost = document.get("ms")
+            if isinstance(cost, (int, float)) and not isinstance(cost, bool):
+                timing = answers.setdefault(TIMING, [])
+                if isinstance(timing, list):
+                    timing.append({"id": str(document[STREAM_ID]), "ms": float(cost)})
     if not saw_stream:
         return None
     if not complete:
@@ -1144,7 +1153,7 @@ class JoernBackend:
             logger.info(
                 "cpg batch %s: %d of %d answered before the kill; asking the other %d in parts",
                 query,
-                len([k for k in merged if k != "__census__"]),
+                len([k for k in merged if k not in ("__census__", TIMING)]),
                 len(requests),
                 len(remaining),
             )
@@ -1249,7 +1258,7 @@ class JoernBackend:
             logger.warning("cpg batch %s returned no parseable payload", query)
             return None
         if parsed.get(PARTIAL):
-            answered = sum(1 for rid, rows in parsed.items() if rid != PARTIAL and isinstance(rows, list) and rid != "__census__")
+            answered = sum(1 for rid, rows in parsed.items() if rid not in (PARTIAL, TIMING, "__census__") and isinstance(rows, list))
             logger.info("cpg batch %s was killed with %d of %d answer(s) already streamed", query, answered, len(requests))
         # A request the engine did not answer is absent, not empty: the caller must be able to tell them apart.
         return {str(rid): list(rows) for rid, rows in parsed.items() if rid != PARTIAL and isinstance(rows, list)}

src/openultrasast/cpg/queries/taint.sc

@@ -886,6 +886,12 @@
     println(ujson.write(ujson.Obj("__census__" -> census)))
     System.out.flush()
     parsed.foreach { case (id, req) =>
+      // What THIS answer cost, measured where it is spent. The driver sizes its portions from it, learns the
+      // fixed start as the wall time the answers do not account for, and -- when a portion is killed -- knows
+      // exactly which request was running, because every one before it reported its own time. A per-portion
+      // wall clock cannot tell one 90 s request from thirty 3 s ones, and fitting a single rate to both
+      // collapsed the sizer to three sink visits per JVM start on the plugin.
+      val started = System.nanoTime()
       def field(name: String): String = req.obj.get(name).map(_.str).getOrElse("")
       val paramSrc = req.obj.get("parameterSources").map(_.str).getOrElse("false")
       val fieldSrc = req.obj.get("fieldSources").map(_.str).getOrElse("true")
@@ -910,7 +916,7 @@
           field("contextEvidence")
         ): _*
       )
-      println(ujson.write(ujson.Obj("id" -> id, "rows" -> rows)))
+      println(ujson.write(ujson.Obj("id" -> id, "rows" -> rows, "ms" -> ((System.nanoTime() - started) / 1000000L).toDouble)))
       System.out.flush()
     }
     // The census rides the stream as its FIRST line rather than costing its own invocation, because JVM