fix(scan): the portioner asks single attempts and reads coverage, not wall time

End-to-end, the adaptive portioner regressed completion -- 64 of 406 against 7.7 hard-cap 153 --
while finding the same defects. The cause: the portioner timed each portion with run_batch, which
splits on failure internally, so one portion folded its own eight-way retry into its number and read
953 s. The sizer fitted a per-visit rate from that and collapsed its budget to 14 visits, spending
the back half of the deadline on tiny portions.

Two portioning layers were fighting. The portioner is now the sole retry layer: it asks
run_batch_once, a single invocation whose kill returns the answers it streamed, so its wall time is
the cost of one attempt. And it observes COVERAGE, not emptiness -- a portion that came back short
of the ids it asked halves the budget, because it was too big for the ceiling; only a whole answer
gives a clean per-visit sample to fit from.

run_batch_once is new on CpgResult, wired on the disposable and sharded paths; a backend without it
falls back to run_batch unchanged. Host controls cover the single-attempt path and the
halve-on-short-answer rule.

1,424 tests pass, nineteen skipped. The end-to-end plugin re-measure on a frozen tree is the open
item; the mechanism (a kill keeps what finished) and the sizer logic are unit-verified.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-22 23:00:14 +02:00
co-authored by Claude Opus 4.8
parent 2e96ccb8ea
commit 0fef399869
3 changed files with 65 additions and 4 deletions
+42
View File
@@ -1269,3 +1269,45 @@ def test_a_killed_batch_retries_only_the_remainder(tmp_path: Path) -> None:
assert answered is not None and answered["a"] == [1] and answered["b"] == ["b"] and answered["c"] == ["c"]
assert calls[0] == ["a", "b", "c"]
assert all("a" not in retry for retry in calls[1:]), calls
def test_the_portioner_asks_single_attempts_and_halves_on_a_short_answer(tmp_path: Path) -> None:
"""A killed portion returns what it streamed; the sizer must read that as too big, not as slow.
Timing `run_batch` folds its internal split-retry into the number -- one portion read 953 s that way and
the sizer fitted a per-visit rate from it and collapsed. The portioner asks `run_batch_once` instead, and
a portion that came back short of the ids it asked halves the budget rather than fitting from a ceiling.
"""
from openultrasast.cpg.backend import CpgResult
from openultrasast.model import scan
once_calls: list[list[str]] = []
def once(kind: str, requests: dict[str, dict[str, object]]) -> dict[str, list[object]] | None:
once_calls.append(sorted(requests))
# Answer everything except a designated heavy id, as a kill would: it streamed the rest and died.
return {rid: [] for rid in requests if rid != "heavy"}
result = CpgResult(cpg_path=tmp_path / "c.bin", run=lambda k, p: None, run_batch=lambda k, r: None, run_batch_once=once)
seen: list[bool] = []
real_observe = scan._PortionSizer.observe
def spy(self, chunk, seconds, *, answered): # type: ignore[no-untyped-def]
seen.append(answered)
return real_observe(self, chunk, seconds, answered=answered)
# Two portions over the budget so the portioner actually splits and observes.
weights = {"heavy": 500, "a": 400, "b": 400}
requests = {rid: {} for rid in weights}
monkey = scan._PortionSizer.observe
scan._PortionSizer.observe = spy # type: ignore[method-assign]
try:
merged = scan._batched(result.run_batch_once, "taint", requests, weights=weights)
finally:
scan._PortionSizer.observe = monkey # type: ignore[method-assign]
assert merged is not None and "a" in merged and "b" in merged and "heavy" not in merged
assert once_calls, "the single-attempt path was not used"
# The portion that came back short of its ids was reported as not-whole (False) to the sizer.
assert False in seen, seen