unseen: configurable pool size and slice count (--pool-size / --slices)

The draw hardcoded 300 repos / 3 slices. At the measured lazy-fetch rate
the full 300 projects ~2.5 days; a 100-repo single slice (slice 1, all
the G0 baseline needs) is ~20h. Add --pool-size (default 300) and
--slices (default 3), validated (>=1, slices<=pool_size, evenly
divisible). Generalize assign_slices' stratified allocation from a
literal 3 to `slices`, the run() break / insufficient check / report
(changes = pool_size*13) to pool_size, and record both in the journal
inputs. Remove bisect's three-slice assumption; freeze reads the size
from the draft. Defaults (300/3) reproduce prior output exactly.

With --pool-size 100 --slices 1 the pool is the first 100 accepted repos
in seed order, all in slice 1 (deterministic, byte-identical across
worker counts). 280 unseen tests pass, ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-10 07:54:49 +02:00
co-authored by Claude Opus 4.8
parent f0bcbe08b5
commit 5ea28ca9f8
5 changed files with 170 additions and 29 deletions
+2 -1
View File
@@ -50,8 +50,9 @@ def read_pair(public_path: Path, private_path: Path) -> tuple[dict, dict]:
raise ValueError
if set(hidden) != {r["id"] for r in rows if r.get("private")}:
raise ValueError
# Slice IDs come from the draft; bisect may leave sparse slices.
for row in rows:
if type(row["slice"]) is not int or row["slice"] not in (1, 2, 3):
if type(row["slice"]) is not int or row["slice"] < 1:
raise ValueError
if row.get("private"):
full = hidden[row["id"]]