unseen: fix systemic-guard miscalibration + the batched-fetch yield regression

Two defects stalled the draw at 40/300:

1. The systemic guard used an absolute `instrument_failures > 200` cap.
   At ~10% extraction yield, reaching 300 entries legitimately produces
   far more than 200 per-candidate failures, and the counter is rebuilt
   from the journal on resume -- so once past 200 the guard tripped
   immediately on every restart (permanent stall; the supervisor's
   no-progress guard then stopped). Drop the absolute cap; keep only
   `>=25 failures with zero entries`, which still catches a genuinely
   broken instrument (it yields no entries) without aborting a working
   low-yield draw. Bounded anyway by the population / max-hours.

2. repository_changes fetched all fixes in one `git fetch origin *fixes`,
   so a single unfetchable secondary fix ref failed the whole repo
   (extract_git_exit_1 was 166 of ~200 extract_ failures -- the dominant
   loss). Fetch the extra advisory fixes per-fix, best-effort, skipping a
   git_exit_ failure; candidate.fix stays strict (already fetched in
   extract_repository). Only resolved fixes are materialized and
   fix-site-indexed.

218 unseen tests pass (incl. 1000 failures after a success not tripping
the guard, and fetch-error regressions), ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-09 23:54:08 +02:00
co-authored by Claude Opus 4.8
parent a9be42b5f8
commit 44311ff2fa
2 changed files with 102 additions and 18 deletions
+18 -10
View File
@@ -210,19 +210,25 @@ def advisory_index(rows: list[dict], github) -> dict[str, set[str]]:
def repository_changes(git, candidate, fixes: set[str], *, seed: int):
# Fetch detached fixes together, then materialize the histories before any diff,
# show or blame can trigger a per-file promisor fetch.
fixes = sorted(fixes | {candidate.fix})
# The candidate fetch in extract_repository retains the original tag behavior.
# Do not import other fixes' tags before introducing() checks its version bracket.
git.run("fetch", "--quiet", "--no-tags", "--filter=blob:none", "origin", *fixes)
git.materialize(["HEAD", *fixes])
resolved = []
for fix in sorted(fixes | {candidate.fix}):
try:
if fix != candidate.fix:
# Extra advisory fixes are best-effort exclusions. The candidate
# was fetched strictly by extract_repository. Do not import extra
# tags before introducing() checks the candidate's version bracket.
git.run("fetch", "--quiet", "--no-tags", "--filter=blob:none", "origin", fix)
resolved.append(git.run("rev-parse", fix).strip())
except InstrumentFailure as exc:
if fix == candidate.fix or not str(exc).startswith("git_exit_"):
raise
# Batch only resolved histories before diff/show/blame can fetch per-file blobs.
git.materialize(["HEAD", *resolved])
proof = prove_source(git, candidate.parent)
change = extract.introducing(git, candidate)
known = set()
expanded = set()
for fix in fixes:
full = git.run("rev-parse", fix).strip()
for full in resolved:
expanded.add(full)
for parent in git.parents(full):
sites = extract.fix_sites(git, full, parent, global_ok=True)
@@ -548,7 +554,9 @@ def run(
else:
selected_networks.add(decision["network"])
entries.append(result["entry"])
if (instrument_failures >= 25 and not entries) or instrument_failures > 200:
# A low-yield draw can accumulate unbounded candidate failures;
# only zero accepted entries indicate a systemic failure.
if instrument_failures >= 25 and not entries:
raise extract.InstrumentFailure("systemic_extraction_failure")
if len(entries) == 300:
break