mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
unseen: fix systemic-guard miscalibration + the batched-fetch yield regression
Two defects stalled the draw at 40/300: 1. The systemic guard used an absolute `instrument_failures > 200` cap. At ~10% extraction yield, reaching 300 entries legitimately produces far more than 200 per-candidate failures, and the counter is rebuilt from the journal on resume -- so once past 200 the guard tripped immediately on every restart (permanent stall; the supervisor's no-progress guard then stopped). Drop the absolute cap; keep only `>=25 failures with zero entries`, which still catches a genuinely broken instrument (it yields no entries) without aborting a working low-yield draw. Bounded anyway by the population / max-hours. 2. repository_changes fetched all fixes in one `git fetch origin *fixes`, so a single unfetchable secondary fix ref failed the whole repo (extract_git_exit_1 was 166 of ~200 extract_ failures -- the dominant loss). Fetch the extra advisory fixes per-fix, best-effort, skipping a git_exit_ failure; candidate.fix stays strict (already fetched in extract_repository). Only resolved fixes are materialized and fix-site-indexed. 218 unseen tests pass (incl. 1000 failures after a success not tripping the guard, and fetch-error regressions), ruff clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
a9be42b5f8
commit
44311ff2fa
+18
-10
@@ -210,19 +210,25 @@ def advisory_index(rows: list[dict], github) -> dict[str, set[str]]:
|
||||
|
||||
|
||||
def repository_changes(git, candidate, fixes: set[str], *, seed: int):
|
||||
# Fetch detached fixes together, then materialize the histories before any diff,
|
||||
# show or blame can trigger a per-file promisor fetch.
|
||||
fixes = sorted(fixes | {candidate.fix})
|
||||
# The candidate fetch in extract_repository retains the original tag behavior.
|
||||
# Do not import other fixes' tags before introducing() checks its version bracket.
|
||||
git.run("fetch", "--quiet", "--no-tags", "--filter=blob:none", "origin", *fixes)
|
||||
git.materialize(["HEAD", *fixes])
|
||||
resolved = []
|
||||
for fix in sorted(fixes | {candidate.fix}):
|
||||
try:
|
||||
if fix != candidate.fix:
|
||||
# Extra advisory fixes are best-effort exclusions. The candidate
|
||||
# was fetched strictly by extract_repository. Do not import extra
|
||||
# tags before introducing() checks the candidate's version bracket.
|
||||
git.run("fetch", "--quiet", "--no-tags", "--filter=blob:none", "origin", fix)
|
||||
resolved.append(git.run("rev-parse", fix).strip())
|
||||
except InstrumentFailure as exc:
|
||||
if fix == candidate.fix or not str(exc).startswith("git_exit_"):
|
||||
raise
|
||||
# Batch only resolved histories before diff/show/blame can fetch per-file blobs.
|
||||
git.materialize(["HEAD", *resolved])
|
||||
proof = prove_source(git, candidate.parent)
|
||||
change = extract.introducing(git, candidate)
|
||||
known = set()
|
||||
expanded = set()
|
||||
for fix in fixes:
|
||||
full = git.run("rev-parse", fix).strip()
|
||||
for full in resolved:
|
||||
expanded.add(full)
|
||||
for parent in git.parents(full):
|
||||
sites = extract.fix_sites(git, full, parent, global_ok=True)
|
||||
@@ -548,7 +554,9 @@ def run(
|
||||
else:
|
||||
selected_networks.add(decision["network"])
|
||||
entries.append(result["entry"])
|
||||
if (instrument_failures >= 25 and not entries) or instrument_failures > 200:
|
||||
# A low-yield draw can accumulate unbounded candidate failures;
|
||||
# only zero accepted entries indicate a systemic failure.
|
||||
if instrument_failures >= 25 and not entries:
|
||||
raise extract.InstrumentFailure("systemic_extraction_failure")
|
||||
if len(entries) == 300:
|
||||
break
|
||||
|
||||
Reference in New Issue
Block a user