unseen: bound the per-repo Python parse phase (--parse-deadline)

Git subprocess calls were deadline-bounded (Git.command_timeout uses the
remaining per-repo budget), but a pure-Python parse between git calls
(repository_changes -> history/blame/diff on a giant file) had no
deadline: when it hung, no further git call fired the git deadline, so
the worker stalled indefinitely (observed >720s with zero git
processes). An external watchdog was papering over it at ~20 min/stall.

Add extract.process_deadline(seconds): a SIGALRM/setitimer wall-clock
alarm installed on the worker's main thread that raises
ExtractionTimeout("wallclock_timeout"), with no-op guards (None/<=0, off
the main thread, or no SIGALRM support) and strict timer-cancel +
handler-restore in finally. Wrap the whole extract_repository body in it
so the timeout propagates to the existing `except source.Rejected` and
journals as a clean rejection, not a worker failure.

Budget = --extract-deadline (git, 240s) + --parse-deadline (new, 120s)
= 360s total ceiling, so git-bound repos keep their full budget (yield
unchanged) while a hung parse is rejected well before any external
watchdog. Record parse_deadline_seconds in the report.

295 unseen tests pass (incl. timeout + handler-restore + no-op cases),
ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-10 14:46:18 +02:00
co-authored by Claude Opus 4.8
parent c4c08d1ca1
commit a49b15b720
4 changed files with 201 additions and 7 deletions
+41
View File
@@ -10,8 +10,11 @@ from __future__ import annotations
import ast
import json
import re
import signal
import subprocess
import threading
import time
from contextlib import contextmanager
from pathlib import Path
from .source import Candidate, Rejected
@@ -33,6 +36,42 @@ class ExtractionTimeout(Rejected):
"""A slow repository is skipped without aborting the draft."""
@contextmanager
def process_deadline(seconds: float | None):
"""Bound Python work in a worker's main thread; cancel the alarm on exit."""
if (
seconds is None
or seconds <= 0
or threading.current_thread() is not threading.main_thread()
or not all(hasattr(signal, name) for name in ("SIGALRM", "ITIMER_REAL", "setitimer"))
):
yield
return
def expired(signum, frame):
raise ExtractionTimeout("wallclock_timeout")
previous = signal.getsignal(signal.SIGALRM)
installed = False
try:
try:
signal.signal(signal.SIGALRM, expired)
installed = True
signal.setitimer(signal.ITIMER_REAL, seconds)
except ValueError:
# Unsupported signal operations must not prevent extraction.
if installed:
signal.signal(signal.SIGALRM, previous)
installed = False
yield
finally:
if installed:
try:
signal.setitimer(signal.ITIMER_REAL, 0)
finally:
signal.signal(signal.SIGALRM, previous)
class Git:
def __init__(self, path: Path, *, runner=None, limit_bytes: int | None = None, deadline_seconds: float | None = None, _now=None):
self._now = _now or time.monotonic
@@ -66,6 +105,8 @@ class Git:
)
except subprocess.TimeoutExpired:
raise ExtractionTimeout("extraction_timeout") from None
except ExtractionTimeout:
raise
except Exception:
raise InstrumentFailure("git_launch_failed") from None
if done.returncode not in allowed: