mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
unseen: cache enumeration + resolution to ousast-corpus (restarts skip the ~26-min prefix)
The decision bank made extraction durable, but every restart still re-paid the ~26-minute advisory enumeration + per-advisory resolution, since those lived only in the local journal. Mirror them into the bucket so a restart reads them back and goes essentially straight to any remaining extraction. bank.py: bank_resolution/resolved at <prefix>/<seed>/resolved/<sha256(ghsa)>.json (streamed one-by-one as each advisory is decided), cache_advisories/ cached_advisories at <prefix>/<seed>/advisories/list.json, and bank_repository/repositories (caches the advisory_index repo-resolution pass, preserving renames and 404s). All gated by RESOLUTION_CONTRACT="v1" (extract.py) -- bumped only when enumeration/eligibility/source.candidate output changes; recent fetch/parse/pool changes do not bump it. draft.py run(): with --bank, advisories load from the cache (else enumerate then cache); eligibility decisions load from the bucket (contract-matched) to skip re-resolution; new resolutions stream to the bucket before the local checkpoint write. Repository lookups reuse the repositories/ cache. No --bank => local-journal behavior unchanged. Net: advisories + resolutions + extraction all resume from ousast-corpus; restarts no longer re-enumerate or re-resolve. Fresh and resumed runs produce byte-identical manifests at 1 and 4 workers. Bank methods proven live against ousast-corpus. 356 tests pass, ruff clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
857263d975
commit
06b8a40c46
@@ -21,6 +21,8 @@ from .source import Candidate, Rejected
|
||||
|
||||
# Bump only when accept/reject outcomes or accepted entry bytes change.
|
||||
EXTRACTION_CONTRACT = "v1"
|
||||
# Bump only when enumeration, eligibility or source.candidate outputs change.
|
||||
RESOLUTION_CONTRACT = "v1"
|
||||
|
||||
LANGUAGES = {".py", ".js", ".jsx", ".mjs", ".cjs", ".ts", ".tsx", ".php", ".java"}
|
||||
HUNK = re.compile(r"^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@", re.M)
|
||||
|
||||
Reference in New Issue
Block a user