marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-17 12:09:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

unseen-eval: sourcing memory -- compact journal records, bounded caches, streamed history (4.6 GB -> 107 MB RSS)

Browse Source 
The sourcing run was killed four times (exit 143/137) after growing to 4.6 GB
RSS on the 7.7 GB VM. Live after the fix: flat 107 MB through 3,029
advisories. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-05 09:18:54 +02:00
co-authored by Claude Opus 5.5
parent c6e87a8517
commit 285bf58f78
8 changed files with 660 additions and 103 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/unseen/README.md
+21
View File
@@ -143,6 +143,27 @@ and sleepers. A stale candidate 404 is counted; other service failures fail the
### Checkpoint and resume
Replay reads bounded JSONL records into a disposable SQLite working store, with a 2 MiB
page cache. Advisory descriptions, non-fix references and vulnerable-version payloads
are discarded; package identities, CWEs, status, publication dates and every fix link
are retained. Repository metadata keeps canonical/fork identities and eligibility fields.
Advisory ordering, fix indexes and checkpoint lookups stay on disk, independent of the
number of journal records. The original seeded ordering and ordinary draws are preserved.
Per-repository history and exclusion sets also spill to private temporary storage, which
is removed on success or failure. Budget disk space for the journal, working store and
spooled Git output as well as the clone.
To bound individual reads, journal records are limited to 16 MiB, decoded Git command
output to 8 MiB, history messages to 1 MiB and source blobs to 512 KiB. Exceeding a limit
fails the instrument explicitly; it never counts as an empty history or a clean result.
The synthetic 20,000-advisory regression checks a 64 MiB traced-memory ceiling across
replay, enumeration, ordering and fix indexing; this is not a live coordinator RSS measurement.
The memory fix can migrate the exact preceding design fingerprints automatically,
keeping `inputs-before-memory-fix.json` and leaving the JSONL journal untouched. All
other input/design mismatches still refuse resume. This compatibility exception must
be removed if a later change alters selection or sampling semantics.
The command above uses journal directory
`/home/mc/ousast-results/plane/unseen-draft/journal`. Run the same command to resume.
`--max-hours` is a per-invocation budget; changing it does not invalidate checkpoints.