mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 20:19:29 +00:00
perf(push): drop per-entry cache fsync; let long runs profile themselves
The cache fsynced every payload and manifest -- two per entry, ~18 ms, and a push publishes one entry per question (~22,000 on a WordPress plugin). A read already rejects any entry whose manifest receipt, payload size or payload SHA-256 does not check, so a torn write is a miss, never a wrong answer; the fsyncs bought nothing the verification does not. The broader problem: three quadratic walls in this replay were found one per two-hour run, because all were Python bookkeeping between engine calls and the only record was the engine log. `OUSAST_SAMPLE_PROFILE=<path>` starts an in-process sampler that rewrites the hottest stacks every 30 s, so a run killed at its deadline still names where its time went. The host cannot ptrace the container, hence in-process. The replay harness turns it on per case. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
e7e2941e37
commit
b4b9308af8
@@ -244,12 +244,13 @@ class ArtifactCache:
|
||||
elif graph_bytes(payload, budget.deadline_monotonic, target) != sha:
|
||||
raise ValueError("publication_source_changed")
|
||||
target.chmod(0o600)
|
||||
with target.open("rb") as stream:
|
||||
os.fsync(stream.fileno())
|
||||
# No fsync. Durability is not what makes an entry safe to read: `_entry` rejects anything
|
||||
# whose manifest receipt, payload size or payload SHA-256 does not check, so a write torn
|
||||
# by a crash is a MISS, never a wrong answer. Two fsyncs per entry were ~18 ms, and a push
|
||||
# publishes one entry per question -- about 22,000 on a WordPress plugin, minutes of the
|
||||
# budget spent making a cache survive power loss that it already survives by verification.
|
||||
with (pending / "manifest.json").open("xb") as stream:
|
||||
stream.write(manifest)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
(pending / "manifest.json").chmod(0o600)
|
||||
if time.monotonic() >= budget.deadline_monotonic:
|
||||
raise TimeoutError("deadline_exhausted")
|
||||
|
||||
Reference in New Issue
Block a user