perf(push): drop per-entry cache fsync; let long runs profile themselves

The cache fsynced every payload and manifest -- two per entry, ~18 ms, and a
push publishes one entry per question (~22,000 on a WordPress plugin). A
read already rejects any entry whose manifest receipt, payload size or
payload SHA-256 does not check, so a torn write is a miss, never a wrong
answer; the fsyncs bought nothing the verification does not.

The broader problem: three quadratic walls in this replay were found one
per two-hour run, because all were Python bookkeeping between engine calls
and the only record was the engine log. `OUSAST_SAMPLE_PROFILE=<path>`
starts an in-process sampler that rewrites the hottest stacks every 30 s,
so a run killed at its deadline still names where its time went. The host
cannot ptrace the container, hence in-process. The replay harness turns it
on per case.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-23 19:05:08 +02:00
co-authored by Claude Opus 5.5
parent e7e2941e37
commit b4b9308af8
5 changed files with 119 additions and 6 deletions
+5 -4
View File
@@ -244,12 +244,13 @@ class ArtifactCache:
elif graph_bytes(payload, budget.deadline_monotonic, target) != sha:
raise ValueError("publication_source_changed")
target.chmod(0o600)
with target.open("rb") as stream:
os.fsync(stream.fileno())
# No fsync. Durability is not what makes an entry safe to read: `_entry` rejects anything
# whose manifest receipt, payload size or payload SHA-256 does not check, so a write torn
# by a crash is a MISS, never a wrong answer. Two fsyncs per entry were ~18 ms, and a push
# publishes one entry per question -- about 22,000 on a WordPress plugin, minutes of the
# budget spent making a cache survive power loss that it already survives by verification.
with (pending / "manifest.json").open("xb") as stream:
stream.write(manifest)
stream.flush()
os.fsync(stream.fileno())
(pending / "manifest.json").chmod(0o600)
if time.monotonic() >= budget.deadline_monotonic:
raise TimeoutError("deadline_exhausted")