search: upload the executor command once, not on every poll

ObjectStoreExecutor.submit re-PUT the identical command.json on every
0.2s result poll, so a task running minutes wrote ~1,500 versions of one
engine-queue/search-exec/<id>/command.json. On the versioned memory
bucket that was the dominant growth: 109,715 versions, 10.3 GB (91% of
the bucket). retry_mailbox already retries a lost PUT acknowledgement
internally, so one upload per task is the correct delivery guarantee.

Guard the PUT with an `uploaded` flag: upload once (still retried on
transient failure), then poll only the result. One command.json version
per task instead of ~1,500. All submit semantics preserved.

59 executor tests pass, ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-09 14:03:30 +02:00
co-authored by Claude Opus 4.8
parent 6773c63634
commit b9f7958414
2 changed files with 45 additions and 5 deletions
+6 -3
View File
@@ -583,13 +583,16 @@ class ObjectStoreExecutor:
retry_end = end + RETRY_SECONDS
last_error = ""
phase = "result upload"
uploaded = False
while time.monotonic() < end:
try:
# A lost PUT acknowledgement is retried with the exact same sequence.
phase = "command upload"
before = time.monotonic()
try:
retry_mailbox(lambda timeout: self.transport.put(self.command_url, payload, timeout), deadline=retry_end)
if not uploaded:
# Retry lost acknowledgements internally; successful delivery needs no further PUTs.
phase = "command upload"
retry_mailbox(lambda timeout: self.transport.put(self.command_url, payload, timeout), deadline=retry_end)
uploaded = True
phase = "result read"
raw = retry_mailbox(lambda timeout: self.transport.get(self.result_url, timeout), deadline=retry_end)
finally: