mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 04:29:27 +00:00
search: upload the executor command once, not on every poll
ObjectStoreExecutor.submit re-PUT the identical command.json on every 0.2s result poll, so a task running minutes wrote ~1,500 versions of one engine-queue/search-exec/<id>/command.json. On the versioned memory bucket that was the dominant growth: 109,715 versions, 10.3 GB (91% of the bucket). retry_mailbox already retries a lost PUT acknowledgement internally, so one upload per task is the correct delivery guarantee. Guard the PUT with an `uploaded` flag: upload once (still retried on transient failure), then poll only the result. One command.json version per task instead of ~1,500. All submit semantics preserved. 59 executor tests pass, ruff clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
6773c63634
commit
b9f7958414
@@ -583,13 +583,16 @@ class ObjectStoreExecutor:
|
||||
retry_end = end + RETRY_SECONDS
|
||||
last_error = ""
|
||||
phase = "result upload"
|
||||
uploaded = False
|
||||
while time.monotonic() < end:
|
||||
try:
|
||||
# A lost PUT acknowledgement is retried with the exact same sequence.
|
||||
phase = "command upload"
|
||||
before = time.monotonic()
|
||||
try:
|
||||
retry_mailbox(lambda timeout: self.transport.put(self.command_url, payload, timeout), deadline=retry_end)
|
||||
if not uploaded:
|
||||
# Retry lost acknowledgements internally; successful delivery needs no further PUTs.
|
||||
phase = "command upload"
|
||||
retry_mailbox(lambda timeout: self.transport.put(self.command_url, payload, timeout), deadline=retry_end)
|
||||
uploaded = True
|
||||
phase = "result read"
|
||||
raw = retry_mailbox(lambda timeout: self.transport.get(self.result_url, timeout), deadline=retry_end)
|
||||
finally:
|
||||
|
||||
Reference in New Issue
Block a user