Commit Graph
934 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Fable 5.1 da804472b9 search-with-proof: readiness from the budget with a probe, 4096-token tool turns, 40-step tasks
HTTP start waits up to 90 s (CLI 30 s) with TCP then HTTP probing; app stdout
and stderr tails are recorded; tool-call turns get 4096 output tokens with one
retry on truncation; tasks get 40 steps with reminders.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 09:37:47 +02:00
Marius CiepluchandClaude Fable 5.1 a4ec526039 tests: verify task identities are plain ids; the prefix is the workload kind [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:21:37 +02:00
Marius CiepluchandClaude Fable 5.1 b1024b8daa ax: verify and probe tasks carry the search-verify/search-probe name prefixes [skip ci]
The kind default "pin" had produced ousast-engine-pin-search-verify-* names,
which the operator per-host egress rules do not match.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:20:40 +02:00
Marius CiepluchandClaude Fable 5.1 72ab4c8925 search-with-proof: 1 GiB spooled verifier archive, tolerant text replies, transport retries
Built projects spool to /workspace without package caches (cap 1 GiB, both
sides); a plain-text model reply is fed back once; mailbox PUT/GET retry for up
to 90 s with idempotent re-send by seq; executor tasks are deleted on every exit
path.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:18:34 +02:00
Marius CiepluchandClaude Fable 5.1 aebc834667 merge(search-with-proof): archive, text replies, transport retries [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:18:34 +02:00
Marius CiepluchandClaude Fable 5.1 dae05a7c1a merge(search-with-proof): demo startup configuration and loop fixes [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:25:57 +02:00
Marius Ciepluch 30cbc1ac76 Merge branch 'main' into search-with-proof 2026-10-06 00:24:26 +02:00
Marius CiepluchandClaude Fable 5.1 ceeaa66717 search-with-proof: startup environment and data files, dead-end dedupe, task_spend continues
Demos may set a bounded environment (loader variables denied) and up to 8 small
data files under .demo/; the reason step sees conclusions and may not re-propose
them; a task spend limit concludes its intent, not the search; finish wins over
stray tool calls.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:24:26 +02:00
Marius CiepluchandClaude Fable 5.1 263be30979 search-with-proof 3.2: oracle probe passes on kube-ax in task-boundary mode [skip ci]
All five oracles demonstrated, gaming demos inconclusive, 38 s, ~500 MB RSS.
The probe dispatcher accepts a skipped isolation check in task-boundary mode.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:14:51 +02:00
Marius CiepluchandClaude Fable 5.1 81cb02cd35 merge(search-with-proof): build recipes and stderr-bearing failures [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:39:39 +02:00
Marius Ciepluch 67eb629c16 Merge branch 'main' into search-with-proof 2026-10-05 21:37:56 +02:00
Marius CiepluchandClaude Fable 5.1 3866e2771b search-with-proof: ecosystem-aware build recipes and stderr in build failures
The agent passes a manifest file or project directory; pip uses -r for
requirements files and the directory for pyproject/setup, npm ci/install runs in
the directory, composer/maven/gradle likewise. Build and verify failures carry
phase, exit code and a sanitised stderr tail.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:37:56 +02:00
Marius CiepluchandClaude Fable 5.1 13a20008e7 merge(probe-task-boundary): probe runs oracle self-tests in task-boundary mode [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:17:00 +02:00
Marius Ciepluch 007415c151 Merge branch 'main' into probe-task-boundary
# Conflicts:
#	src/openultrasast/search/probe.py
2026-10-05 21:14:55 +02:00
Marius CiepluchandClaude Fable 5.1 2d3c94e33b search probe: task-boundary mode runs the oracle self-tests in-process
On kube-ax the probe detected task-boundary and then failed the bwrap preflight it
no longer needs; now it skips the preflight, runs every oracle pair through the
verifier path a task uses, and reports ok only if every real demo is
demonstrated and no gaming demo is. Fixes a double fixture-path translation.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:11:26 +02:00
Marius CiepluchandClaude Fable 5.1 a40122d1b5 merge(search-with-proof): oracle mapping, build memory, tolerant tool calls [skip ci]
Image rebuild dispatched by hand (code in the last layer, ~10 MB).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:09:08 +02:00
Marius CiepluchandClaude Fable 5.1 bf243e6680 search-with-proof: family-to-oracle mapping, build memory, tolerant tool calls
Demos declare their oracle within the family allowed set (injection: sql or
command; output_encoding: xss; untrusted_destination: ssrf; path: path);
builds and app starts use the search memory budget instead of the 256 MB tool
limit (npm died with SIGTRAP); malformed tool-call JSON is fed back once.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:07:58 +02:00
Marius CiepluchandClaude Fable 5.1 0750c6fdd7 merge(search-with-proof): named exhaustion limits and partial evidence [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:01:31 +02:00
Marius Ciepluch c190d34f60 Merge branch 'main' into search-with-proof
# Conflicts:
#	tests/test_search_task_storage.py
2026-10-05 20:58:10 +02:00
Marius CiepluchandClaude Fable 5.1 c33b053b15 search-with-proof: name the exhausted limit, keep partial evidence, tighter reservation
budget_spent carries end_detail (spend, task_spend, reason_rounds, ...); a task
whose allowance cannot cover the next call finishes with its facts; per-task
metrics in pilot records; input reservation bytes/2 + 1024 per message (measured
3-4x over-reservation before).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:57:56 +02:00
Marius CiepluchandClaude Fable 5.1 35f942f4c8 images: pin task base b696c08 (base and browser) by digest
Triggers the one intended rebuild of engine, replay and search on the shared base.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:50:16 +02:00
Marius Ciepluch b696c08446 merge(image-layering): shared task base and code-last layering [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>

# Conflicts:
#	tests/test_search_task_storage.py
2026-10-05 20:44:02 +02:00
Marius CiepluchandClaude Fable 5.1 94d6a7dd11 search executor: NPROC bound counts threads, as the kernel does [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:44:02 +02:00
Marius CiepluchandClaude Fable 5.1 1b9d1ca85c images: shared task base, code-last layers, no per-push rebuilds [skip ci]
One base (Temurin 21 JDK, node/npm/tsc, php/composer, maven, sqlite, bwrap)
built only when its Dockerfile or dependencies change, plus a browser variant;
engine/search/VM images FROM base@sha256 with the code in the last layer; the
image workflow runs on dispatch, tags, Dockerfile or pin changes, with a build
cache. Executor NPROC bound relative to the current process count.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:40:13 +02:00
Marius CiepluchandClaude Fable 5.1 509c6562fe search budget: recalibrated caps after the first measured searches [skip ci]
Settled ~$0.003 per model call; worst-case reservations exhausted the $0.05 task
allowance after 2-4 calls. Now $0.40 per task, $2 per search (maintainer decision).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:35:27 +02:00
Marius CiepluchandClaude Fable 5.1 b44fb56d8d ax batch: workload-kind prefixes in task names (pass-, replay-) [skip ci]
The operator keys per-host egress rules on the task-name prefix; engine passes and
replays had none. Names stay within 49 bytes.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:33:35 +02:00
Marius Ciepluch daf95b16a1 merge(search-with-proof): cluster-unavailable handling [skip ci]
No image rebuild: VM-side changes only (operator asked to stop per-merge rebuilds).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

# Conflicts:
#	benchmarks/ax/batch.py
2026-10-05 20:27:01 +02:00
Marius CiepluchandClaude Fable 5.1 4be2063d40 search-with-proof: cluster-unavailable detection, kept messages, run-level abort
Zero running workers behind "no free workers available" now raises
ClusterUnavailable instead of waiting out the capacity deadline; CapacityDeadline
carries its message; cleanup failures no longer mask the original exception; the
pilot driver aborts the run on infrastructure failure before any model call.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:26:37 +02:00
Marius CiepluchandClaude Opus 5.5 7fb061fb69 merge(search-with-proof): search contracts
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 19:18:12 +02:00
Marius CiepluchandClaude Opus 5.5 e39a4882b0 search-with-proof: bootstrap intent, real reason/explore contracts, visible refusals
First live run: the reason step got ~190 tokens and returned {} -> silent
no_evidence. Now: a bootstrap intent reads the candidate and its entry point; the
reason prompt states the goal, family oracles, demo schema and must propose
intents when none are open; empty replies retry once then end as reason_refused;
failure records keep messages.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 19:18:12 +02:00
Marius CiepluchandClaude Opus 5.5 0fb098c478 merge(search-with-proof): pilot driver
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:58:24 +02:00
Marius CiepluchandClaude Opus 5.5 87b82564b4 search-with-proof: pilot driver (6.2)
One side per run over selected pilot pairs: board, brain on the VM with the
project DeepSeek client, executor tasks on kube-ax, verification on the VM lane
(AX verify lane switchable). Hard run ceiling with reservation before every call,
per-search and step caps; fixed side never sees the vulnerable revision; records
carry digests only. Dry run with the stub model reaches demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:58:24 +02:00
Marius CiepluchandClaude Opus 5.5 89516d4000 ax batch: fast capacity retries in the first minute (golden-snapshot window)
Operator 2026-10-05: Substrate boots a ~1 s golden-snapshot actor per new task on the
other worker; retry apply after 2-5 s for the first minute, then 15-30 s.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:07:47 +02:00
Marius CiepluchandClaude Opus 5.5 941fbb5fb0 search-with-proof: diagnosable failures, pip install race, no duplicate tasks
Reasons carry type plus a sanitised message tail; command failures carry phase,
exit code and bounded stderr. Scratch walk tolerates directories pip removes
mid-walk; writable /workspace/home and runtime paths. Capacity retries delete a
possibly created task before resubmitting. Docker smoke checks the image first.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:58:27 +02:00
Marius CiepluchandClaude Opus 5.5 7c1af9f518 merge(search-with-proof): diagnosable failures and dispatch fixes
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:58:27 +02:00
Marius CiepluchandClaude Opus 5.5 cf10f38589 search-task image: clear the base image GPG_KEY fingerprint
The official Python base sets GPG_KEY (public CPython signing fingerprint); the
executor rightly refused it as a credential-like name on kube-ax.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:42:03 +02:00
Marius CiepluchandClaude Opus 5.5 75f2134eb4 search executor: name the credential-like variables it refuses (names only)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:36:12 +02:00
Marius CiepluchandClaude Opus 5.5 1d67412e2b search-with-proof: executor smoke driver and pilot selection (6.1)
20 development pairs, one per repository, 4 per language, 5 per family; identities
in a gitignored manifest, counts and digest committed. Executor smoke drives one
executor task end to end without a model and fails loudly on an unread clone.
kube-ax operator constraints as steering.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:26:06 +02:00
Marius CiepluchandClaude Opus 5.5 1bfb729422 merge(search-with-proof): executor smoke and pilot selection
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:26:06 +02:00
Marius CiepluchandClaude Opus 5.5 85f4f0b3d4 search-with-proof: declarative demos, brain/executor split, kube-ax constraints
Demos are data the verifier executes (fixed build recipes, runtime+path start,
HTTP/CLI steps with captures; the canary is never referable). The agent loop with
the key never runs repository code: executor tasks without a key exchange commands
and results through presigned objects. AX capacity errors are back-pressure with at
most 2 tasks in flight; caches and builds under /workspace with a 2 GiB guard;
verifiers take one presigned archive per side. Launcher fix: private /dev in bwrap.
190 tests on the host; probe 5/5 demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:12:46 +02:00
Marius CiepluchandClaude Opus 5.5 ccd64757c8 merge(search-with-proof): trust-domain split and operator constraints
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:12:46 +02:00
Marius CiepluchandClaude Opus 5.5 721c4b0997 spec(search-with-proof): separate trust domains (brain with key, executor with repo code)
Replaces the accepted key-exposure exception: the key never shares a sandbox with
repository code; executor AX tasks exchange commands and results via presigned
objects. Firecracker executors only if the pilot build-failure rate requires it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 16:26:19 +02:00
Marius CiepluchandClaude Opus 5.5 5e03bdf07f spec(search-with-proof): gVisor is the cluster boundary; declarative demos; capped rotated key
kube-ax probe: AX tasks have no capabilities (no namespaces, no setuid), so no
in-task isolation. Demos become data the verifier executes; the explore key risk is
accepted and bounded by cap and rotation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 16:20:26 +02:00
Marius CiepluchandClaude Opus 5.5 e5ce630edc merge(search-with-proof): gVisor isolation mode
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 15:00:25 +02:00
Marius CiepluchandClaude Opus 5.5 03857c9bec search-with-proof: root-no-userns isolation mode for gVisor
The kube-ax probe showed bwrap cannot map uids in gVisor as an unprivileged user.
As root (gVisor is the outer boundary) the sandbox uses pid/ipc/uts/net/mount
namespaces without a user namespace and drops the artefact to its own uid via
setpriv; the mode is recorded. The probe reports each namespace kind separately.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 15:00:25 +02:00
Marius CiepluchandClaude Opus 5.5 e6871fef9a merge(search-with-proof): probe harness and agent worker
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:45:48 +02:00
Marius CiepluchandClaude Opus 5.5 0f74767db9 search-with-proof: probe harness (3.2) and agent worker (group 5)
Probe runs each oracle toy pair with a real demo and two gaming demos, records
image facts, RSS and disk, fails loudly if isolation is unavailable; dispatchable
as an AX task. Worker: reason step without tools, explore tool loop whose run tool
goes through the sandbox with a scrubbed env (model key absent), spend reserved
before each call. Host probe: 5/5 real demos demonstrated, 10/10 gaming demos
inconclusive. 89 tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:45:48 +02:00
Marius CiepluchandClaude Opus 5.5 a17a6afaf8 merge(search-with-proof): group 4 and the search-task image
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:26:43 +02:00
Marius CiepluchandClaude Opus 5.5 4b38239f68 search-with-proof group 4 + image: board, spend reservation, coordinator, search-task image
Board as immutable versioned blobs (coordinator sole writer); budget reserve/settle
before model calls; per-task and run caps; coordinator loop over an injected
executor; batch.py accepts a fixed allow-list of search variables; XSS browser
oracle; ousast-search-task image (Python, Node/TypeScript, PHP, Java 17, sqlite,
bwrap, chromium) built by CI. 103 tests on the host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:26:43 +02:00
Marius CiepluchandClaude Opus 5.5 6394ee473d merge(search-with-proof): group 2 verifier and oracles
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:09:48 +02:00