Commit Graph
100 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 4.8 90e2400e6f unseen: filter advisory enumeration server-side (ecosystem/date/cwes)
enumerate_advisories paged the entire global GitHub reviewed-advisory
database (every ecosystem, every year, every CWE) before extraction, so
a 300-repo draft never converged -- a live run reached 21,962 advisories
in 4.5h with extraction never started.

Push the existing candidate() rejections into the query: page per
ecosystem, published>=2019-01-01, and cwes restricted to the seven
families. The cwes filter uses bare CWE numbers (89), which is what the
GitHub advisories API matches -- "CWE-89" silently returns zero. The
whole cwes filter is omitted if the taxonomy ever holds a non-exact
token, so a selectable advisory is never dropped. Journal version 3
(ecosystem:page keys) refuses incompatible older journals.

Live page-1 smoke (pip, npm): 100/100 rows carry a family CWE, none
published before 2019. Offline: 166 unseen tests pass, ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-08 08:19:30 +02:00
Marius CiepluchandClaude Opus 4.8 d7bf0ba750 unseen: sourcing robustness -- retry transient HTTP/connection errors, shrink the advisory journal
A live G0 sourcing run failed on both: a transient connection/decode error aborted
the whole enumeration (the bare except raised http_failure on the first blip), and
the advisory journal stored full raw advisory JSON and reached 1.77 GB on a near-full
host. Fix: the JSON transport now retries URLError/timeout/ConnectionError/IncompleteRead/
JSONDecodeError through the same attempt-4 backoff as HTTP 5xx/429 (fatal only after the
cap or on a genuinely non-retryable error); enumerate_advisories journals only the fields
resolution/extraction consume via source.reduce_advisory (ghsa id, published/withdrawn,
type, CWE ids, package ecosystem/name for OSV, fix-commit links), with journal_version
bumped to 2 so an old full-format journal is rejected and resume-skip stays intact.

Offline: test_unseen_source/draft/extract 133 passed; ruff + format clean; mypy unchanged
(pre-existing verify.py lambda only).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 18:23:40 +02:00
Marius CiepluchandClaude Opus 4.8 56cde9f535 spec(proof-arm-opencode): OpenCode MCP is the endpoint for PoC AND fix [skip ci]
Maintainer direction: the same agentic endpoint authors the PoC, writes a candidate
fix against the repair direction + failing PoC, and hands it to the differential
verifier, which re-runs the PoC and confirms absence. Closes the no-auto-patch gap:
the proof is the acceptance test for the fix (a wrong fix is caught), the fix stays a
developer-reviewed proposal, and the green proof is the objective signal. Specced
alongside proof authoring, after qualification.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:34:52 +02:00
Marius CiepluchandClaude Opus 4.8 7f8a492857 docs: use-case reference -- catch, prove, fix, re-verify absence (+ GitHub Actions example) [skip ci]
The loop: catch a regression in the diff, prove it with a reproducible check,
the dev/agent fixes it (tool gives a repair direction + the failing proof, not an
auto-patch), then the same proof re-runs and no longer triggers = absence
demonstrated by execution; the proof becomes a committed regression test. Example
GitHub Actions workflow uses the real 'ousast pre-push . --base --head --deadline
--mode --artifact' entry; PR-comment formatter marked as a placeholder. Honest
status separates what exists (re-verify by construction, --base/--head) from
delivery work (PR formatter, auto-landing the regression test) and the open
large-repo latency.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:34:06 +02:00
Marius CiepluchandClaude Opus 4.8 dfc0956086 docs: frame the proof as prioritization evidence for busy devs [skip ci]
Revise the developer guide for engineers with full backlogs and no spare schedule:
a vague warning dies in the backlog, so the value of the demonstrated proof is the
prioritization conversation -- reproducible evidence (input, behavior, the one-line
fix difference) a dev takes to a product owner or the business to win capacity,
dropped into the ticket system as a location, repair direction, and reproduction.
mcp-shift-left-agent brief: the agent's return should be ticket-ready evidence.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:30:14 +02:00
Marius CiepluchandClaude Opus 4.8 c49f334651 docs: developer-facing pre-push security guide + mcp-shift-left-agent brief [skip ci]
docs/pre-push-for-developers.md: COBESY-structured guide for engineers who are
strong at dev but not security. Leads with the conclusion (zero-setup, no
annotations, diff-scoped, proof-backed), frames the agent-era case (a pre-push
check is how you oversee coding agents), states control (advisory vs blocking,
budget, lane, models), and is honest about limits with an explicit disproof
condition (coverage, large-repo latency, generalization being measured by G0).

.kiro/specs/mcp-shift-left-agent/: follow-up brief for an MCP agent + skills that
give devs and coding agents early shift-left feedback and a required pre-push
security step for agent oversight; builds on the existing ousast mcp server;
sequenced after the safety net is qualified.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:28:49 +02:00
Marius CiepluchandClaude Opus 4.8 7bfe6fff7e merge(search-with-proof): verification-architecture doc + proof-arm-opencode brief [skip ci]
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 16:06:56 +02:00
Marius CiepluchandClaude Opus 4.8 60617a92ba docs(search-with-proof): verification architecture + proof-arm-opencode follow-up brief
verification-architecture.md documents the existing two-stage design: the Cairn-inspired
search (brain + model calls on the VM) and the DAST proof (gVisor AX tasks in kube-ax),
the trust boundary (secrets VM-only, cluster tasks secret-free with restricted egress),
and the three verify lanes (ax=cluster default, vm=local Bubblewrap for $0 proofs,
docker=replay). Clarifies the proof is dynamic, not abstract interpretation: AI triage
stays the static backbone, proof earns the block by execution.

proof-arm-opencode/ is a follow-up brief: drive proof-authoring with OpenCode MCP, an
abliterated/GLM proof model (DeepSeek stays the search model), DSPy-optimized from
OpenObserve traces, to beat the safety-classifier refusals and nondeterministic wiring.
Flags the trust-domain decision that must come first (A: scoped key + egress inside the
cluster; B: brain on VM, only DAST in gVisor — recommended to start). Credentials live in
.env by name only (ABLITERATION_*, OPENOBSERVE_*); no secret value is in any tracked file.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 16:06:56 +02:00
Marius CiepluchandClaude Opus 4.8 b1a050e16b merge(search-with-proof): reproducible npm builds via --before commit date [skip ci]
Lockless old npm projects resolve era-appropriate deps (fixes glance filed->mime@4 crash);
npm ci untouched; git-failure falls back to plain install.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 13:37:23 +02:00
Marius CiepluchandClaude Opus 4.8 c30dd9126a search-with-proof: reproducible npm builds for old lockless projects (--before commit date)
A lockless npm project (e.g. glance, 2015) resolved an unpinned transitive range to the
latest major ([email protected] -> mime@4, which dropped .lookup()), so the built app crashed
serving files -- a build-reproducibility defect, not an app or agent fault. The executor
now, for the npm-install (no-lock) case only and only when .git is present, resolves
dependencies as of the checkout's HEAD commit date via `npm install --before <date>`,
giving era-appropriate, compatible versions. npm ci (lockfile present) is untouched, and
any git/date failure falls back to plain install. demo.py also recognizes
npm-shrinkwrap.json as a lock for the ci-vs-install choice.

Tests: executor + smoke 75 passed, demo 107 passed; ruff clean; mypy clean on executor.py.
Executor-side change -> image rebuild required.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 13:37:23 +02:00
Marius CiepluchandClaude Opus 4.8 8de0c2732b spec+docs(search-with-proof): record first cluster demonstration + fixed-side negative [skip ci]
Feed the milestone back into the spec and steering. HANDOFF: the open "first demonstrated"
question is RESOLVED -- xiaomusic (path, prefix-check-bypass) demonstrated on the vulnerable
side (3/3 affected, 0/3 safe) and the blind fixed-side search produced 0 false proofs; pair
total $1.22. Records the six-fix chain that enabled it (parity, verify-time-only, fair-share
budget, served-root guidance, oracle prefix-bypass coverage, control-signal feedback) plus the
keystone. tasks.md: implementation note; pilot (6.2/6.3) in progress. spec.json: pilot status.
safety-net steering: proof-based BLOCK mechanism reached first execution-backed demonstration
(feasibility-pilot scope, not release readiness; pilot exit + 73-negative qualification open;
classifier stays triage). search README: status note. Payload-free; demos/boards stay gitignored.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 13:25:52 +02:00
Marius CiepluchandClaude Opus 4.8 ecd1372568 measurements: FIRST cluster demonstration (xiaomusic prefix-bypass path traversal) [skip ci]
search-with-proof produced its first demonstrated finding on a real CVE fix-pair end to
end on kube-ax: vuln observes, fixed does not, 0 false proofs, $0.37, 55 calls. Enabled
by the full session fix chain (parity, verify-time-only, fair-share budget, served-root
guidance, oracle prefix-bypass coverage, control-signal feedback). Payload-free record.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 12:41:01 +02:00
Marius CiepluchandClaude Opus 4.8 1ec4ee4c55 merge(search-with-proof): served-root control-signal feedback to the agent [skip ci]
Non-blocking advisory hint when the control shows the app didn't serve ${served_root},
so the agent self-corrects its served-directory wiring. Host-side; verdicts unchanged.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 12:03:45 +02:00
Marius CiepluchandClaude Opus 4.8 20c5742d84 search-with-proof: feed the control signal back to the agent (served-root hint)
When a path demo does not serve ${served_root}, the keystone control fails
(SideRecord.control_observed False) but the search only reported a generic
inconclusive, so the agent kept making the same wiring mistake. The pilot now
surfaces the affected side's control_observed in the verify result, and the
coordinator appends one advisory, deduplicated "served-root" hint (visible in the
board snapshot the agent reads) telling it to set the served-directory config to the
${served_root} placeholder. Advisory only: verdicts, _end, the oracle, the differential
and the demo validator are unchanged; the hint fires only when control_observed is False.

Tests: coordinator + pilot_run suites 78 passed; ruff clean; mypy unchanged (pre-existing
benchmarks errors only). Host-side change, no image rebuild.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 12:03:45 +02:00
Marius CiepluchandClaude Opus 4.8 d1fe78987b measurements: xiaomusic vuln search -- oracle fix landed (agent targets prefix-sibling) but served-dir wiring missed this run [skip ci]
Oracle broadening works in the agent: demo now targets the prefix-sibling vector. Still
inconclusive (0 false proofs, $0.29): the agent set the served-dir env var to its own
value, not ${served_root}, so the sibling canary was outside the served tree. Served-root
wiring is nondeterministic (a prior run wired it). Last-mile agent-reliability gap.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 11:52:19 +02:00
Marius CiepluchandClaude Opus 4.8 84509fbdc0 merge(search-with-proof): path oracle covers prefix-check-bypass class [skip ci]
Plant a sibling canary so prefix-bypass path traversals (startswith without separator)
are demonstrable; parent canary + differential unchanged; glance still demonstrates.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 11:24:31 +02:00
Marius CiepluchandClaude Opus 4.8 6c7d6bb1d9 search-with-proof: path oracle covers the prefix-check-bypass class (sibling canary)
The path oracle only modelled classic parent traversal (a `canary` file in the parent of
${served_root}). A common real path-traversal class is the prefix-check bypass:
startswith(base) without a trailing separator, which the fix closes with base + os.sep;
it reaches a SIBLING whose path string-extends the served directory, not the parent.
PathOracle now also plants a sibling canary `public_leak` beside the served `public`
(same nonce, 0600), so a prefix-bypass demo leaks it on the vulnerable side and the fix
blocks it. ORACLE_GUIDE names the second target and tells the agent to pick the vector
its code shows. One shared nonce stays correct: a canary leaks only when the demo's own
request reaches it, and each demo targets one vector whose fix blocks it on the fixed side.

observe()/capture()/the differential/SideRecord are UNCHANGED. Local glance ceiling proof
still demonstrates (parent canary, 3/3 vuln, 0/3 fixed, control on both sides). Gate:
worker+control+verify suites 106 passed / 4 socket-skips; ruff clean; mypy clean on oracles.py.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 11:24:31 +02:00
Marius CiepluchandClaude Opus 4.8 4c356caa21 measurements: py re-validation 2 -- served-root fix works (demo now wires ${served_root}); still inconclusive, budget-bound [skip ci]
Served-root guidance validated: authored demo now references ${served_root}. Still
inconclusive (0 false proofs); agent spread budget across vectors and hit run_ceiling
(164 calls, $1.47/$1.50) without converging, and the committed traversal did not
differential. Remaining = per-app exploit-authoring + convergence (the layer the $0 local
glance proof did by hand).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 11:05:07 +02:00
Marius CiepluchandClaude Opus 4.8 ec09a39deb merge(search-with-proof): path demo must serve ${served_root} [skip ci]
Agent guidance: point the app's served directory at the ${served_root} placeholder via
arg/env/config so the planted observable is reachable. Prompt strings only; 37 worker tests pass.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 10:29:40 +02:00
Marius CiepluchandClaude Opus 4.8 00d1c26f5f search-with-proof: require the path demo to serve ${served_root} (via arg/env/config)
Path searches authored and verified many demos but all came back with no differential:
the demo never pointed the app's served directory at ${served_root}, so the observable
(which lives relative to ${served_root}) was outside the served tree and unreachable. The
agent had set an app env var to a value of its own; the guide only showed the --dir form.

ORACLE_GUIDE now states the agent MUST set the app's served/root to the ${served_root}
placeholder via whichever mechanism the app exposes (command-line argument, environment
variable, or config written to start.files), not only --dir, and that serving elsewhere
makes the observable unreachable. Verify-time-only clarification, observable names, and all
prohibitions are kept. Prompt strings only.

Tests: tests/test_search_worker.py 37 passed; ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 10:29:40 +02:00
Marius CiepluchandClaude Opus 4.8 e387ca16a2 measurements: py re-validation -- budget fix works (18 verify tasks, 0 before); demo didn't wire served_root [skip ci]
Fair-share budget validated: the Python agent completed the full loop and verified 18
demos within $0.74 of $1.50 (0 verify tasks before). Still inconclusive, 0 false proofs.
Root cause of no-differential: the authored demo never references ${served_root}; the
agent pointed the app's served dir at its own value, so the observable was outside the
served tree. Next: guidance to wire the served dir to ${served_root} for the path oracle.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 09:23:33 +02:00
Marius CiepluchandClaude Opus 4.8 f3ce5be8c4 merge(search-with-proof): per-search fair-share pilot budget [skip ci]
Each pilot search gets a fair share of --ceiling-usd with settled rollover, so one
expensive search can no longer starve the others; total stays within the ceiling.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 08:35:24 +02:00
Marius CiepluchandClaude Opus 4.8 2c0c45fd1f search-with-proof: per-search fair-share pilot budget (stop one search starving the rest)
--ceiling-usd was one shared SpendBudget passed to every search, so an expensive
search drained the pool and starved later ones: in the last 2-pair/$2 run the first
search spent $1.60 and the second hit the ceiling at $0.34 after finishing its analysis.
The driver now allocates a dynamic fair share per search (remaining_ceiling /
searches_not_yet_started), rolls unused settled budget forward, caps each search at its
share, and keeps total settled <= ceiling. A search now gets room to author and verify
after it finds the path.

Driver-only change (benchmarks/search/pilot_run.py) plus tests. tests/test_search_pilot_run.py
39 passed; ruff clean; mypy unchanged (pre-existing benchmarks errors only).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 08:35:24 +02:00
Marius CiepluchandClaude Opus 4.8 e01fb513f2 measurements: validation of explore-trap fix (ts now authors a demo; py analysis complete but out of budget) [skip ci]
2 path searches (ts/py), vulnerable, cluster, $1.95 of $2. The trap fix is validated
behaviourally: ts pair 9 now authors a demo (verify_tasks 0->2; fails could_not_build on
entrypoint), py pair 13 drops 118->36 calls and its board shows a complete route-to-sink
+ entrypoint trace but hits the per-run budget ceiling before authoring/verifying. Still
0 demonstrations, 0 false proofs. Payload-free aggregate; raw stays gitignored. Next
levers: per-run budget/step allocation for demo-write+verify after analysis; demo
entrypoint/build (start.path) for compiled/served apps.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 08:28:37 +02:00
Marius CiepluchandClaude Opus 4.8 e8e6637833 merge(search-with-proof): verify-time-only observable guidance (fix explore-time reachability trap) [skip ci]
Agent guidance so path searches stop trying to confirm the oracle observable during
exploration (where it is absent) and instead author a demo for the differential verifier.
Prompt strings only; validators untouched; 37 worker tests pass.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 07:43:29 +02:00
Marius CiepluchandClaude Opus 4.8 563d006c77 search-with-proof: tell the agent the oracle observable is verify-time-only (fix the explore-time reachability trap)
The parity fix named each oracle's observable, but path searches then spent their
whole budget without authoring a demo: the agent tried to confirm reachability during
exploration, where the oracle fixture does not exist (it is injected only in the
verifier's sandbox at verify time), concluded "unreachable", and never committed a demo.

ORACLE_GUIDE now states the fixture/observable is injected at VERIFY time only and is
absent during exploration; its explore-time absence never means unreachable or justifies
ending the search. The agent confirms the mechanism (sink reachability, access to data the
app should not expose) from app code and its own sandbox test data, targets the observable's
known verify-time location in the demo, and relies on differential verification as the only
proof. All prior prohibitions (no out-of-band/off-disk access, no input capture, no revision
sniffing) and the parity observable names are kept. Adjacent wording shortened to fit.

Tests: tests/test_search_worker.py 37 passed; ruff clean; mypy unchanged (pre-existing
verify.py lambda note only). Prompt strings only; validators untouched.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 07:43:13 +02:00
Marius CiepluchandClaude Opus 4.8 7c220307d4 measurements: path-family pilot after keystone + target-parity (0 demos, parity validated) [skip ci]
3 path searches (php/ts/py), vulnerable side, cluster verify lane, $2.95 of $3.
Payload-free aggregate; raw boards/demos stay gitignored. Finding: the parity fix
produced directed, target-aware search (the agent reasons about reaching the named
observable and inspects the app's serving guard), but no demonstration landed -- all
three spent their per-pair allowance before a differential, and two wrote zero verify
demos (budget consumed in reason/explore). Next levers: allowance/budget split toward
demo-writing, entrypoint discovery for php, and the explore-vs-verify path layout the
py agent flagged.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 19:35:42 +02:00
Marius CiepluchandClaude Opus 4.8 dc307c7d2e search-with-proof: path-oracle agent guidance to parity with command/ssrf (name the observable)
The path family's agent guidance named no target, while command (${marker_dir}/marker)
and ssrf (${callback_url}) disclose theirs, so path searches could not author a demo
that surfaces the observable through the app. ORACLE_GUIDE now names it (canary in the
parent of ${served_root}, matching PathOracle.prepare). The reason/explore constraints
are refined to forbid only out-of-band access (direct disk reads, non-app channels),
input capture, and revision sniffing, while allowing the app to disclose it through the
vulnerability under test. Prompt strings only; the demo validator and its _PRIVATE guard
are untouched, so direct references to verifier paths remain blocked.

Tests: tests/test_search_worker.py 37 passed; ruff clean; mypy unchanged (pre-existing
verify.py lambda note only).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 18:06:00 +02:00
Marius CiepluchandClaude Fable 5.1 70553cb7c6 measurements: first search-with-proof demonstrated (glance ceiling proof, $0 local) [skip ci]
Outcome-only record of the first end-to-end demonstrated verdict and the two
masking defects (regenerated npm lock pinning an incompatible mime; agent demo
targeting the wrong file). Payloads stay in the gitignored private area.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 17:54:47 +02:00
Marius CiepluchandClaude Fable 5.1 de2771997a merge(search-with-proof): positive control for path oracle [skip ci]
Honest serve signal for the HTTP verifier: SideRecord.control_observed records
whether a benign verifier-planted in-root marker came back through the app's
interface, as its own guarded app start that never changes the verdict. A
2xx-based block was rejected (any HTTP status counts as alive). Gate green;
ceiling re-proof on glance still demonstrated with control_observed on both
sides. The search task image needs a rebuild before the next cluster pilot.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 17:54:47 +02:00
Marius CiepluchandClaude Opus 4.8 328406ad41 search-with-proof: positive control for path oracle — honest serve signal, no false negatives
The HTTP verifier could not distinguish a mis-wired or non-serving app from a
genuine no-effect: both yielded `inconclusive`. This adds a positive control for
the path oracle.

- PathOracle plants a benign in-root marker file (random token from os.urandom,
  distinct from the secret canary) and exposes it via a new Oracle.control()
  hook (default None; only the path oracle implements it today).
- On the first run of an HTTP side, _side runs a benign GET for that marker as
  its own fully-guarded app start. The marker returning proves the interface
  serves the fixture; its absence — or any control failure — is recorded in the
  new SideRecord.control_observed field but never changes the verdict, since a
  valid app may answer only its own routes. A crash is still caught by the
  existing could_not_run path.

A 2xx-based "nothing served" block was rejected: the verifier deliberately
treats any HTTP status as the app being alive (a 503-only app is a valid
observed side), so status-based blocking would false-fail valid apps. Soundly
converting a mis-wired-but-responding app into could_not_run requires a
demo-declared control, which is a separate follow-up.

Tests: new tests/test_search_control.py covers marker observed, absent marker
not blocking, control failure not blocking a working attack, and crash still
could_not_run. Full search-verify/probe/worker suites pass; ruff clean; mypy
unchanged (only the pre-existing file_bindings lambda note).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 17:07:24 +02:00
Marius CiepluchandClaude Opus 4.8 d17ceb0e0c spec(search-with-proof): handoff -- plumbing cleared, open question is demo attack step vs oracle capture [skip ci]
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 15:14:45 +02:00
Marius CiepluchandClaude Fable 5.1 918b1f5eec tests: a timed-out AX attempt may carry its reason instead of None [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 14:51:07 +02:00
Marius CiepluchandClaude Fable 5.1 af6a68691c merge(search-with-proof): demo replay tool [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 14:50:14 +02:00
Marius CiepluchandClaude Fable 5.1 eb05ed18da search-with-proof: no-model demo replay (executor + verifier on a stored demo)
Replays a stored demo through the executor task and the verifier lane for one
side or both, with per-phase timings, so verifier changes are checked on the
exact image before any paid search.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 14:50:14 +02:00
Marius CiepluchandClaude Fable 5.1 cc2d0b3d1f merge(search-with-proof): one interpreter, sys.path root, node_modules path [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 14:40:15 +02:00
Marius CiepluchandClaude Fable 5.1 630603ee28 search-with-proof: one interpreter for pip and apps; checkout root on sys.path; real node_modules path
The pip recipe ran under the system Python 3.13 while apps started under the
venv 3.12, so compiled extensions never loaded. Both now use sys.executable.
Package-style starts see the checkout root; NODE_PATH lists the project
node_modules under the checkout before the products mount.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 14:37:58 +02:00
Marius CiepluchandClaude Fable 5.1 b227e74da8 merge(search-with-proof): oracle placeholders and venv interpreter [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 12:43:11 +02:00
Marius CiepluchandClaude Fable 5.1 7a91cc1482 search-with-proof: oracle placeholders for demos; Python apps run under the venv interpreter
Demos may reference ${served_root}, ${database_path}, ${marker_dir} or
${callback_url}; the verifier substitutes them per side, so apps that take a
directory or database argument can be wired to the oracle. Python apps start
under the verifier interpreter (the one pip built the products for): the image
carries a second system Python whose ABI tag broke compiled extensions.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 12:43:11 +02:00
Marius CiepluchandClaude Fable 5.1 dcbaec0043 merge(search-with-proof): archive parts, isolated pip, verifier module paths [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 11:15:09 +02:00
Marius CiepluchandClaude Fable 5.1 0d3e6df31c search-with-proof: split compressed verifier archives, isolated pip installs, verifier module paths
Archives are gzipped and uploaded in parts of at most 90 MB with per-part
digests (edge request limit); pip installs ignore the task venv so transitive
dependencies land in products; the verifier sets NODE_PATH and the Java
classpath from products when starting the app.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 11:15:09 +02:00
Marius Ciepluch c9bd1ec152 merge(search-with-proof): readiness probe, longer tool turns, 40-step tasks [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>

# Conflicts:
#	benchmarks/ax/search_verify.py
#	benchmarks/search/pilot_run.py
2026-10-06 09:41:57 +02:00
Marius CiepluchandClaude Fable 5.1 da804472b9 search-with-proof: readiness from the budget with a probe, 4096-token tool turns, 40-step tasks
HTTP start waits up to 90 s (CLI 30 s) with TCP then HTTP probing; app stdout
and stderr tails are recorded; tool-call turns get 4096 output tokens with one
retry on truncation; tasks get 40 steps with reminders.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 09:37:47 +02:00
Marius CiepluchandClaude Fable 5.1 a4ec526039 tests: verify task identities are plain ids; the prefix is the workload kind [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:21:37 +02:00
Marius CiepluchandClaude Fable 5.1 b1024b8daa ax: verify and probe tasks carry the search-verify/search-probe name prefixes [skip ci]
The kind default "pin" had produced ousast-engine-pin-search-verify-* names,
which the operator per-host egress rules do not match.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:20:40 +02:00
Marius CiepluchandClaude Fable 5.1 72ab4c8925 search-with-proof: 1 GiB spooled verifier archive, tolerant text replies, transport retries
Built projects spool to /workspace without package caches (cap 1 GiB, both
sides); a plain-text model reply is fed back once; mailbox PUT/GET retry for up
to 90 s with idempotent re-send by seq; executor tasks are deleted on every exit
path.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:18:34 +02:00
Marius CiepluchandClaude Fable 5.1 aebc834667 merge(search-with-proof): archive, text replies, transport retries [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:18:34 +02:00
Marius CiepluchandClaude Fable 5.1 dae05a7c1a merge(search-with-proof): demo startup configuration and loop fixes [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:25:57 +02:00
Marius Ciepluch 30cbc1ac76 Merge branch 'main' into search-with-proof 2026-10-06 00:24:26 +02:00
Marius CiepluchandClaude Fable 5.1 ceeaa66717 search-with-proof: startup environment and data files, dead-end dedupe, task_spend continues
Demos may set a bounded environment (loader variables denied) and up to 8 small
data files under .demo/; the reason step sees conclusions and may not re-propose
them; a task spend limit concludes its intent, not the search; finish wins over
stray tool calls.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:24:26 +02:00
Marius CiepluchandClaude Fable 5.1 263be30979 search-with-proof 3.2: oracle probe passes on kube-ax in task-boundary mode [skip ci]
All five oracles demonstrated, gaming demos inconclusive, 38 s, ~500 MB RSS.
The probe dispatcher accepts a skipped isolation check in task-boundary mode.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:14:51 +02:00
Marius CiepluchandClaude Fable 5.1 81cb02cd35 merge(search-with-proof): build recipes and stderr-bearing failures [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:39:39 +02:00
Marius Ciepluch 67eb629c16 Merge branch 'main' into search-with-proof 2026-10-05 21:37:56 +02:00
Marius CiepluchandClaude Fable 5.1 3866e2771b search-with-proof: ecosystem-aware build recipes and stderr in build failures
The agent passes a manifest file or project directory; pip uses -r for
requirements files and the directory for pyproject/setup, npm ci/install runs in
the directory, composer/maven/gradle likewise. Build and verify failures carry
phase, exit code and a sanitised stderr tail.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:37:56 +02:00
Marius CiepluchandClaude Fable 5.1 13a20008e7 merge(probe-task-boundary): probe runs oracle self-tests in task-boundary mode [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:17:00 +02:00
Marius Ciepluch 007415c151 Merge branch 'main' into probe-task-boundary
# Conflicts:
#	src/openultrasast/search/probe.py
2026-10-05 21:14:55 +02:00
Marius CiepluchandClaude Fable 5.1 2d3c94e33b search probe: task-boundary mode runs the oracle self-tests in-process
On kube-ax the probe detected task-boundary and then failed the bwrap preflight it
no longer needs; now it skips the preflight, runs every oracle pair through the
verifier path a task uses, and reports ok only if every real demo is
demonstrated and no gaming demo is. Fixes a double fixture-path translation.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:11:26 +02:00
Marius CiepluchandClaude Fable 5.1 a40122d1b5 merge(search-with-proof): oracle mapping, build memory, tolerant tool calls [skip ci]
Image rebuild dispatched by hand (code in the last layer, ~10 MB).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:09:08 +02:00
Marius CiepluchandClaude Fable 5.1 bf243e6680 search-with-proof: family-to-oracle mapping, build memory, tolerant tool calls
Demos declare their oracle within the family allowed set (injection: sql or
command; output_encoding: xss; untrusted_destination: ssrf; path: path);
builds and app starts use the search memory budget instead of the 256 MB tool
limit (npm died with SIGTRAP); malformed tool-call JSON is fed back once.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:07:58 +02:00
Marius CiepluchandClaude Fable 5.1 0750c6fdd7 merge(search-with-proof): named exhaustion limits and partial evidence [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:01:31 +02:00
Marius Ciepluch c190d34f60 Merge branch 'main' into search-with-proof
# Conflicts:
#	tests/test_search_task_storage.py
2026-10-05 20:58:10 +02:00
Marius CiepluchandClaude Fable 5.1 c33b053b15 search-with-proof: name the exhausted limit, keep partial evidence, tighter reservation
budget_spent carries end_detail (spend, task_spend, reason_rounds, ...); a task
whose allowance cannot cover the next call finishes with its facts; per-task
metrics in pilot records; input reservation bytes/2 + 1024 per message (measured
3-4x over-reservation before).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:57:56 +02:00
Marius CiepluchandClaude Fable 5.1 35f942f4c8 images: pin task base b696c08 (base and browser) by digest
Triggers the one intended rebuild of engine, replay and search on the shared base.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:50:16 +02:00
Marius Ciepluch b696c08446 merge(image-layering): shared task base and code-last layering [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>

# Conflicts:
#	tests/test_search_task_storage.py
2026-10-05 20:44:02 +02:00
Marius CiepluchandClaude Fable 5.1 94d6a7dd11 search executor: NPROC bound counts threads, as the kernel does [skip ci]
Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:44:02 +02:00
Marius CiepluchandClaude Fable 5.1 1b9d1ca85c images: shared task base, code-last layers, no per-push rebuilds [skip ci]
One base (Temurin 21 JDK, node/npm/tsc, php/composer, maven, sqlite, bwrap)
built only when its Dockerfile or dependencies change, plus a browser variant;
engine/search/VM images FROM base@sha256 with the code in the last layer; the
image workflow runs on dispatch, tags, Dockerfile or pin changes, with a build
cache. Executor NPROC bound relative to the current process count.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:40:13 +02:00
Marius CiepluchandClaude Fable 5.1 509c6562fe search budget: recalibrated caps after the first measured searches [skip ci]
Settled ~$0.003 per model call; worst-case reservations exhausted the $0.05 task
allowance after 2-4 calls. Now $0.40 per task, $2 per search (maintainer decision).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:35:27 +02:00
Marius CiepluchandClaude Fable 5.1 b44fb56d8d ax batch: workload-kind prefixes in task names (pass-, replay-) [skip ci]
The operator keys per-host egress rules on the task-name prefix; engine passes and
replays had none. Names stay within 49 bytes.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:33:35 +02:00
Marius Ciepluch daf95b16a1 merge(search-with-proof): cluster-unavailable handling [skip ci]
No image rebuild: VM-side changes only (operator asked to stop per-merge rebuilds).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

# Conflicts:
#	benchmarks/ax/batch.py
2026-10-05 20:27:01 +02:00
Marius CiepluchandClaude Fable 5.1 4be2063d40 search-with-proof: cluster-unavailable detection, kept messages, run-level abort
Zero running workers behind "no free workers available" now raises
ClusterUnavailable instead of waiting out the capacity deadline; CapacityDeadline
carries its message; cleanup failures no longer mask the original exception; the
pilot driver aborts the run on infrastructure failure before any model call.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:26:37 +02:00
Marius CiepluchandClaude Opus 5.5 7fb061fb69 merge(search-with-proof): search contracts
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 19:18:12 +02:00
Marius CiepluchandClaude Opus 5.5 e39a4882b0 search-with-proof: bootstrap intent, real reason/explore contracts, visible refusals
First live run: the reason step got ~190 tokens and returned {} -> silent
no_evidence. Now: a bootstrap intent reads the candidate and its entry point; the
reason prompt states the goal, family oracles, demo schema and must propose
intents when none are open; empty replies retry once then end as reason_refused;
failure records keep messages.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 19:18:12 +02:00
Marius CiepluchandClaude Opus 5.5 0fb098c478 merge(search-with-proof): pilot driver
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:58:24 +02:00
Marius CiepluchandClaude Opus 5.5 87b82564b4 search-with-proof: pilot driver (6.2)
One side per run over selected pilot pairs: board, brain on the VM with the
project DeepSeek client, executor tasks on kube-ax, verification on the VM lane
(AX verify lane switchable). Hard run ceiling with reservation before every call,
per-search and step caps; fixed side never sees the vulnerable revision; records
carry digests only. Dry run with the stub model reaches demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:58:24 +02:00
Marius CiepluchandClaude Opus 5.5 89516d4000 ax batch: fast capacity retries in the first minute (golden-snapshot window)
Operator 2026-10-05: Substrate boots a ~1 s golden-snapshot actor per new task on the
other worker; retry apply after 2-5 s for the first minute, then 15-30 s.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:07:47 +02:00
Marius CiepluchandClaude Opus 5.5 941fbb5fb0 search-with-proof: diagnosable failures, pip install race, no duplicate tasks
Reasons carry type plus a sanitised message tail; command failures carry phase,
exit code and bounded stderr. Scratch walk tolerates directories pip removes
mid-walk; writable /workspace/home and runtime paths. Capacity retries delete a
possibly created task before resubmitting. Docker smoke checks the image first.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:58:27 +02:00
Marius CiepluchandClaude Opus 5.5 7c1af9f518 merge(search-with-proof): diagnosable failures and dispatch fixes
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:58:27 +02:00
Marius CiepluchandClaude Opus 5.5 cf10f38589 search-task image: clear the base image GPG_KEY fingerprint
The official Python base sets GPG_KEY (public CPython signing fingerprint); the
executor rightly refused it as a credential-like name on kube-ax.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:42:03 +02:00
Marius CiepluchandClaude Opus 5.5 75f2134eb4 search executor: name the credential-like variables it refuses (names only)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:36:12 +02:00
Marius CiepluchandClaude Opus 5.5 1d67412e2b search-with-proof: executor smoke driver and pilot selection (6.1)
20 development pairs, one per repository, 4 per language, 5 per family; identities
in a gitignored manifest, counts and digest committed. Executor smoke drives one
executor task end to end without a model and fails loudly on an unread clone.
kube-ax operator constraints as steering.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:26:06 +02:00
Marius CiepluchandClaude Opus 5.5 1bfb729422 merge(search-with-proof): executor smoke and pilot selection
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:26:06 +02:00
Marius CiepluchandClaude Opus 5.5 85f4f0b3d4 search-with-proof: declarative demos, brain/executor split, kube-ax constraints
Demos are data the verifier executes (fixed build recipes, runtime+path start,
HTTP/CLI steps with captures; the canary is never referable). The agent loop with
the key never runs repository code: executor tasks without a key exchange commands
and results through presigned objects. AX capacity errors are back-pressure with at
most 2 tasks in flight; caches and builds under /workspace with a 2 GiB guard;
verifiers take one presigned archive per side. Launcher fix: private /dev in bwrap.
190 tests on the host; probe 5/5 demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:12:46 +02:00
Marius CiepluchandClaude Opus 5.5 ccd64757c8 merge(search-with-proof): trust-domain split and operator constraints
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:12:46 +02:00
Marius CiepluchandClaude Opus 5.5 721c4b0997 spec(search-with-proof): separate trust domains (brain with key, executor with repo code)
Replaces the accepted key-exposure exception: the key never shares a sandbox with
repository code; executor AX tasks exchange commands and results via presigned
objects. Firecracker executors only if the pilot build-failure rate requires it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 16:26:19 +02:00
Marius CiepluchandClaude Opus 5.5 5e03bdf07f spec(search-with-proof): gVisor is the cluster boundary; declarative demos; capped rotated key
kube-ax probe: AX tasks have no capabilities (no namespaces, no setuid), so no
in-task isolation. Demos become data the verifier executes; the explore key risk is
accepted and bounded by cap and rotation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 16:20:26 +02:00
Marius CiepluchandClaude Opus 5.5 e5ce630edc merge(search-with-proof): gVisor isolation mode
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 15:00:25 +02:00
Marius CiepluchandClaude Opus 5.5 03857c9bec search-with-proof: root-no-userns isolation mode for gVisor
The kube-ax probe showed bwrap cannot map uids in gVisor as an unprivileged user.
As root (gVisor is the outer boundary) the sandbox uses pid/ipc/uts/net/mount
namespaces without a user namespace and drops the artefact to its own uid via
setpriv; the mode is recorded. The probe reports each namespace kind separately.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 15:00:25 +02:00
Marius CiepluchandClaude Opus 5.5 e6871fef9a merge(search-with-proof): probe harness and agent worker
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:45:48 +02:00
Marius CiepluchandClaude Opus 5.5 0f74767db9 search-with-proof: probe harness (3.2) and agent worker (group 5)
Probe runs each oracle toy pair with a real demo and two gaming demos, records
image facts, RSS and disk, fails loudly if isolation is unavailable; dispatchable
as an AX task. Worker: reason step without tools, explore tool loop whose run tool
goes through the sandbox with a scrubbed env (model key absent), spend reserved
before each call. Host probe: 5/5 real demos demonstrated, 10/10 gaming demos
inconclusive. 89 tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:45:48 +02:00
Marius CiepluchandClaude Opus 5.5 a17a6afaf8 merge(search-with-proof): group 4 and the search-task image
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:26:43 +02:00
Marius CiepluchandClaude Opus 5.5 4b38239f68 search-with-proof group 4 + image: board, spend reservation, coordinator, search-task image
Board as immutable versioned blobs (coordinator sole writer); budget reserve/settle
before model calls; per-task and run caps; coordinator loop over an injected
executor; batch.py accepts a fixed allow-list of search variables; XSS browser
oracle; ousast-search-task image (Python, Node/TypeScript, PHP, Java 17, sqlite,
bwrap, chromium) built by CI. 103 tests on the host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:26:43 +02:00
Marius CiepluchandClaude Opus 5.5 6394ee473d merge(search-with-proof): group 2 verifier and oracles
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:09:48 +02:00
Marius CiepluchandClaude Opus 5.5 742469d671 search-with-proof group 2: verifier with owned oracles and differential
The verifier, not the artefact, observes the effect: SQL, path, command and SSRF
oracles owned by the verifier (XSS needs a browser executor, else no_oracle);
artefact exit code and output ignored; immutable checkout and dependencies;
3 runs per side; one-sided build or readiness failure is inconclusive. An
isolation failure raises IsolationUnavailable instead of reading as
could_not_build (the NPROC limit now applies inside the user namespace).
35 tests, including gaming attempts and 300 host processes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:09:48 +02:00
Marius CiepluchandClaude Opus 5.5 c74b5eaeb0 spec(learned-decision-engine): engine becomes triage where a demonstration oracle exists
Req 6.5 (maintainer decision 2026-10-05, with search-with-proof): BLOCK comes from
verified demonstrations; the engine is judged by top-K recall per push and advisory
false alarms; the pooled block gate governs only families without an oracle.
Task 17 adds the metric and re-scores existing arms at no cost.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:45:10 +02:00
Marius CiepluchandClaude Opus 5.5 9d70ef9257 spec(search-with-proof): requirements and design approved; tasks for probe and pilot
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:35:55 +02:00
Marius CiepluchandClaude Opus 5.5 c3820e184c spec(search-with-proof): requirements amended after two Codex reviews; design v2
False alarms count every visible finding; the verifier owns per-family oracles;
inconclusive on one-sided build failure; feasibility pilot separated from a
73-negative BLOCK qualification; coverage ceilings; spend reserved before calls;
key isolated from executed code; cluster model calls supersede the pool VM-only
rule for this arm only. Requirements approval reopened.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:35:18 +02:00
Marius CiepluchandClaude Opus 5.5 8bbea86cf8 spec(search-with-proof): requirements approved
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 12:01:39 +02:00
Marius CiepluchandClaude Opus 5.5 45e641f25c spec(search-with-proof): brief and requirements for review
After reviewing Cairn: detection so far classified excerpts and hit a precision
wall; this adds directed search with an objective goal (a re-verified
demonstration), a fact/intent board, and reason/explore steps as AX Tasks on
kube-ax. Design only learns from Cairn (AGPL); no code copied.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 12:01:07 +02:00
Marius CiepluchandClaude Opus 5.5 6d27c9bfcf unseen-eval: calibrated replay deadline + native agreement check, docker-lane diagnostics, proof 4.4 record; engine runner cleans up fetched sources
gVisor replays exhausted the hook's 30 s deadline in the snapshot step on a
1,725-file repository; cluster replays now use 30 s x a measured slowdown
factor, verified on a native sample. The engine runner deletes each pin's fetched
source (1.8 GB had accumulated), caps the cache at 500 MB and refuses below
1.5 GB free. Proof 4.4: two engine pins via the generic dispatcher match the VM
baseline exactly. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 08:14:57 +02:00
Marius CiepluchandClaude Opus 5.5 5dc86c2f3f merge(unseen-eval): groups 1-2 and 4-6 -- used set, sourcing with resume, AX batch dispatcher, replay task, scorer
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 20:28:15 +02:00