Commit Graph
19 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Fable 5.1 0d7d3f0dc9 chore: format the whole tree and lint the scratch measurement scripts; CI has been red since 25ae4b4
CI runs ruff format --check and ruff check over the whole repository; 29 files had drifted from the formatter and
four measurement scripts carried lint findings, so every push since 25ae4b4 failed before pytest ran.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-14 02:49:44 +02:00
Marius CiepluchandClaude Fable 5.1 f63703a604 chore(p0): drop never-enabled hunter tools, test-only functions and receiver delivery
Off every active line (PATHFINDER-2026-10-13/03-unified-proposal.md, cuts).
Deleted: the four hunter tools no caller enables (read_definition,
entry_points, flows, obligations) with their schemas and dispatch, the dead
mapping branch of find_refs; 20 functions whose only caller was one test
(mapping task records, compute_ranking_metrics, build_quick_hunter_prompt,
validate_evidence_transition, should_fuse, harness processors,
rank_targets_with_model, model/report slice rendering, taxonomy
hierarchical/related, config_value.verdict, taint.verdict, roles.load_roles,
features.language_of and build_for_scan, folds.leave_one_*,
features.recorded_case, search BatchExecutor and the three __main__ shims,
delivery.put_input); the receiver delivery mode of the plane. push_inputs.py
moved to benchmarks/push. memory.purge stays: the live-S3 test cleanup uses
it. Net: 2,730 lines deleted, 207 added. 473 targeted tests passed; 3,282
tests collect. breadth.toml records the removals.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-13 11:06:09 +02:00
Marius CiepluchandClaude Opus 4.8 6c7d6bb1d9 search-with-proof: path oracle covers the prefix-check-bypass class (sibling canary)
The path oracle only modelled classic parent traversal (a `canary` file in the parent of
${served_root}). A common real path-traversal class is the prefix-check bypass:
startswith(base) without a trailing separator, which the fix closes with base + os.sep;
it reaches a SIBLING whose path string-extends the served directory, not the parent.
PathOracle now also plants a sibling canary `public_leak` beside the served `public`
(same nonce, 0600), so a prefix-bypass demo leaks it on the vulnerable side and the fix
blocks it. ORACLE_GUIDE names the second target and tells the agent to pick the vector
its code shows. One shared nonce stays correct: a canary leaks only when the demo's own
request reaches it, and each demo targets one vector whose fix blocks it on the fixed side.

observe()/capture()/the differential/SideRecord are UNCHANGED. Local glance ceiling proof
still demonstrates (parent canary, 3/3 vuln, 0/3 fixed, control on both sides). Gate:
worker+control+verify suites 106 passed / 4 socket-skips; ruff clean; mypy clean on oracles.py.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 11:24:31 +02:00
Marius CiepluchandClaude Opus 4.8 00d1c26f5f search-with-proof: require the path demo to serve ${served_root} (via arg/env/config)
Path searches authored and verified many demos but all came back with no differential:
the demo never pointed the app's served directory at ${served_root}, so the observable
(which lives relative to ${served_root}) was outside the served tree and unreachable. The
agent had set an app env var to a value of its own; the guide only showed the --dir form.

ORACLE_GUIDE now states the agent MUST set the app's served/root to the ${served_root}
placeholder via whichever mechanism the app exposes (command-line argument, environment
variable, or config written to start.files), not only --dir, and that serving elsewhere
makes the observable unreachable. Verify-time-only clarification, observable names, and all
prohibitions are kept. Prompt strings only.

Tests: tests/test_search_worker.py 37 passed; ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 10:29:40 +02:00
Marius CiepluchandClaude Opus 4.8 563d006c77 search-with-proof: tell the agent the oracle observable is verify-time-only (fix the explore-time reachability trap)
The parity fix named each oracle's observable, but path searches then spent their
whole budget without authoring a demo: the agent tried to confirm reachability during
exploration, where the oracle fixture does not exist (it is injected only in the
verifier's sandbox at verify time), concluded "unreachable", and never committed a demo.

ORACLE_GUIDE now states the fixture/observable is injected at VERIFY time only and is
absent during exploration; its explore-time absence never means unreachable or justifies
ending the search. The agent confirms the mechanism (sink reachability, access to data the
app should not expose) from app code and its own sandbox test data, targets the observable's
known verify-time location in the demo, and relies on differential verification as the only
proof. All prior prohibitions (no out-of-band/off-disk access, no input capture, no revision
sniffing) and the parity observable names are kept. Adjacent wording shortened to fit.

Tests: tests/test_search_worker.py 37 passed; ruff clean; mypy unchanged (pre-existing
verify.py lambda note only). Prompt strings only; validators untouched.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 07:43:13 +02:00
Marius CiepluchandClaude Opus 4.8 dc307c7d2e search-with-proof: path-oracle agent guidance to parity with command/ssrf (name the observable)
The path family's agent guidance named no target, while command (${marker_dir}/marker)
and ssrf (${callback_url}) disclose theirs, so path searches could not author a demo
that surfaces the observable through the app. ORACLE_GUIDE now names it (canary in the
parent of ${served_root}, matching PathOracle.prepare). The reason/explore constraints
are refined to forbid only out-of-band access (direct disk reads, non-app channels),
input capture, and revision sniffing, while allowing the app to disclose it through the
vulnerability under test. Prompt strings only; the demo validator and its _PRIVATE guard
are untouched, so direct references to verifier paths remain blocked.

Tests: tests/test_search_worker.py 37 passed; ruff clean; mypy unchanged (pre-existing
verify.py lambda note only).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 18:06:00 +02:00
Marius CiepluchandClaude Fable 5.1 7a91cc1482 search-with-proof: oracle placeholders for demos; Python apps run under the venv interpreter
Demos may reference ${served_root}, ${database_path}, ${marker_dir} or
${callback_url}; the verifier substitutes them per side, so apps that take a
directory or database argument can be wired to the oracle. Python apps start
under the verifier interpreter (the one pip built the products for): the image
carries a second system Python whose ABI tag broke compiled extensions.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 12:43:11 +02:00
Marius CiepluchandClaude Fable 5.1 da804472b9 search-with-proof: readiness from the budget with a probe, 4096-token tool turns, 40-step tasks
HTTP start waits up to 90 s (CLI 30 s) with TCP then HTTP probing; app stdout
and stderr tails are recorded; tool-call turns get 4096 output tokens with one
retry on truncation; tasks get 40 steps with reminders.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 09:37:47 +02:00
Marius CiepluchandClaude Fable 5.1 72ab4c8925 search-with-proof: 1 GiB spooled verifier archive, tolerant text replies, transport retries
Built projects spool to /workspace without package caches (cap 1 GiB, both
sides); a plain-text model reply is fed back once; mailbox PUT/GET retry for up
to 90 s with idempotent re-send by seq; executor tasks are deleted on every exit
path.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 08:18:34 +02:00
Marius CiepluchandClaude Fable 5.1 ceeaa66717 search-with-proof: startup environment and data files, dead-end dedupe, task_spend continues
Demos may set a bounded environment (loader variables denied) and up to 8 small
data files under .demo/; the reason step sees conclusions and may not re-propose
them; a task spend limit concludes its intent, not the search; finish wins over
stray tool calls.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-06 00:24:26 +02:00
Marius CiepluchandClaude Fable 5.1 3866e2771b search-with-proof: ecosystem-aware build recipes and stderr in build failures
The agent passes a manifest file or project directory; pip uses -r for
requirements files and the directory for pyproject/setup, npm ci/install runs in
the directory, composer/maven/gradle likewise. Build and verify failures carry
phase, exit code and a sanitised stderr tail.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:37:56 +02:00
Marius CiepluchandClaude Fable 5.1 bf243e6680 search-with-proof: family-to-oracle mapping, build memory, tolerant tool calls
Demos declare their oracle within the family allowed set (injection: sql or
command; output_encoding: xss; untrusted_destination: ssrf; path: path);
builds and app starts use the search memory budget instead of the 256 MB tool
limit (npm died with SIGTRAP); malformed tool-call JSON is fed back once.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 21:07:58 +02:00
Marius CiepluchandClaude Fable 5.1 c33b053b15 search-with-proof: name the exhausted limit, keep partial evidence, tighter reservation
budget_spent carries end_detail (spend, task_spend, reason_rounds, ...); a task
whose allowance cannot cover the next call finishes with its facts; per-task
metrics in pilot records; input reservation bytes/2 + 1024 per message (measured
3-4x over-reservation before).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-10-05 20:57:56 +02:00
Marius CiepluchandClaude Opus 5.5 e39a4882b0 search-with-proof: bootstrap intent, real reason/explore contracts, visible refusals
First live run: the reason step got ~190 tokens and returned {} -> silent
no_evidence. Now: a bootstrap intent reads the candidate and its entry point; the
reason prompt states the goal, family oracles, demo schema and must propose
intents when none are open; empty replies retry once then end as reason_refused;
failure records keep messages.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 19:18:12 +02:00
Marius CiepluchandClaude Opus 5.5 87b82564b4 search-with-proof: pilot driver (6.2)
One side per run over selected pilot pairs: board, brain on the VM with the
project DeepSeek client, executor tasks on kube-ax, verification on the VM lane
(AX verify lane switchable). Hard run ceiling with reservation before every call,
per-search and step caps; fixed side never sees the vulnerable revision; records
carry digests only. Dry run with the stub model reaches demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 18:58:24 +02:00
Marius CiepluchandClaude Opus 5.5 941fbb5fb0 search-with-proof: diagnosable failures, pip install race, no duplicate tasks
Reasons carry type plus a sanitised message tail; command failures carry phase,
exit code and bounded stderr. Scratch walk tolerates directories pip removes
mid-walk; writable /workspace/home and runtime paths. Capacity retries delete a
possibly created task before resubmitting. Docker smoke checks the image first.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:58:27 +02:00
Marius CiepluchandClaude Opus 5.5 85f4f0b3d4 search-with-proof: declarative demos, brain/executor split, kube-ax constraints
Demos are data the verifier executes (fixed build recipes, runtime+path start,
HTTP/CLI steps with captures; the canary is never referable). The agent loop with
the key never runs repository code: executor tasks without a key exchange commands
and results through presigned objects. AX capacity errors are back-pressure with at
most 2 tasks in flight; caches and builds under /workspace with a 2 GiB guard;
verifiers take one presigned archive per side. Launcher fix: private /dev in bwrap.
190 tests on the host; probe 5/5 demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:12:46 +02:00
Marius CiepluchandClaude Opus 5.5 03857c9bec search-with-proof: root-no-userns isolation mode for gVisor
The kube-ax probe showed bwrap cannot map uids in gVisor as an unprivileged user.
As root (gVisor is the outer boundary) the sandbox uses pid/ipc/uts/net/mount
namespaces without a user namespace and drops the artefact to its own uid via
setpriv; the mode is recorded. The probe reports each namespace kind separately.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 15:00:25 +02:00
Marius CiepluchandClaude Opus 5.5 0f74767db9 search-with-proof: probe harness (3.2) and agent worker (group 5)
Probe runs each oracle toy pair with a real demo and two gaming demos, records
image facts, RSS and disk, fails loudly if isolation is unavailable; dispatchable
as an AX task. Worker: reason step without tools, explore tool loop whose run tool
goes through the sandbox with a scrubbed env (model key absent), spend reserved
before each call. Host probe: 5/5 real demos demonstrated, 10/10 gaming demos
inconclusive. 89 tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:45:48 +02:00