Demos may reference ${served_root}, ${database_path}, ${marker_dir} or
${callback_url}; the verifier substitutes them per side, so apps that take a
directory or database argument can be wired to the oracle. Python apps start
under the verifier interpreter (the one pip built the products for): the image
carries a second system Python whose ABI tag broke compiled extensions.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Archives are gzipped and uploaded in parts of at most 90 MB with per-part
digests (edge request limit); pip installs ignore the task venv so transitive
dependencies land in products; the verifier sets NODE_PATH and the Java
classpath from products when starting the app.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
HTTP start waits up to 90 s (CLI 30 s) with TCP then HTTP probing; app stdout
and stderr tails are recorded; tool-call turns get 4096 output tokens with one
retry on truncation; tasks get 40 steps with reminders.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Built projects spool to /workspace without package caches (cap 1 GiB, both
sides); a plain-text model reply is fed back once; mailbox PUT/GET retry for up
to 90 s with idempotent re-send by seq; executor tasks are deleted on every exit
path.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Demos may set a bounded environment (loader variables denied) and up to 8 small
data files under .demo/; the reason step sees conclusions and may not re-propose
them; a task spend limit concludes its intent, not the search; finish wins over
stray tool calls.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
The agent passes a manifest file or project directory; pip uses -r for
requirements files and the directory for pyproject/setup, npm ci/install runs in
the directory, composer/maven/gradle likewise. Build and verify failures carry
phase, exit code and a sanitised stderr tail.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Demos declare their oracle within the family allowed set (injection: sql or
command; output_encoding: xss; untrusted_destination: ssrf; path: path);
builds and app starts use the search memory budget instead of the 256 MB tool
limit (npm died with SIGTRAP); malformed tool-call JSON is fed back once.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
budget_spent carries end_detail (spend, task_spend, reason_rounds, ...); a task
whose allowance cannot cover the next call finishes with its facts; per-task
metrics in pilot records; input reservation bytes/2 + 1024 per message (measured
3-4x over-reservation before).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Zero running workers behind "no free workers available" now raises
ClusterUnavailable instead of waiting out the capacity deadline; CapacityDeadline
carries its message; cleanup failures no longer mask the original exception; the
pilot driver aborts the run on infrastructure failure before any model call.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
First live run: the reason step got ~190 tokens and returned {} -> silent
no_evidence. Now: a bootstrap intent reads the candidate and its entry point; the
reason prompt states the goal, family oracles, demo schema and must propose
intents when none are open; empty replies retry once then end as reason_refused;
failure records keep messages.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
One side per run over selected pilot pairs: board, brain on the VM with the
project DeepSeek client, executor tasks on kube-ax, verification on the VM lane
(AX verify lane switchable). Hard run ceiling with reservation before every call,
per-search and step caps; fixed side never sees the vulnerable revision; records
carry digests only. Dry run with the stub model reaches demonstrated.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Reasons carry type plus a sanitised message tail; command failures carry phase,
exit code and bounded stderr. Scratch walk tolerates directories pip removes
mid-walk; writable /workspace/home and runtime paths. Capacity retries delete a
possibly created task before resubmitting. Docker smoke checks the image first.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
20 development pairs, one per repository, 4 per language, 5 per family; identities
in a gitignored manifest, counts and digest committed. Executor smoke drives one
executor task end to end without a model and fails loudly on an unread clone.
kube-ax operator constraints as steering.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Demos are data the verifier executes (fixed build recipes, runtime+path start,
HTTP/CLI steps with captures; the canary is never referable). The agent loop with
the key never runs repository code: executor tasks without a key exchange commands
and results through presigned objects. AX capacity errors are back-pressure with at
most 2 tasks in flight; caches and builds under /workspace with a 2 GiB guard;
verifiers take one presigned archive per side. Launcher fix: private /dev in bwrap.
190 tests on the host; probe 5/5 demonstrated.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The kube-ax probe showed bwrap cannot map uids in gVisor as an unprivileged user.
As root (gVisor is the outer boundary) the sandbox uses pid/ipc/uts/net/mount
namespaces without a user namespace and drops the artefact to its own uid via
setpriv; the mode is recorded. The probe reports each namespace kind separately.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Probe runs each oracle toy pair with a real demo and two gaming demos, records
image facts, RSS and disk, fails loudly if isolation is unavailable; dispatchable
as an AX task. Worker: reason step without tools, explore tool loop whose run tool
goes through the sandbox with a scrubbed env (model key absent), spend reserved
before each call. Host probe: 5/5 real demos demonstrated, 10/10 gaming demos
inconclusive. 89 tests.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Board as immutable versioned blobs (coordinator sole writer); budget reserve/settle
before model calls; per-task and run caps; coordinator loop over an injected
executor; batch.py accepts a fixed allow-list of search variables; XSS browser
oracle; ousast-search-task image (Python, Node/TypeScript, PHP, Java 17, sqlite,
bwrap, chromium) built by CI. 103 tests on the host.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The verifier, not the artefact, observes the effect: SQL, path, command and SSRF
oracles owned by the verifier (XSS needs a browser executor, else no_oracle);
artefact exit code and output ignored; immutable checkout and dependencies;
3 runs per side; one-sided build or readiness failure is inconclusive. An
isolation failure raises IsolationUnavailable instead of reading as
could_not_build (the NPROC limit now applies inside the user namespace).
35 tests, including gaming attempts and 300 host processes.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Implemented with Codex; task 1.5 run by the coordinator against the real
sources (S3 memory read-only).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Through Cloudflare, signed requests for keys ending .tar/.gz/.zip/.bin reached
RustFS altered and failed with 403 (cf-cache-status BYPASS); the same signed
request straight to RustFS on the LAN returned 200. A Cloudflare cache-bypass
rule on files.because-security.com fixes it (all extensions 200, DYNAMIC); the
engine executors keep .dat as a safeguard, the plane keeps output.tar.
Implemented with Codex.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A lean overlay omits otel-collector, prometheus and jaeger and disables the
exporters; on an existing cluster up.sh scales them to 0. Verified live: render
has no otel-system resources, up.sh idempotent (54 s second run), smoke Run
done, kind node 2.66 -> 1.82 GiB. Docs say server VM. Implemented with Codex.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Batch 3 of real (vulnerable function, fixed function) pointer pairs, mined at
zero model cost from reviewed GHSA advisories (pip, npm, maven, composer,
published since 2023, newest first). Same rules as batch 2, widened three
ways: a merged PR's single commit when the advisory links no commit, commits
up to 6 files / 240 lines, and two-family advisories with the family read
from the diff. Every candidate's function diff was read: 412 reviewed, 255
kept, 3 one-per-repository, 2 withheld by the reserved-population guard
(bisected with output discarded). Permissive licences only; no repository
named elsewhere under benchmarks/; no WordPress plugin.
Kept 251: injection 56, untrusted_destination 26, path 71, output_encoding
69, access_control 20, deserialization 9; python 83, typescript 48, php 46,
java 37, javascript 37. Only 12 post-cutoff: batch 2 had used up the
post-2026-06-01 pool. Memory 1133 -> 1635 static examples; complete pairs in
the whole memory 459 -> 710. Embeddings: 15 calls, $0.0030. No chat-model
call, no Docker, no Joern. Record:
benchmarks/measurements/2026-10-04-decision-engine-harvest-3.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The query fix stops a class-level call from aborting the whole taint batch on
real PHP code; gate, map_gate and pair_gate outputs are unchanged. Trace parsing
lives with the runner (benchmarks/learn/engine_trace_parse.py).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
On real PHP source (CPS-IT/mailqueue) a call outside any method has a null
.method; inScope/nestedInLabeled dereferenced it (NullPointerException) and the
script exited 1, losing every taint answer for the repository. 16 sites in
taint/dominance/config/vocabulary.sc now treat a null method as out of scope;
non-null behaviour is unchanged (same outputs on the repro graph for the other
scripts). Static regression test added. Implemented with Codex.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
slice.py renders source -> step -> sink lines with fixed wording for flow,
no-flow and not-analysable; the prompt option is default-off (byte-identical
default prompt). semantic IR: JS/C declarations count as assignments, augmented
assignment keeps its target (gate, map_gate, pair_gate outputs unchanged).
Audit over exp-004's 501 units: 59 flow, 161 no-flow, 281 not analysable;
has-flow within-pair accuracy 0.52, pooled AUC 0.55 -- no leak. Fixes a
numbering bug that rejected every excerpt with a blank line. With Codex.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The estimator priced all input at the full rate (\$5.58 estimated vs \$1.24
spent); it now uses the family mean billed cost of cached responses when usage
is recorded. Implemented with Codex.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
A pending-then-frozen registration left two rows and rows[0] could be the stale
pending one, so fill and run refused a properly frozen experiment. Implemented
with Codex.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Implemented with Codex; review fix: the bounded client keeps the configured
thinking setting so a fill answers as the run would.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Maintainer decision 2026-10-03: BLOCK precision is shown pooled across
families. Req 6.4 (amended; 7.4 annotated): one t_block on a per-family
Platt-normalised score (calibrate.fit_map, training folds, inner
cross-fit), Wilson 95% LB >= 0.95 over all families' training flags; a
family blocks only with >= F = 10 held-out flags and zero errors, else
advisory; t_quiet per family keeps >= 90% of training paired positives.
evaluate: normalised, pooled_threshold, quiet_threshold,
pooled_block_gate, family_floor, apply_floor. experiments: rule
pooled_block_gate (normalise: platt, floor, recall), decide_pooled on the
pre-floor pooled held-out LB, floor table, quiet_recall and after-floor
totals in analyse_rule; exp-003's analysis is unchanged.
plane/experiments/exp-004-pooled-block-gate.yaml: procedure and decision
rule registered now; units frozen at run time from the post-harvest
memory (units_status pending until then).
Tests (tests/test_learn_rules.py): pooled threshold where no family alone
reaches the bound, no held-out label moves a threshold or the map, a
family with one held-out error cannot block, t_quiet and no blind
silencing, the pooled verdict, the exp-004 manifest.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Real (vulnerable function, fixed function) pointer pairs for the six compiled
families, mined at zero model cost from reviewed GHSA advisories (pip, npm,
maven, composer), newest first: 79 of 98 published after 2026-06-01
(post_cutoff = true). Each pair: one linked one-parent fix commit, <= 4 source
files / 120 lines, the function holding the changed code lines declared once on
both sides with a changed body; every function diff read and kept only when the
named function holds the fix (145 reviewed, 100 kept); 2 dropped by the
reserved-population test; permissive licences only, no repository named
elsewhere under benchmarks/.
New label source advisory-fixes-2; the advisory-fix tests run over both
batches. Memory 937 -> 1133 static examples (98 paired groups added);
positives-only groups stay 99 (vibe-py/dev-php/v2 data gaps), their share of
positive groups in the six families 0.40 -> 0.29. Embeddings: 7 calls,
$0.0014. No chat-model call. Record:
benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
kind: rule experiments score once from the response cache behind a
zero-ceiling meter (a miss is excluded and counted, never asked) and apply
each arm's BLOCK rule to the same held-out scores. Arm B chooses a per-family
raw-score threshold on the other outer folds only (lowest t with Wilson 95%
LB >= 0.95); a test fails if a fold's own labels move its threshold.
Manifest and frozen units (353 units, 146 groups, 244 paired) for exp-003.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Kind proofs: records 01 and 02. Task 2.6 (receiver removal) is paused while the
safety net and the decision engine come first. deployment.md conflicts resolved
in the plain-language style; corrects the RuntimeClass claim (Substrate runs
runsc inside its worker pod).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Rerun records, serve-static and unrolled/secure (same planted commits) with
this build installed from its wheel: 6 replays and 10 hook pushes, all exit 0
in 0.69-1.21 s against the 30 s deadline, input read proven per run. The
planted JS eval in a new file surfaces in the quick tier on the host and, in
one Docker engine run (190.9 s, deadline 300), as two advisory engine findings
classified new/file_added_in_head. The planted Python %-formatted SQL is still
missed (no quick rule, no host engine): recorded as a coverage gap, no rule
edited. Go is named as not covered. The run caught a wrong install hint in
the hook (the package is not on PyPI); corrected here.
Record: benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
`ousast pre-push install|uninstall [REPO] [--force] [--docker]` ships in the
wheel with the hook as package data; it honours core.hooksPath, refuses an
existing hook unless --force (which chains it as pre-push.before-ousast), and
uninstall restores the chained hook. ops/ousast-docker is a hook-callable
script: the repository read-only and the artifact directory at their own
paths, stdin passed through, no network, --memory 3g, the GHCR 2.0.1 image by
default, advisory exit unless blocking. Help text for every pre-push flag; a
first-run guide, settings, skip-reason glossary, CI recipe and full --help in
docs/scanning.md; README tiers 1 and 2 pull the published image. The image
was not rebuilt: 1.1 GB free, below the 4 GB threshold. Requirements 7.4, 9.6.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
--engine inline|background|off (hook: OUSAST_PUSH_ENGINE). background returns
after the quick tier and starts one detached explicit replay of the same base
and head (push/background.py), one at a time per artifact directory; its
artifact lands under <artifact dir>/engine/<repo key>/<head>.json and the next
run prints "Engine result for <head> from the previous push" once. A deadline
miss says so in one line naming the deadline and the background option. No
engine speed work. Requirement 9.5.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
ChangeContext records the paths Git reports as added. An entailed,
undischarged operation in an added file, with complete head evidence and no
identical operation in any base answer, is classified new
(file_added_in_head) instead of base_incomplete; admission accepts that reason
as change evidence. Unpaired add/delete pushes and every other unknown
comparison keep their vetoes. An engine finding whose novelty is established
and whose only vetoes are capability vetoes is shown as an advisory engine
finding, not an alert. Requirement 9.4.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Before the engine and inside the same deadline, the shipped quick rules run
over the head side of the changed files (push/quick.py) and keep matches on
changed lines or in changed functions. They print as advisory pattern matches,
labeled apart from admitted alerts, are kept in the artifact's quick_tier with
the files and bytes read, and never admit or block. A new branch with no
remote base runs the quick tier over its last commit and says so. The planted
eval in serve-static's new debug.js now surfaces; the planted Python
%-formatted SQL in records does not (a quick-rule coverage gap, no rule
edited). Requirement 9.3.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
One terminal line per distinct coverage reason (push/explain.py): hex-encoded
paths decoded, per-question sha256 tokens dropped and counted, unknown reasons
still printed; the JSON artifact keeps the raw values. A changed language with
no engine frontend and no quick rules is recorded and printed as "language not
covered: <lang> (...)", in pre-push and in `ousast scan`. The scan report
renders the eight degradations cli.py recorded but never showed, and any
reason without a sentence. The hook forwards OUSAST_COMPARISON_BASE.
Requirement 9.2.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The installed hook exits 0 with one line when ousast (or OUSAST_COMMAND) is
not on PATH, still chaining a prior hook. In Git hook mode without an explicit
--mode blocking, argument/usage errors (an invalid OUSAST_PUSH_DEADLINE, a cache
directory inside the repository, a refused experimental flag) and unexpected
exceptions print one "check did not run" line and exit 0. Blocking mode keeps
its usage errors. Requirement 9.1.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
_execute builds the task's `delivery {put, inputs}` from Delivery, passes it to
router.starter, grants the store's host in the egress policy, awaits
Delivery.delivered (the tar's exact key in the store), then collect()s:
download, extract with the member checks, re-put the declared artifacts, write
done.json; the outcome is the collected summary.json. Three findings from the
tests: a rerun of an unfinished task completed at once on the previous
attempt's tar, so Delivery.reset(task) deletes output.tar and done.json before
every attempt; a worker thread that raised left the run blocked on its queue,
so the worker now turns any exception into a failed task; a consumer may
declare a producer's summary.json without the producer listing it (the loop
Run does), so declared(run) publishes outputs[] plus every artifact a consumer
names. The receiver thread survives behind OUSAST_DELIVERY=receiver for this
group only (no presigned URLs, the http rule, OUSAST_ARTIFACT_URL in the env);
the default path renders no receiver address into any Task. The fake ax PUTs
to the start body's URL and GETs its inputs from theirs; the reconciler tests
run through the S3-shaped fake store and assert the objects, done.json, the
seeded rerun and the receiver mode. reconciler.py: 549 lines (bound 550).
Gates: pytest 2031 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
policy_for(task, workspaces, model, store_host, receiver_host=None) emits one
tlsPassthrough rule on 443 over the sorted union of the memory store's host,
the bound Workspaces' https Git hosts and the bound Model's egress hosts. There
is no http rule: an http Git repo is refused by name (it could not be reached),
a store given as an address is refused naming S3_ENDPOINT (the gateway allows
hostnames only). receiver_host adds the old receiver's http rule for the path
group 2 keeps until 2.6; the reconciler passes it by keyword (passing the
receiver's address positionally would have raised outside the worker's try and
hung the run on the queue -- 2.4 hardens the worker). Tests: the store-only
policy, the Model-bound policy, the two refusals, the receiver exception, and
every committed template under plane/tasks/ rendering exactly the store plus,
for verify and roles, the provider host.
Gates: pytest 2027 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
StartGate.offer validates an optional `delivery` beside the credentials: https
URLs to a hostname (never an address, no userinfo; AX_RUNNER_INSECURE_DELIVERY=1
admits http in unit tests only), names ^[A-Z][A-Z0-9_]*$, a 400 that names the
field and echoes no URL. The runner PUTs its output tar to delivery.put and
GETs each declared input from delivery.inputs[NAME] (deliver() gains a method;
log lines and DeliveryError carry the URL without its query, so no presigned
signature reaches a log, a marker or a summary). OUSAST_ARTIFACT_URL/DIAL stay
honoured until task 2.6 so both paths run in the same build; the completion
marker and the retry-only boot are unchanged, and a re-start brings a fresh
URL for the retry. Router.start_task carries `delivery` in the body and
redacts its URLs like credential values. Tests: the gate's acceptance and
refusals, an end-to-end PUT/GET against the S3-shaped fake with no URL in any
log or file, the failed-PUT-then-retry-only boot, deliver() with PUT.
Gates: pytest 2024 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
