Commit Graph
23 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 5.5 c6e87a8517 unseen-eval groups 4-6: generic AX batch dispatcher (engine_trace on it), replay task and image stage, scorer with adoption gate
Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 20:19:14 +02:00
Marius CiepluchandClaude Opus 5.5 02b2cbf99c engine: retry a pin once when its AX sandbox dies, then hand it to the VM lane
Two engine actors died at gVisor level during a 12-pin run (AX phase Failed, no
result). Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 17:14:20 +02:00
Marius CiepluchandClaude Opus 5.5 187e3c14c5 engine: one resident Joern session per pin (frontend + one query JVM instead of four), ActiveProcessorCount=2 on the cluster
Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 16:41:22 +02:00
Marius CiepluchandClaude Opus 5.5 b715ff7d0e engine: stage presigned objects as .dat; task entry reports the failing step and HTTP status
Through Cloudflare, signed requests for keys ending .tar/.gz/.zip/.bin reached
RustFS altered and failed with 403 (cf-cache-status BYPASS); the same signed
request straight to RustFS on the LAN returned 200. A Cloudflare cache-bypass
rule on files.because-security.com fixes it (all extensions 200, DYNAMIC); the
engine executors keep .dat as a safeguard, the plane keeps output.tar.
Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 16:18:01 +02:00
Marius CiepluchandClaude Opus 5.5 abca40b072 engine: AX-task executor for kube-ax (engine-task image FROM ax-task-runner v0.3.1, presigned HTTPS I/O, ousast-engine- names, 60 s egress-ready retry, 1.7 GB cluster heap, VM OOM fallback)
Implemented with Codex. Live smoke pending the operator egress watcher.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 15:22:44 +02:00
Marius CiepluchandClaude Opus 5.5 afff080874 merge(engine): Kubernetes engine workers (Deployment from the image, S3 queue); leak-audit records carry digests, not repository names
The Joern leak-audit record named spent population v1/v2 repositories, which
the reserved-repository guard forbids outside benchmarks/independent/ (no v3
repository was involved: 0 matches). Records now carry counts and SHA-256
digests. With Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 14:39:36 +02:00
Marius CiepluchandClaude Opus 5.5 e174c06ee7 engine: worker Deployment from the image (S3 queue with leases, dispatcher --executor queue, engine-image workflow, scoped RustFS policy); merge main
Implemented with Codex. Cluster deploy pending the k3s kubeconfig and the scoped account.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 14:23:04 +02:00
Marius Ciepluch e555ba68dc Merge branch 'joern-leak-audit' 2026-10-04 10:53:23 +02:00
Marius CiepluchandClaude Opus 5.5 231dca1f04 merge: taint-query null-guard fix and trace-enabled engine runner
The query fix stops a class-level call from aborting the whole taint batch on
real PHP code; gate, map_gate and pair_gate outputs are unchanged. Trace parsing
lives with the runner (benchmarks/learn/engine_trace_parse.py).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 10:53:23 +02:00
Marius CiepluchandClaude Opus 5.5 2e6ba93aee learn: Kubernetes executor for the engine trace runner (Jobs, presigned source/result via the S3 store, mixed lanes, --k8s-check)
Implemented with Codex; live cluster check pending.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 10:35:59 +02:00
Marius CiepluchandClaude Opus 5.5 431ab1e738 learn: engine_trace materialises catalog pairs without commits (dedented parseable excerpts, in-function scope), protects labelled files in exports, scopes large repos, records frontend diagnostics, retries the jssrc2cpg engine bug, writes progress.json
Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 08:19:46 +02:00
Marius CiepluchandClaude Opus 5.5 94ec4cef16 learn: Joern-slice leak audit (has-path, steps, sanitizer, guard, sink, asked; both-asked pairs; per-corpus AUC)
Implemented with Codex. The record is written after the trace re-run completes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 23:03:52 +02:00
Marius CiepluchandClaude Opus 5.5 41f56170bb learn: engine_trace worker renders request params through the production _render; records query stderr
The worker sent tuple params as JSON arrays; taint.sc expects comma-joined
strings (ujson InvalidData, exit 1). Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 22:41:54 +02:00
Marius CiepluchandClaude Opus 5.5 fe2a424018 learn: engine_trace fetches real source at the pinned commit, 2.5 GB JVM heap, one CPG per language
Root causes of the 50 failed pins in trace run 1: advisory pairs were fed as
excerpts (invalid Java/JS fragments), the JVM defaulted to ~768 MB heap in a
3 GB container, and mixed-language pins were refused. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 22:36:23 +02:00
Marius CiepluchandClaude Opus 5.5 36a3355b81 learn: engine_trace guard judges frontends by their CPG, query steps by time and question markers
php2cpg legitimately finishes in ~2 s on small files. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 19:22:37 +02:00
Marius CiepluchandClaude Opus 5.5 34d00391f1 learn: engine_trace container worker imports only the engine; dry-run writes nothing; resume keyed on selection
The 2026-09-14 engine image has no PyYAML or boto3; the worker no longer imports
learn.examples. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 19:19:22 +02:00
Marius CiepluchandClaude Opus 5.5 19c936343b learn: trace-enabled Joern run over labelled units (opt-in trace in taint.sc, trace parsing, resumable runner)
Implemented with Codex. Default scan queries do not request a trace.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 19:10:47 +02:00
Marius Ciepluch 77e47ac7d3 Merge branch 'graph-slice' 2026-10-03 16:58:37 +02:00
Marius CiepluchandClaude Opus 5.5 1b2c7993a7 learn: in-function def-use slice from the tree-sitter layer + leak audit (\$0)
slice.py renders source -> step -> sink lines with fixed wording for flow,
no-flow and not-analysable; the prompt option is default-off (byte-identical
default prompt). semantic IR: JS/C declarations count as assignments, augmented
assignment keeps its target (gate, map_gate, pair_gate outputs unchanged).
Audit over exp-004's 501 units: 59 flow, 161 no-flow, 281 not analysable;
has-flow within-pair accuracy 0.52, pooled AUC 0.55 -- no leak. Fixes a
numbering bug that rejected every excerpt with a blank line. With Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:56:52 +02:00
Marius CiepluchandClaude Opus 5.5 463fe1503a learn: build_features reads v2 _a/_b scans and TypeScript repository-scan witnesses; --dry-run
Population v2 scans are named <case>--<side>_a/_b.json; the script looked only
for <case>--<side>.json and recorded those examples as engine not-run. The two
repeats combine symmetrically for both sides. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:41:55 +02:00
Marius CiepluchandClaude Fable 5.1 ef6945ec53 bench(learn): labels, features and memory rebuilt with the recovered negatives
Labels regenerated (identical sha: the builder's fixed-side rule already
accepted these pairs), feature records built for the 9 corpus pairs the
matcher now resolves (36 rows; the store's ingest index had remembered them
from a build before the declares() gate and skipped them as a repeat after a
later build dropped them -- build_features now checks presence after an index
hit and restores the rows), static memory rebuilt on RustFS: 904 -> 937
examples (+16 fixed-side negatives, +17 positives: 7 advisory Ruby pairs, 9
corpus pairs, 1 population positive), 9 wrong excerpts corrected (Ruby blocks,
a TS default-value brace), no embedding call (33 new + 9 changed excerpts
without a vector, ~$0.0003 to embed). Positives-only groups 98 -> 99: all
data gaps (94 vibe-py same-snapshot fixed sides, 3 dev-php, 2 population
fixes that did not change the function). Leak audit: pairs 376 -> 385; static
no new flag; plane two new injection flags (agree.final, verify.votes at
0.2292 over 48 pairs), v1 profile unchanged, reported.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 16:31:30 +02:00
Marius CiepluchandClaude Opus 5.5 9e3c83cfdc fix(learn): withhold label-tracking signals from the v1 classifier; leak audit
Maintainer decision "Neutralise both for v1".

1. Phantom candidates. A pair side's candidates are now only the functions that
   side declares (plane/harvest.py `declares`, the excerpt builder's
   function_bounds); a labelled function absent from a side is listed under
   `absent_on_side` and never asked, and a side declaring none of its verify
   family's functions is no verify unit. The recorded harvest units held 58
   phantom verify candidates (53 fixed, 5 vulnerable sides). build_features.py
   skips label rows whose side does not declare the function and drops their
   earlier rows (MemoryStore.drop_rows); rebuilt the pair records from the
   recorded signals, no model call: 126 label rows absent_on_side (117 fixed,
   9 vulnerable) -- access_control 13, config_secrets 4, deserialization 20,
   injection 28, memory 4, output_encoding 16, path 15, untrusted_destination
   26 -- so 252 feature rows dropped (1712 -> 1460); no other row changed.

2. Engine withheld. Input profiles in learn/schema.py (`v1` withholds the
   engine instrument, `full` withholds nothing); ProgramSpec.inputs defaults to
   v1 and compile.toml records it. Neither the engine's state nor its values
   reach the candidate, examples, demonstrations or the instruction proposer's
   error summary, and the Gower retrieval distance leaves the engine block out.
   Stored records are unchanged. design.md Risks records the decision and that
   the engine returns only through a pre-registered A/B on assumed-benign and
   benign-control negatives.

3. `ousast learn audit-leaks` (learn/leaks.py): per feature and per instrument
   state, separation = (pairs the signal alone puts on the right side - on the
   wrong side) / pairs, over all pairs and per family; flagged at >= 0.20 over
   >= 10 pairs. Counts under
   benchmarks/measurements/2026-10-01-decision-engine-leak-audit/ (before and
   after the rebuild). After: facts.function_lines (0.80 untrusted_destination,
   0.75 config_secrets, 0.26 injection; higher on fixed), verify.turns_mean
   0.67 config_secrets, eng.* 0.35-0.40 output_encoding, verify.flag_a/votes/
   site_offset and agree.final 0.20-0.30 on 10-12 pair families. Before the
   rebuild the phantom fixed sides drove state.source/state.roles/ms.flagged
   to 1.0 and state.engine to 0.95 on deserialization.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 19:40:58 +02:00
Marius CiepluchandClaude Opus 5.5 34da916030 bench(learn): harvest signals for the decision engine (decision-engine task 5)
`ousast plane harvest` (plane/harvest.py) writes two Runs over a label snapshot:
verify a/b + agree on the verify-family units (both sides of each labelled pair as
an opaque files-only Workspace, candidates outside the hunt's root, no declared
sites; v1 and dev-php pins with repo-facts first) and model roles on the candidate
files (pair excerpts grouped, one Task per population pin). Budgets sum under the
ceiling by construction. New roles task template; the generator re-pins it.

Spent by attribution: verify $2.54 (1961 calls, 150 units, 182 hunts per pass),
roles $1.40 (1034 calls, 799 files, 0 unclassified chunks); DeepSeek balance
19.63 -> 18.23. Engine on the pair corpus (benchmarks/learn/engine_pairs.py,
frozen e6ed785, one 3g container) is PARTIAL: 384 of 686 sides at commit, still
running; unfinished sides stay missing. benchmarks/learn/build_features.py built
1712 static+plane records into the memory store; counts-only record under
benchmarks/measurements/2026-10-01-decision-engine-harvest/.

source/roles parts are `none`, not `failed`, for a language without a declaration
pattern (C). ops/ax/README: Substrate golden-actor disk leak and the inline
Workspace limit (~20 KB).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 14:04:28 +02:00