Commit Graph
432 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 4.8 4c4b7d375a measure(g2): E1/E2 clean-host results -- the engine 'query=0 wall' was contention, not intrinsic
On a clean host the shipped engine runs dataflow (query 0->35s, selected 0->356 regions).
The evidence/taint batch (~225s) is invariant to warm session and graph preload; importCpg
is cheap (~35s). The real barrier is taint.sc's ~160s per-batch flow work, so build+evidence
~255s of a 300s budget defers 1009/1365 regions. Lever = change-scope the family/file set the
taint batch walks, not session/preload. Corrects the prior ~200s graph-load wall diagnosis.

Next decisive experiment: engine-on unseen pilot vs record-07 regex floor (catch 3/92).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-12 08:20:16 +02:00
Marius CiepluchandClaude Opus 4.8 e716e3beab docs(g2): record-07 baseline is regex-only; prioritize semantic detection + barrier analysis
Verified on the records: the push replay arms hardcode model="disabled",
max_model_calls=0, cost_usd=0 (NO LLM) and engine_covered=0 (engine never ran,
~200s wall). So the measured catch~2.5% is the quick pattern-rule tier ALONE --
grep-with-structure. Pattern rules alone will never be enough.

Reprioritize G2 after record 07 to an efficiency barrier analysis over the two
tiers that carry detection and both sat out:
- engine (Joern dataflow): E1 split the ~150s evidence-fixed (JVM/graph-load/
  census); E2 does a warm session + graph preload at build kill it (decisive);
  E3 analysis per-pair; E4 build vs repo size.
- LLM adjudication: L1 model latency + calls per push; L2 recall lift over
  regex+dataflow on the pool.
Deliverable: a barrier table (budget x tier combo -> fits? measured recall/FP)
for a realistic pre-push window (<=120s target, 300s hard) -> minimal releasable
config. Change-scoping branch is a late add-on (trims only the per-pair tail),
not the lever. Also fixed a stray </content> tag in the brief.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 15:05:51 +02:00
Marius CiepluchandClaude Opus 4.8 df09fe4030 docs(g2): engine-arm wall diagnosed -- ~200s fixed overhead, change-scoping insufficient
Records the full G2 engine-detection finding on main (code attempt parked on
branch engine-change-scoped-evidence-pass, unmerged). The engine arm has ~200s
intrinsic fixed overhead -- ~48s CPG build + ~150s first-query graph-load+census
(measured uncontended: build 48s, evidence 243.79s over 1015 pairs, query 0) --
so a 300s budget cannot reach the flow queries. Pair-count change-scoping trims
only the ~86s per-pair tail; the warm-session env times out at startup contended
and spares only subsequent batches. Real fix is clean-host engine work (reliable
session startup + graph preload, or census-cost reduction). Parked to finish G0.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 13:31:55 +02:00
Marius CiepluchandClaude Opus 4.8 15992d3090 docs(g2): increment 2 measured -- the 236s is 99.9% the evidence-tier pass
Saved-artifact run: total=291s build=52s query=0s arbitrate=238.75s, and
query_seconds_by_kind[evidence]=238.54s. The evidence-tier pass is the whole
middle; affected_context + _scope_work are 0.21s. It tiered 1015 pairs at
~235ms/pair (contended) and exhausted the budget -> selected=0, deferred=1365
all deadline_exhausted. Secondary anomaly: ~50-100x the design's assumed
2.5ms/pair -- the evidence query is pathologically slow on a cold large CPG.

Increment 2's single target is the evidence pass (scan.py:338-359). Fork
recorded: (i) scope+bound the pass change-first, (ii) two-phase neighbourhood
pass, (iii) per-pair speedup. Must preserve reach-into-unchanged.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 12:11:57 +02:00
Marius CiepluchandClaude Opus 4.8 a0637f2f9d docs(g2): correct increment 2 -- preserve reach-into-unchanged, measure the 236s split first
A first scoping attempt (restrict analyzed regions to changed files) was reverted:
it breaks a tested design invariant. The engine deliberately selects an unchanged
region's question when a changed region reaches it
(test_changed_dependency_reaches_unchanged_ranked_question;
test_change_does_not_replace_ranker_selection). _changed_files feeds _evidence_pass
only as contextFilter/contextPaths (itemise vs summarise), not to shrink the tier
population. Any scope must keep changed regions PLUS the regions they reach.

Also: increment 1 over-claimed. Increment 0 measured build=67s/query=0s but NOT
the split of the ~236s middle (evidence-tier pass vs affected_context/context
resolution vs _scope_work enumeration). Increment 2 must capture per_kind /
other_seconds / affected_context timing and cut the dominant phase while
preserving reach-into-unchanged -- instrument before fixing.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 11:50:41 +02:00
Marius CiepluchandClaude Opus 4.8 9580ad1e30 docs(g2): increment 1 located -- regions are discovered whole-repo, never change-scoped
The 4095 candidates are not an in-scan artifact: push/runner.py:345 discovers
head_regions across the whole head tree, then runner.py:397 passes them wholesale
to scan_repository with change_context threaded in but unused for scoping. The
scan enumerates all regions x families (scan.py:167/364) and, with
order_by_evidence=True, evidence-tiers every pair before analysis -> the ~236s
from increment 0 -> deadline at query_seconds=0.

Fix locus (increment 2): scope the analyzed region set to context.changed_paths +
dataflow neighbourhood (regions.py:279 primitive exists; compare_targeted_base is
precedent). Keep the CPG whole-repo so taint can trace into the change from
unchanged code; only the question set is change-scoped.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 11:24:43 +02:00
Marius CiepluchandClaude Opus 4.8 7a1e47e0cc docs(g2): increment 0 confirmed -- CPG build cheap, candidate enumeration is the overspend
Timed live bedrock replay: build_seconds=66.69, query_seconds=0.0, total 303s,
4095 candidate questions all analysis_not_started=deadline_exhausted, selected=[],
regions_scanned=0. ~67s builds the CPG; ~236s enumerates/ranks/prepares 4095
repo-wide candidates for a 3-file change, then the deadline hits before any query
runs. The CPG build is not the bottleneck; the over-broad candidate set is ->
change-scoping is the confirmed lever (and compounds with the correspondence fix,
since per-candidate prep includes correspondence work).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 11:21:33 +02:00
Marius CiepluchandClaude Opus 4.8 9f312177ce docs(g2): open the change-scoped engine-analysis workstream (brief + roadmap)
Track the engine-detection lever found while driving G0. Live diagnosis on the
frozen pool: the engine's "no finding" on unseen changes is usually
analysis-never-runs -- it ranks repo-wide candidates and defers them all
(deadline_exhausted) before analyzing the pushed change (bedrock injection,
303s, selected=[], ranked sites unrelated to the change). Lever: seed analysis
from the change's declarations + dataflow neighbourhood (rank.py/mapping.py) so
it runs within budget.

- .kiro/specs/engine-change-scoped-analysis/brief.md: problem (measured),
  hypothesis, anchors, and checkable increments (0 confirm build-vs-analysis
  cost split -> 1 locate selection/budget -> 2 seed from change -> 3 soundness
  + tests -> 4 live re-test -> 5 measure delta on the engine arm).
- roadmap.md G2: active workstreams (change-scoped analysis; the done-but-
  necessary-not-sufficient line-correspondence fix), for progress tracking.

Honest scope: G2, gated on the pool metric; not part of the G0 quick-rule
record-07. Coverage also needs the capability registry (Task 8.3); arm-scale
measurement needs the kube lane.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 11:10:26 +02:00
Marius CiepluchandClaude Opus 4.8 abb2058a54 push(snapshot): run-aware base/head line-correspondence (stop discarding intra-run anchors)
The correspondence mapped an equal-region base<->head line pair only when the
line's content was GLOBALLY unique in the file, so common lines (`}`, blanks,
imports) tripped `ambiguous_line_correspondence` and the finding's anchor was
discarded -> the operation came back `unmapped` -> novelty `unknown` ->
`comparison_unknown` -> the engine finding was suppressed (a direct cause of
"near-blind to fixes").

Replace the global-uniqueness gate with run-aware positional anchoring: segment
the equal-region pairs into contiguous runs (strictly +1/+1, as _equal_line_pairs
emits), and map a pair when its content occupies exactly one old run and one new
run. Duplicates WITHIN a contiguous run are positionally forced and now map;
duplicates ACROSS runs split by an edit are aligned arbitrarily by git and stay
withheld + flagged (soundness preserved -- the moved-repeated-operation guard
still passes, so no false `new`/identity claims).

Tests: update test_change_context_unknown_base_binary_and_ambiguous_lines_are_
unresolved (its two in-run sinks now anchor); keep the moved-operation soundness
test; add intra-run-maps, cross-run-withheld, and distinctive-line-anchors cases.
163 push tests pass, ruff clean.

Honest scope (design.md amendment): this removes one real cause but is
necessary-not-sufficient -- live-testing a messy 14-file Java change showed
coverage stays `incomplete` due to other, dominant blockers (capability_unavailable
until Task 8.3; the engine produced no actionable defect; eligibility/projection
gaps). Engine coverage on unseen diffs needs those too, not just correspondence.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 10:12:03 +02:00
Marius CiepluchandClaude Opus 4.8 d03590148f docs(g0): record the 2026-10-11 state and readiness plan; bucket-first + image provenance
Fold the night's learnings into the roadmap, steering and the
unseen-repo-evaluation spec so the process is reproducible and the
project is "ready now":

- roadmap.md: a 2026-10-11 state table (pool p1 frozen; harvest hardened
  + bucket-first; replay-image provenance pinned; baseline-on-kube next)
  and a 5-step readiness plan to record 07.
- storage-streaming.md: RustFS-over-local as the default (local disk is
  the scarce resource); the implemented ousast-corpus layout; replay
  outputs belong on RustFS; the AX store endpoint is operator-owned.
- design.md: an "Amendments (2026-10-11)" section (bucket-first
  persistence, ~2.2% rate-bound yield, machine-label record 07, pinned
  replay image provenance).
- tasks.md: a folded-in amendment block (pool p1 frozen + freeze recipe;
  bucket-first; CI replay image; machine-label baseline; kube lane +
  ax/store as operator-owned infra).

Facts stated are verified (measured yield, freeze digest, GHCR anon pull,
batch.py's endpoint check); infra capability/policy is left to the
operator, not assumed. The frozen pool artifacts themselves are not
committed here (a separate samples/licensing decision).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-11 08:17:06 +02:00
Marius CiepluchandClaude Opus 4.8 197fe8a19c unseen: implement freeze p1 (group 3: bisect guard, freeze writer, protocol)
Group 3 was unimplemented, so there was no way to turn the draft into a
frozen pool. Add:

- bisect_guard.py: stages the cache draft and bisects it against the
  reserved-repository guard, run as a black box
  (pytest -k referenced, stdout/stderr discarded, exit code only; an exit
  other than 0/1 aborts as an instrument failure). Drops guard-tripping
  candidates and reports the dropped COUNT only -- never a name or URL.
  Guard runner is injectable for offline tests.
- freeze.py: writes canonical pool-p1.toml (+ private counterpart,
  reusing draft's public/private split and private_digest) and
  freeze-p1.json (used-set digest, per-source counts, guard result,
  dropped count, label-check counts, freeze digest = sha256 of the
  canonical public manifest). A matching retry is a no-op; any changed
  frozen artifact requires a new pool id (never an overwrite).
- protocol-p1.md: the frozen scoring/statistics protocol from design
  sections 3 and 4.

Offline only; the guard subprocess is the sole subprocess and is mocked
in tests. 42 new tests pass (incl the group-1.3 guard digest check on a
fixture pool); neighbouring unseen suites green; ruff clean. New files
only -- does not touch the journal-pinned draft/extract modules, so a
running draw is unaffected.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-09 19:23:26 +02:00
Marius CiepluchandClaude Opus 4.8 56cde9f535 spec(proof-arm-opencode): OpenCode MCP is the endpoint for PoC AND fix [skip ci]
Maintainer direction: the same agentic endpoint authors the PoC, writes a candidate
fix against the repair direction + failing PoC, and hands it to the differential
verifier, which re-runs the PoC and confirms absence. Closes the no-auto-patch gap:
the proof is the acceptance test for the fix (a wrong fix is caught), the fix stays a
developer-reviewed proposal, and the green proof is the objective signal. Specced
alongside proof authoring, after qualification.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:34:52 +02:00
Marius CiepluchandClaude Opus 4.8 dfc0956086 docs: frame the proof as prioritization evidence for busy devs [skip ci]
Revise the developer guide for engineers with full backlogs and no spare schedule:
a vague warning dies in the backlog, so the value of the demonstrated proof is the
prioritization conversation -- reproducible evidence (input, behavior, the one-line
fix difference) a dev takes to a product owner or the business to win capacity,
dropped into the ticket system as a location, repair direction, and reproduction.
mcp-shift-left-agent brief: the agent's return should be ticket-ready evidence.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:30:14 +02:00
Marius CiepluchandClaude Opus 4.8 c49f334651 docs: developer-facing pre-push security guide + mcp-shift-left-agent brief [skip ci]
docs/pre-push-for-developers.md: COBESY-structured guide for engineers who are
strong at dev but not security. Leads with the conclusion (zero-setup, no
annotations, diff-scoped, proof-backed), frames the agent-era case (a pre-push
check is how you oversee coding agents), states control (advisory vs blocking,
budget, lane, models), and is honest about limits with an explicit disproof
condition (coverage, large-repo latency, generalization being measured by G0).

.kiro/specs/mcp-shift-left-agent/: follow-up brief for an MCP agent + skills that
give devs and coding agents early shift-left feedback and a required pre-push
security step for agent oversight; builds on the existing ousast mcp server;
sequenced after the safety net is qualified.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 17:28:49 +02:00
Marius CiepluchandClaude Opus 4.8 60617a92ba docs(search-with-proof): verification architecture + proof-arm-opencode follow-up brief
verification-architecture.md documents the existing two-stage design: the Cairn-inspired
search (brain + model calls on the VM) and the DAST proof (gVisor AX tasks in kube-ax),
the trust boundary (secrets VM-only, cluster tasks secret-free with restricted egress),
and the three verify lanes (ax=cluster default, vm=local Bubblewrap for $0 proofs,
docker=replay). Clarifies the proof is dynamic, not abstract interpretation: AI triage
stays the static backbone, proof earns the block by execution.

proof-arm-opencode/ is a follow-up brief: drive proof-authoring with OpenCode MCP, an
abliterated/GLM proof model (DeepSeek stays the search model), DSPy-optimized from
OpenObserve traces, to beat the safety-classifier refusals and nondeterministic wiring.
Flags the trust-domain decision that must come first (A: scoped key + egress inside the
cluster; B: brain on VM, only DAST in gVisor — recommended to start). Credentials live in
.env by name only (ABLITERATION_*, OPENOBSERVE_*); no secret value is in any tracked file.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 16:06:56 +02:00
Marius CiepluchandClaude Opus 4.8 8de0c2732b spec+docs(search-with-proof): record first cluster demonstration + fixed-side negative [skip ci]
Feed the milestone back into the spec and steering. HANDOFF: the open "first demonstrated"
question is RESOLVED -- xiaomusic (path, prefix-check-bypass) demonstrated on the vulnerable
side (3/3 affected, 0/3 safe) and the blind fixed-side search produced 0 false proofs; pair
total $1.22. Records the six-fix chain that enabled it (parity, verify-time-only, fair-share
budget, served-root guidance, oracle prefix-bypass coverage, control-signal feedback) plus the
keystone. tasks.md: implementation note; pilot (6.2/6.3) in progress. spec.json: pilot status.
safety-net steering: proof-based BLOCK mechanism reached first execution-backed demonstration
(feasibility-pilot scope, not release readiness; pilot exit + 73-negative qualification open;
classifier stays triage). search README: status note. Payload-free; demos/boards stay gitignored.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-07 13:25:52 +02:00
Marius CiepluchandClaude Opus 4.8 d17ceb0e0c spec(search-with-proof): handoff -- plumbing cleared, open question is demo attack step vs oracle capture [skip ci]
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 15:14:45 +02:00
Marius CiepluchandClaude Opus 5.5 1d67412e2b search-with-proof: executor smoke driver and pilot selection (6.1)
20 development pairs, one per repository, 4 per language, 5 per family; identities
in a gitignored manifest, counts and digest committed. Executor smoke drives one
executor task end to end without a model and fails loudly on an unread clone.
kube-ax operator constraints as steering.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:26:06 +02:00
Marius CiepluchandClaude Opus 5.5 85f4f0b3d4 search-with-proof: declarative demos, brain/executor split, kube-ax constraints
Demos are data the verifier executes (fixed build recipes, runtime+path start,
HTTP/CLI steps with captures; the canary is never referable). The agent loop with
the key never runs repository code: executor tasks without a key exchange commands
and results through presigned objects. AX capacity errors are back-pressure with at
most 2 tasks in flight; caches and builds under /workspace with a 2 GiB guard;
verifiers take one presigned archive per side. Launcher fix: private /dev in bwrap.
190 tests on the host; probe 5/5 demonstrated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 17:12:46 +02:00
Marius CiepluchandClaude Opus 5.5 721c4b0997 spec(search-with-proof): separate trust domains (brain with key, executor with repo code)
Replaces the accepted key-exposure exception: the key never shares a sandbox with
repository code; executor AX tasks exchange commands and results via presigned
objects. Firecracker executors only if the pilot build-failure rate requires it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 16:26:19 +02:00
Marius CiepluchandClaude Opus 5.5 5e03bdf07f spec(search-with-proof): gVisor is the cluster boundary; declarative demos; capped rotated key
kube-ax probe: AX tasks have no capabilities (no namespaces, no setuid), so no
in-task isolation. Demos become data the verifier executes; the explore key risk is
accepted and bounded by cap and rotation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 16:20:26 +02:00
Marius CiepluchandClaude Opus 5.5 0f74767db9 search-with-proof: probe harness (3.2) and agent worker (group 5)
Probe runs each oracle toy pair with a real demo and two gaming demos, records
image facts, RSS and disk, fails loudly if isolation is unavailable; dispatchable
as an AX task. Worker: reason step without tools, explore tool loop whose run tool
goes through the sandbox with a scrubbed env (model key absent), spend reserved
before each call. Host probe: 5/5 real demos demonstrated, 10/10 gaming demos
inconclusive. 89 tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:45:48 +02:00
Marius CiepluchandClaude Opus 5.5 4b38239f68 search-with-proof group 4 + image: board, spend reservation, coordinator, search-task image
Board as immutable versioned blobs (coordinator sole writer); budget reserve/settle
before model calls; per-task and run caps; coordinator loop over an injected
executor; batch.py accepts a fixed allow-list of search variables; XSS browser
oracle; ousast-search-task image (Python, Node/TypeScript, PHP, Java 17, sqlite,
bwrap, chromium) built by CI. 103 tests on the host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:26:43 +02:00
Marius CiepluchandClaude Opus 5.5 6394ee473d merge(search-with-proof): group 2 verifier and oracles
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:09:48 +02:00
Marius CiepluchandClaude Opus 5.5 742469d671 search-with-proof group 2: verifier with owned oracles and differential
The verifier, not the artefact, observes the effect: SQL, path, command and SSRF
oracles owned by the verifier (XSS needs a browser executor, else no_oracle);
artefact exit code and output ignored; immutable checkout and dependencies;
3 runs per side; one-sided build or readiness failure is inconclusive. An
isolation failure raises IsolationUnavailable instead of reading as
could_not_build (the NPROC limit now applies inside the user namespace).
35 tests, including gaming attempts and 300 host processes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 14:09:48 +02:00
Marius CiepluchandClaude Opus 5.5 c74b5eaeb0 spec(learned-decision-engine): engine becomes triage where a demonstration oracle exists
Req 6.5 (maintainer decision 2026-10-05, with search-with-proof): BLOCK comes from
verified demonstrations; the engine is judged by top-K recall per push and advisory
false alarms; the pooled block gate governs only families without an oracle.
Task 17 adds the metric and re-scores existing arms at no cost.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:45:10 +02:00
Marius CiepluchandClaude Opus 5.5 9d70ef9257 spec(search-with-proof): requirements and design approved; tasks for probe and pilot
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:35:55 +02:00
Marius CiepluchandClaude Opus 5.5 c3820e184c spec(search-with-proof): requirements amended after two Codex reviews; design v2
False alarms count every visible finding; the verifier owns per-family oracles;
inconclusive on one-sided build failure; feasibility pilot separated from a
73-negative BLOCK qualification; coverage ceilings; spend reserved before calls;
key isolated from executed code; cluster model calls supersede the pool VM-only
rule for this arm only. Requirements approval reopened.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 13:35:18 +02:00
Marius CiepluchandClaude Opus 5.5 8bbea86cf8 spec(search-with-proof): requirements approved
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 12:01:39 +02:00
Marius CiepluchandClaude Opus 5.5 45e641f25c spec(search-with-proof): brief and requirements for review
After reviewing Cairn: detection so far classified excerpts and hit a precision
wall; this adds directed search with an objective goal (a re-verified
demonstration), a fact/intent board, and reason/explore steps as AX Tasks on
kube-ax. Design only learns from Cairn (AGPL); no code copied.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 12:01:07 +02:00
Marius CiepluchandClaude Opus 5.5 6d27c9bfcf unseen-eval: calibrated replay deadline + native agreement check, docker-lane diagnostics, proof 4.4 record; engine runner cleans up fetched sources
gVisor replays exhausted the hook's 30 s deadline in the snapshot step on a
1,725-file repository; cluster replays now use 30 s x a measured slowdown
factor, verified on a native sample. The engine runner deletes each pin's fetched
source (1.8 GB had accumulated), caps the cache at 500 MB and refuses below
1.5 GB free. Proof 4.4: two engine pins via the generic dispatcher match the VM
baseline exactly. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 08:14:57 +02:00
Marius CiepluchandClaude Opus 5.5 c6e87a8517 unseen-eval groups 4-6: generic AX batch dispatcher (engine_trace on it), replay task and image stage, scorer with adoption gate
Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 20:19:14 +02:00
Marius CiepluchandClaude Opus 5.5 48ae918dec unseen-eval group 2 (2.1-2.4): advisory sourcing, vulnerability-introducing ranges, ordinary-commit draw, draft manifest and slices; journal with resume
Implemented with Codex. Sourcing run (2.5) next, by the coordinator.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 19:51:30 +02:00
Marius CiepluchandClaude Opus 5.5 8213d3baad unseen-eval group 1: used set (1,029 repositories, every source read), extended guard, usage ledger; record 01
Implemented with Codex; task 1.5 run by the coordinator against the real
sources (S3 memory read-only).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 19:17:35 +02:00
Marius CiepluchandClaude Opus 5.5 60512fff88 spec(unseen-repo-evaluation): design and tasks approved; run through the baseline
Decisions: long-deadline arm (300 s), adoption needs no unseen regression
(Req 5.4), permissive repositories by name only, two AX lanes. Also adds the
trained-classifier and joint-optimisation arms to learned-decision-engine.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 18:48:13 +02:00
Marius CiepluchandClaude Opus 5.5 d4467f4236 steering: roadmap re-centred on generalisation to unseen repositories (G0-G5); specs updated; unseen-repo-evaluation brief and requirements (approved)
Maintainer 2026-10-04: the safety net is for unseen code and the A/B tests exist
to prove the approach generalises; infrastructure work had drifted from that.
Runs are designed for AX on kube-ax, kind is the fallback.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 17:34:57 +02:00
Marius CiepluchandClaude Opus 5.5 ee422b63e0 spec(learned-decision-engine): fix-mechanisms plan approved -- discharge facts first, >=2 proposing repos, per language
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 17:25:35 +02:00
Marius CiepluchandClaude Opus 5.5 8d6921d2df spec(learned-decision-engine): fix-mechanisms plan for review -- learn discharge facts from 710 fix diffs, admitted by held-out pair separation
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 14:11:41 +02:00
Marius CiepluchandClaude Opus 5.5 2ad9c1db7a learn(exp-004): Req 6.4 pooled BLOCK gate with a family floor; manifest before units
Maintainer decision 2026-10-03: BLOCK precision is shown pooled across
families. Req 6.4 (amended; 7.4 annotated): one t_block on a per-family
Platt-normalised score (calibrate.fit_map, training folds, inner
cross-fit), Wilson 95% LB >= 0.95 over all families' training flags; a
family blocks only with >= F = 10 held-out flags and zero errors, else
advisory; t_quiet per family keeps >= 90% of training paired positives.

evaluate: normalised, pooled_threshold, quiet_threshold,
pooled_block_gate, family_floor, apply_floor. experiments: rule
pooled_block_gate (normalise: platt, floor, recall), decide_pooled on the
pre-floor pooled held-out LB, floor table, quiet_recall and after-floor
totals in analyse_rule; exp-003's analysis is unchanged.

plane/experiments/exp-004-pooled-block-gate.yaml: procedure and decision
rule registered now; units frozen at run time from the post-harvest
memory (units_status pending until then).

Tests (tests/test_learn_rules.py): pooled threshold where no family alone
reaches the bound, no held-out label moves a threshold or the map, a
family with one held-out error cannot block, t_quiet and no blind
silencing, the pooled verdict, the exp-004 manifest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 09:40:58 +02:00
Marius CiepluchandClaude Opus 5.5 b24eeb15f3 merge(plane-on-kubernetes): groups 1-2 -- cluster profile, no host literals, delivery through the S3 store
Kind proofs: records 01 and 02. Task 2.6 (receiver removal) is paused while the
safety net and the decision engine come first. deployment.md conflicts resolved
in the plain-language style; corrects the RuntimeClass claim (Substrate runs
runsc inside its worker pod).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 07:57:13 +02:00
Marius CiepluchandClaude Opus 5.5 c8b418c34c plane(2.5): kind proof and record 02 -- artifacts through the store, one TLS rule
Live on kind with the runner image that carries 2.2
(ousast-runner@sha256:d2a55293...; templates and validation-46 Tasks repinned,
128 lines; kind-images.json updated): the smoke Run on store delivery finished
in 26 s and 24 s; the tar was PUT to runs/ax-e2e-smoke/facts/output.tar
(10240 B) and collected 2 s later; read back from the store its members
(facts.json 451 B, summary.json 238 B, units.jsonl 291 B) equal the extracted
files; facts.json, summary.json and done.json (149 B) re-put, state.json
(350 B) mirrored; a rerun from an empty results root seeded state.json from the
store and finished in 2 s without resubmitting. The actor's egress policy read
back live: one tlsPassthrough rule on 443 naming the store host, nothing else.
`ousast plane status ax-e2e-smoke` printed the table each time. Record:
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json with the
smoke outputs, store read-back and policy beside it. Task 2.6 (receiver
removal) is paused by the maintainer's reprioritisation.

Gates: pytest 2031 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0. (The first run, at 0.97 GiB free disk, failed 11 tests and errored 5
on the FileStore's 1 GiB free-space refusal; rerun after the disk was freed.)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 07:45:19 +02:00
Marius CiepluchandClaude Opus 5.5 2ec8071c26 pre-push(17.7): prove it on the audit's three repositories
Rerun records, serve-static and unrolled/secure (same planted commits) with
this build installed from its wheel: 6 replays and 10 hook pushes, all exit 0
in 0.69-1.21 s against the 30 s deadline, input read proven per run. The
planted JS eval in a new file surfaces in the quick tier on the host and, in
one Docker engine run (190.9 s, deadline 300), as two advisory engine findings
classified new/file_added_in_head. The planted Python %-formatted SQL is still
missed (no quick rule, no host engine): recorded as a coverage gap, no rule
edited. Go is named as not covered. The run caught a wrong install hint in
the hook (the package is not on PyPI); corrected here.
Record: benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 23:02:32 +02:00
Marius CiepluchandClaude Opus 5.5 591818d5fc pre-push(17.6): install, Docker wrapper and first-run docs
`ousast pre-push install|uninstall [REPO] [--force] [--docker]` ships in the
wheel with the hook as package data; it honours core.hooksPath, refuses an
existing hook unless --force (which chains it as pre-push.before-ousast), and
uninstall restores the chained hook. ops/ousast-docker is a hook-callable
script: the repository read-only and the artifact directory at their own
paths, stdin passed through, no network, --memory 3g, the GHCR 2.0.1 image by
default, advisory exit unless blocking. Help text for every pre-push flag; a
first-run guide, settings, skip-reason glossary, CI recipe and full --help in
docs/scanning.md; README tiers 1 and 2 pull the published image. The image
was not rebuilt: 1.1 GB free, below the 4 GB threshold. Requirements 7.4, 9.6.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:51:22 +02:00
Marius CiepluchandClaude Opus 5.5 66357ffea1 pre-push(17.5): say when the engine cannot finish, and offer it in the background
--engine inline|background|off (hook: OUSAST_PUSH_ENGINE). background returns
after the quick tier and starts one detached explicit replay of the same base
and head (push/background.py), one at a time per artifact directory; its
artifact lands under <artifact dir>/engine/<repo key>/<head>.json and the next
run prints "Engine result for <head> from the previous push" once. A deadline
miss says so in one line naming the deadline and the background option. No
engine speed work. Requirement 9.5.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:31:14 +02:00
Marius CiepluchandClaude Opus 5.5 511de4b75f pre-push(17.4): a file new in head is new, not comparison unknown
ChangeContext records the paths Git reports as added. An entailed,
undischarged operation in an added file, with complete head evidence and no
identical operation in any base answer, is classified new
(file_added_in_head) instead of base_incomplete; admission accepts that reason
as change evidence. Unpaired add/delete pushes and every other unknown
comparison keep their vetoes. An engine finding whose novelty is established
and whose only vetoes are capability vetoes is shown as an advisory engine
finding, not an alert. Requirement 9.4.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:24:24 +02:00
Marius CiepluchandClaude Opus 5.5 15999a09e1 pre-push(17.3): quick rules as the fast first tier
Before the engine and inside the same deadline, the shipped quick rules run
over the head side of the changed files (push/quick.py) and keep matches on
changed lines or in changed functions. They print as advisory pattern matches,
labeled apart from admitted alerts, are kept in the artifact's quick_tier with
the files and bytes read, and never admit or block. A new branch with no
remote base runs the quick tier over its last commit and says so. The planted
eval in serve-static's new debug.js now surfaces; the planted Python
%-formatted SQL in records does not (a quick-rule coverage gap, no rule
edited). Requirement 9.3.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:18:14 +02:00
Marius CiepluchandClaude Opus 5.5 f555dee640 pre-push(17.2): name every skipped check in plain words
One terminal line per distinct coverage reason (push/explain.py): hex-encoded
paths decoded, per-question sha256 tokens dropped and counted, unknown reasons
still printed; the JSON artifact keeps the raw values. A changed language with
no engine frontend and no quick rules is recorded and printed as "language not
covered: <lang> (...)", in pre-push and in `ousast scan`. The scan report
renders the eight degradations cli.py recorded but never showed, and any
reason without a sentence. The hook forwards OUSAST_COMPARISON_BASE.
Requirement 9.2.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:10:33 +02:00
Marius CiepluchandClaude Opus 5.5 bf07cd4d1d pre-push(17.1): never block a push the user did not choose to block
The installed hook exits 0 with one line when ousast (or OUSAST_COMMAND) is
not on PATH, still chaining a prior hook. In Git hook mode without an explicit
--mode blocking, argument/usage errors (an invalid OUSAST_PUSH_DEADLINE, a cache
directory inside the repository, a refused experimental flag) and unexpected
exceptions print one "check did not run" line and exit 0. Blocking mode keeps
its usage errors. Requirement 9.1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:57:42 +02:00
Marius CiepluchandClaude Fable 5.1 605b7af5ba plane(2.4): reconciler on Delivery -- the start body carries the URLs, the store completes the task
_execute builds the task's `delivery {put, inputs}` from Delivery, passes it to
router.starter, grants the store's host in the egress policy, awaits
Delivery.delivered (the tar's exact key in the store), then collect()s:
download, extract with the member checks, re-put the declared artifacts, write
done.json; the outcome is the collected summary.json. Three findings from the
tests: a rerun of an unfinished task completed at once on the previous
attempt's tar, so Delivery.reset(task) deletes output.tar and done.json before
every attempt; a worker thread that raised left the run blocked on its queue,
so the worker now turns any exception into a failed task; a consumer may
declare a producer's summary.json without the producer listing it (the loop
Run does), so declared(run) publishes outputs[] plus every artifact a consumer
names. The receiver thread survives behind OUSAST_DELIVERY=receiver for this
group only (no presigned URLs, the http rule, OUSAST_ARTIFACT_URL in the env);
the default path renders no receiver address into any Task. The fake ax PUTs
to the start body's URL and GETs its inputs from theirs; the reconciler tests
run through the S3-shaped fake store and assert the objects, done.json, the
seeded rerun and the receiver mode. reconciler.py: 549 lines (bound 550).

Gates: pytest 2031 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:49:58 +02:00
Marius CiepluchandClaude Opus 5.5 a656fda2f4 pre-push(17): operational-readiness task group and Requirement 9
Append task group 17 "Operational readiness (2026-10-02)" with one task per
maintainer-approved item (never block, name every skip, quick tier, new-file
classification, engine honesty and background mode, install/Docker/docs, proof),
add Requirement 9 (9.1-9.6, marked added 2026-10-02, maintainer-approved), and
check 13.4 and the parent of 14 as the audit found them done.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:46:25 +02:00