Merge doc-g0-readiness-2026-10-11: G0 state, readiness plan, bucket-first docs

Co-Authored-By: Claude Opus 4.8 <[email protected]>

.kiro/specs/unseen-repo-evaluation/design.md

@@ -430,6 +430,36 @@ and dispatcher resume and fallback with a fake lane. Guard tests run in CI; clus
 | 5.3 experiments report unseen next to folds | 7.2 |
 | 5.4 adoption needs no significant unseen regression | 7.3 |
 
+## Amendments (2026-10-11)
+
+Folded in after the G0 pool was frozen; the original sections above stay as the design of record, these refine it
+where reality differed.
+
+- **Persistence is bucket-first, not local disk.** The draw streams every stage to the `ousast-corpus` RustFS
+  bucket and resumes from it, so a restart (even after a code change) is cheap. Layout: `unseen/<seed>/decisions/`
+  (accept/reject per repo, key `sha256(repo+fix)`, idempotent), `unseen/<seed>/resolved/` (per-advisory
+  resolution), `unseen/<seed>/advisories/list.json`, `unseen/repos/` (get_repo cache, **not** seed-scoped, shared
+  across seeds), and `unseen/<seed>/pool/` (the frozen pool + provenance — the "don't redo sourcing" archive). The
+  local journal is a within-process cursor only. Implementation: `benchmarks/unseen/bank.py` (`CorpusBank`). The
+  VM's local disk is the scarce resource; run artifacts and replay outputs belong on RustFS (see steering
+  `storage-streaming.md`). Credentials: `.env` `CORPUS_S3_*`; boto3 needs path-style + SigV4.
+- **Harvest cost is GitHub-rate-limit bound, measured yield ~2.2%.** Through the strict change criteria (1 vuln +
+  12 stratified ordinary, >=40 eligible commits) ~2.2% of candidates are accepted, so "100 repos" is ~4,500
+  candidate attempts at ~4 GitHub calls each — the bottleneck is the 5,000/hr API limit, not CPU or cores. Pure
+  GitHub-API harvesting cannot replace the clone: the positive is the SZZ introducing commit and the negatives are
+  stratified over full history, both needing repo history. The convergence fix was a full-objects clone (dropping
+  `--filter=blob:none`, which caused ~600 per-blob lazy fetches) plus the bucket caches above.
+- **Record 07 is a machine-label baseline.** The pool's 100 vulnerability labels are SZZ-derived and unchecked
+  (`freeze-p1.json` `label_check_counts`: 100 unchecked). The G0 baseline is reported as a machine-label number
+  with that caveat; a human label pass is a later quality refinement, required before any published precision
+  claim, not before record 07.
+- **Replay image provenance is pinned.** CI `engine-image.yml` builds `ghcr.io/norandom/ousast-replay-task` (the
+  replay harness, which `COPY`s `/app/benchmarks/unseen/replay_entry.py`) and `ousast-engine-task` (the Joern
+  hook), publicly pullable by digest. `ousast-runner` (the 248 MB ax-task-runner, no `/app/benchmarks`, no Joern)
+  is **not** the replay image. The AX lane's store check (`benchmarks/ax/batch.py`) requires a public-HTTPS DNS
+  presign-capable endpoint; how that and the cluster's store access are provided is operator-owned infra, not
+  assumed here.
+
 ## Open Questions
 
 Maintainer decisions of 2026-10-04 resolve questions 1, 2, 4 and 5; question 3 stays open for G3.

.kiro/specs/unseen-repo-evaluation/tasks.md

@@ -7,6 +7,24 @@ on the unseen slice (Req 5.4 as amended, design 7.3, tasks 6.4 and 8.2); only pe
 are published by name, everything else goes to the gitignored private manifest under a digest (Req 1.4, design
 1.4); concurrency stays at two AX lanes plus the VM lane (design 5.3).
 
+**2026-10-11 amendments folded in** (see design.md "Amendments (2026-10-11)"):
+
+- **Pool p1 is frozen.** `benchmarks/unseen/pool-p1.toml` = 100 repos, slice 1; `freeze-p1.json` guard=0,
+  dropped=0, deterministic (manifest `6bd2fa5d`); archived to `ousast-corpus/unseen/20260601/pool/`. Groups 1-3 are
+  effectively done (the draw, draft and freeze ran end to end); formal records 02/03 were not written as separate
+  `record.json` files — the freeze was run directly (`bisect_guard` then `freeze`). Freeze recipe: `bisect_guard
+  --root $PWD --draft $CACHE/journal/draft-p1.toml` then `freeze --root $PWD --used-report <FILE>` where the file
+  is the draw report's `used_set` object (not the whole report).
+- **Persistence is bucket-first** (`ousast-corpus`, `benchmarks/unseen/bank.py`); the local journal is a
+  within-process cursor only. Raw per-change replay records go to **RustFS**, not `~/ousast-results/` on local disk
+  — amend tasks 5.3/7.2's `--out` accordingly.
+- **Group 5 image note:** the replay image is the CI-built `ghcr.io/norandom/ousast-replay-task` (+ engine-task),
+  publicly pullable by digest — not a local export-HEAD build and not `ousast-runner`. Task 7.1's "build and push
+  the replay image" is satisfied by the CI `engine-image.yml` run; use that digest.
+- **Record 07 is a machine-label baseline** (labels SZZ-derived, 100 unchecked); state the caveat in the record.
+- **Baseline lane (7.2):** run on kube (`--lane ax`). The `ax` binary and the AX-lane store endpoint are
+  operator-owned infra to stand up before 7.2; do not assume the cluster/network state.
+
 Execution rules:
 
 - **Who does what.** Codex implements every coding task offline: no network, no model, no store, no cluster. The

.kiro/steering/roadmap.md

@@ -41,6 +41,36 @@ Correction recorded 2026-10-04: the previous two days went mostly into infrastru
 not move a generalisation number. From here, infrastructure and engine work is done only when it removes a
 bottleneck for the measurements below.
 
+### Where we are (2026-10-11)
+
+The G0 measurement is now runnable: the unseen pool is **frozen** and the whole pipeline is hardened and
+**bucket-first**. The baseline replay is the one remaining step, to run on kube.
+
+| Area | State | Evidence |
+| --- | --- | --- |
+| Unseen pool (G0.1) | **Frozen.** `pool-p1.toml` = 100 repos (slice 1); `freeze-p1.json` guard=0, dropped=0; deterministic (manifest `6bd2fa5d`). License 49 permissive / 51 private; methods 81 introducing / 19 last_touch. | `benchmarks/unseen/pool-p1.toml`; archived `ousast-corpus/unseen/20260601/pool/` |
+| Labels | 100 machine-derived (SZZ), **unchecked**. Record 07 is a **machine-label baseline** (caveat stated); a human label pass is a later refinement, before any published claim. | `freeze-p1.json` label_check_counts |
+| Harvest | Rebuilt to converge. Pure GitHub-API harvesting is a dead end for this benchmark (it needs the SZZ introducing commit + 12 stratified history negatives, not one commit). The fix: a full-objects clone (drop `--filter=blob:none`, which caused ~600 per-blob lazy fetches) + bucket-first durability + a get_repo cache that paid the 4,499-name used-set canonicalization once. Yield ~2.2%; **GitHub-rate-limit bound, not CPU**. | 95bed70, 857263d, 21aebe8, 7b711a6 |
+| Persistence | **Bucket-first, not local disk.** `ousast-corpus` is the source of truth: enumeration, resolution, get_repo, and accept/reject decisions all stream to and resume from it. The host disk (~9-13 GB free) is the scarce resource; RustFS/S3 has the room, so run artifacts and **replay outputs go to RustFS too**, not `~/ousast-results` on local disk. | `storage-streaming.md`, `bank.py` |
+| Replay image | Provenance pinned. CI `engine-image.yml` builds `ousast-replay-task` (harness with `/app/benchmarks/unseen/replay_entry.py`) + `ousast-engine-task` (Joern hook), publicly pullable by digest. The old `ousast-runner:dev` (248 MB ax-runner, no `/app/benchmarks`, no Joern) is the **wrong** image for replay. | CI run 38116281676 |
+| Baseline (G0.2) | **Not yet.** Replay reads the frozen pool cleanly (dry-run = 1,300 changes, `freeze_digest` verified). Remaining: mirror CI's passing `replay` job wiring, and stand up the kube lane (`ax` binary + a public-HTTPS presign-capable store endpoint; GHCR pull already public). | CI `replay` job (passed) |
+
+Lesson, recorded so it is not re-paid: sourcing and engine-image work cost real time. The durable takeaway now lives
+in the pipeline and these docs — bucket-first persistence, pinned image provenance, a measured yield — so the next
+pool or arm is cheap, not a rediscovery.
+
+### Readiness plan (to G0 exit = record 07)
+
+1. **Kube replay lane (infra, operator-owned):** install the `ax` binary; provide the store endpoint the AX lane
+   needs. The only thing settled in-repo is a code fact: `benchmarks/ax/batch.py` currently expects a public-HTTPS
+   DNS, presign-capable S3 endpoint. How that endpoint and the cluster's access to the store are provided is an
+   operator/AX decision, not an assumption to make here. GHCR pull of the images is verified public.
+2. **Unblock the replay** by mirroring CI's passing `replay` job (replay-task + engine-task wired, store + network).
+3. **Run slice-1 baseline on kube:** `--lane ax --arm default`, 1,300 changes, no model cost, resumable; outputs to
+   RustFS.
+4. **Adjudicate (7.3) -> record 07 (7.4)** = the measured machine-label unseen baseline = **G0 exit**.
+5. Then **G1 arms** (DeepSeek decision engine) on the frozen pool.
+
 ### Phases
 
 **G0. The unseen-repository pool and the push-level metric** (new spec `unseen-repo-evaluation`)

.kiro/steering/storage-streaming.md

@@ -48,8 +48,33 @@ buckets rather than inventing a new transport. New buckets and their grants are
 admin** action — the store credentials cannot create buckets or set lifecycle (confirmed by the
 `AccessDenied` on the 2026-10-09 lifecycle apply).
 
-## Status
+## Default: RustFS over local disk
 
-Direction recorded; planned as a follow-up spec, not retrofitted into an in-flight run. The current
-unseen draw finishes to its local journal and then archives its result to `sast-pools` once that
-bucket exists.
+Local disk on the VM is the scarce resource (it runs near full; the draw stops itself under 2 GB free).
+RustFS/S3 has far more room. So the default for any run's durable state and large artifacts is the bucket,
+not local disk. Local files are a rebuildable scratch cache only (the live git clone, a within-process
+journal). A long run must be resumable purely from the bucket, and its outputs — including replay result
+records — are written to RustFS, not `~/ousast-results` on local disk.
+
+## Status (implemented 2026-10-10/11)
+
+Bucket-first persistence is live, not a future spec. The unseen draw streams every stage to the
+`ousast-corpus` bucket (`http://10.0.0.54:9001`, creds in `.env` as `CORPUS_S3_*`; boto3 needs path-style +
+SigV4) and resumes from it:
+
+- `unseen/<seed>/decisions/<sha256(repo+fix)>.json` — accept/reject per repository (idempotent; content dedup).
+- `unseen/<seed>/resolved/<sha256(ghsa)>.json` — per-advisory resolution.
+- `unseen/<seed>/advisories/list.json` — the enumerated advisory list.
+- `unseen/repos/<sha256(name)>.json` — get_repo cache, **not** seed-scoped (shared across seeds/runs), so the
+  thousands of used-set canonicalization lookups are paid once ever.
+- `unseen/<seed>/pool/` — the frozen pool and its provenance (`pool-p1.toml`, `private-pool-p1.toml`,
+  `freeze-p1.json`, `bisect-p1.json`, `protocol-p1.md`): the durable "don't redo the sourcing" archive.
+
+Implementation: `benchmarks/unseen/bank.py` (`CorpusBank`, `corpus_settings`). The old local journal is a
+within-process cursor only; the bucket is the cross-run source of truth, so a restart (even after a code
+change) resumes cheaply. Still to do and consistent with the default above: replay result records
+(`replay.py --out`) currently default to `~/ousast-results/unseen/...` on local disk — point them at RustFS
+for the slice-1 baseline and after. The AX (kube) lane already uses presigned object transfer, which is the
+same principle. One in-repo code fact: `benchmarks/ax/batch.py` currently expects a public-HTTPS DNS,
+presign-capable endpoint. How the store endpoint and the cluster's access to it are provided is operator/AX-owned
+infra, not something this doc should presume.