mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-21 14:09:27 +00:00
docs(g0): record the 2026-10-11 state and readiness plan; bucket-first + image provenance
Fold the night's learnings into the roadmap, steering and the unseen-repo-evaluation spec so the process is reproducible and the project is "ready now": - roadmap.md: a 2026-10-11 state table (pool p1 frozen; harvest hardened + bucket-first; replay-image provenance pinned; baseline-on-kube next) and a 5-step readiness plan to record 07. - storage-streaming.md: RustFS-over-local as the default (local disk is the scarce resource); the implemented ousast-corpus layout; replay outputs belong on RustFS; the AX store endpoint is operator-owned. - design.md: an "Amendments (2026-10-11)" section (bucket-first persistence, ~2.2% rate-bound yield, machine-label record 07, pinned replay image provenance). - tasks.md: a folded-in amendment block (pool p1 frozen + freeze recipe; bucket-first; CI replay image; machine-label baseline; kube lane + ax/store as operator-owned infra). Facts stated are verified (measured yield, freeze digest, GHCR anon pull, batch.py's endpoint check); infra capability/policy is left to the operator, not assumed. The frozen pool artifacts themselves are not committed here (a separate samples/licensing decision). Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
7b711a6150
commit
d03590148f
@@ -430,6 +430,36 @@ and dispatcher resume and fallback with a fake lane. Guard tests run in CI; clus
|
||||
| 5.3 experiments report unseen next to folds | 7.2 |
|
||||
| 5.4 adoption needs no significant unseen regression | 7.3 |
|
||||
|
||||
## Amendments (2026-10-11)
|
||||
|
||||
Folded in after the G0 pool was frozen; the original sections above stay as the design of record, these refine it
|
||||
where reality differed.
|
||||
|
||||
- **Persistence is bucket-first, not local disk.** The draw streams every stage to the `ousast-corpus` RustFS
|
||||
bucket and resumes from it, so a restart (even after a code change) is cheap. Layout: `unseen/<seed>/decisions/`
|
||||
(accept/reject per repo, key `sha256(repo+fix)`, idempotent), `unseen/<seed>/resolved/` (per-advisory
|
||||
resolution), `unseen/<seed>/advisories/list.json`, `unseen/repos/` (get_repo cache, **not** seed-scoped, shared
|
||||
across seeds), and `unseen/<seed>/pool/` (the frozen pool + provenance — the "don't redo sourcing" archive). The
|
||||
local journal is a within-process cursor only. Implementation: `benchmarks/unseen/bank.py` (`CorpusBank`). The
|
||||
VM's local disk is the scarce resource; run artifacts and replay outputs belong on RustFS (see steering
|
||||
`storage-streaming.md`). Credentials: `.env` `CORPUS_S3_*`; boto3 needs path-style + SigV4.
|
||||
- **Harvest cost is GitHub-rate-limit bound, measured yield ~2.2%.** Through the strict change criteria (1 vuln +
|
||||
12 stratified ordinary, >=40 eligible commits) ~2.2% of candidates are accepted, so "100 repos" is ~4,500
|
||||
candidate attempts at ~4 GitHub calls each — the bottleneck is the 5,000/hr API limit, not CPU or cores. Pure
|
||||
GitHub-API harvesting cannot replace the clone: the positive is the SZZ introducing commit and the negatives are
|
||||
stratified over full history, both needing repo history. The convergence fix was a full-objects clone (dropping
|
||||
`--filter=blob:none`, which caused ~600 per-blob lazy fetches) plus the bucket caches above.
|
||||
- **Record 07 is a machine-label baseline.** The pool's 100 vulnerability labels are SZZ-derived and unchecked
|
||||
(`freeze-p1.json` `label_check_counts`: 100 unchecked). The G0 baseline is reported as a machine-label number
|
||||
with that caveat; a human label pass is a later quality refinement, required before any published precision
|
||||
claim, not before record 07.
|
||||
- **Replay image provenance is pinned.** CI `engine-image.yml` builds `ghcr.io/norandom/ousast-replay-task` (the
|
||||
replay harness, which `COPY`s `/app/benchmarks/unseen/replay_entry.py`) and `ousast-engine-task` (the Joern
|
||||
hook), publicly pullable by digest. `ousast-runner` (the 248 MB ax-task-runner, no `/app/benchmarks`, no Joern)
|
||||
is **not** the replay image. The AX lane's store check (`benchmarks/ax/batch.py`) requires a public-HTTPS DNS
|
||||
presign-capable endpoint; how that and the cluster's store access are provided is operator-owned infra, not
|
||||
assumed here.
|
||||
|
||||
## Open Questions
|
||||
|
||||
Maintainer decisions of 2026-10-04 resolve questions 1, 2, 4 and 5; question 3 stays open for G3.
|
||||
|
||||
Reference in New Issue
Block a user