docs: plain-English pass over the active specs

Ran the humanizer rules over 79 prose files in the active specs (brief,
design, requirements, research and narrative docs). Removed em dashes,
significance inflation, forced rule-of-three, bold-label list items and
filler. Skipped tasks.md checklists and the retired specs (harnessx-*,
openrouter-sast-harness, unseen-repo-evaluation).

Every number, path, command, hash and identifier was preserved: checked
each file against HEAD for drift in numbers, inline code and code fences,
zero drift across all 79 files.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-12 18:16:41 +02:00
co-authored by Claude Opus 4.8
parent b08f372022
commit 39a1873e5b
79 changed files with 1103 additions and 1103 deletions
+17 -17
View File
@@ -1,32 +1,32 @@
# Search-with-proof — handoff (2026-10-06)
# Search-with-proof, handoff (2026-10-06)
State at handoff, for resuming on the search problem and abstract interpretation.
## RESOLVED 2026-10-07 — the one open question is CLOSED: first cluster demonstration achieved
## RESOLVED 2026-10-07, the one open question is CLOSED: first cluster demonstration achieved
The "first `demonstrated`" question below is answered. search-with-proof produced its first autonomous
`demonstrated` finding end to end on kube-ax, and a blind negative confirmed no false proof.
- **Pair:** hanxi/xiaomusic (pilot pair 13), family `path`, vulnerability class **prefix-check bypass**
- Pair: hanxi/xiaomusic (pilot pair 13), family `path`, vulnerability class **prefix-check bypass**
(`startswith(root)` without a trailing separator; the fix adds `os.sep`). Image `e2c8b460`.
- **Vulnerable side:** `demonstrated` — owned effect in 3/3 affected runs, 0/3 safe runs. $0.37, 55 calls.
- **Fixed side (blind negative, Req 6.3):** `inconclusive`, **0 false proofs**. $0.56, 85 calls.
- **Pair total:** $1.22. Record: `benchmarks/measurements/2026-10-07-search-first-cluster-demonstration/`.
- Vulnerable side: `demonstrated`, owned effect in 3/3 affected runs, 0/3 safe runs. $0.37, 55 calls.
- Fixed side (blind negative, Req 6.3): `inconclusive`, **0 false proofs**. $0.56, 85 calls.
- Pair total: $1.22. Record: `benchmarks/measurements/2026-10-07-search-first-cluster-demonstration/`.
**The original pair-6 both-sides-false was a TWO-part diagnosis, both now fixed:** the demo did not reach the
observable (demo/agent), and — the decisive one — the path oracle only modelled **classic parent traversal** while
observable (demo/agent), and (the decisive one) the path oracle only modelled **classic parent traversal** while
many real path CVEs (xiaomusic included) are **prefix-bypass**, which reaches a sibling that string-extends the served
dir, not the parent. The oracle now also plants a prefix-sibling canary.
**Fix chain that enabled the demonstration (each merged to main and validated by a paid run):**
1. Parity — agent is told the observable's location (path oracle reached parity with command/ssrf).
2. Verify-time-only — the observable is injected at verify time and absent during exploration; stop the explore-time
1. Parity. The agent is told the observable's location (path oracle reached parity with command/ssrf).
2. Verify-time-only. The observable is injected at verify time and absent during exploration; stop the explore-time
reachability trap that burned budget without authoring a demo.
3. Fair-share pilot budget — each search gets a fair share of `--ceiling-usd` with rollover, so one expensive search
3. Fair-share pilot budget. Each search gets a fair share of `--ceiling-usd` with rollover, so one expensive search
no longer starves the rest.
4. Served-root guidance — the demo MUST set the app's served directory to the `${served_root}` placeholder.
5. Oracle prefix-bypass coverage — plant a sibling canary for the `startswith`-without-separator class.
6. Control-signal feedback — the keystone control (does the app serve `${served_root}`?) is fed back to the agent as a
4. Served-root guidance. The demo MUST set the app's served directory to the `${served_root}` placeholder.
5. Oracle prefix-bypass coverage. Plant a sibling canary for the `startswith`-without-separator class.
6. Control-signal feedback. The keystone control (does the app serve `${served_root}`?) is fed back to the agent as a
`served-root` hint so it self-corrects its wiring. This was the decisive last step.
**Keystone (honest measurement), merged earlier:** the verifier runs a benign positive control; a broken or mis-wired
@@ -38,7 +38,7 @@ dependency build-break (`filed`→`mime@4`) is pinned.
---
## The one open question (HISTORICAL — resolved above)
## The one open question (HISTORICAL, resolved above)
The full chain runs end to end on kube-ax. Every execution layer is cleared. The **first `demonstrated` has not
happened yet**, and the remaining gap is narrow and reproducible at **$0 model cost** via `demo_replay`.
@@ -59,7 +59,7 @@ Both-sides-false means it is **not the harness** any more. It is one of:
capture in `oracles.py` `PathOracle.observe`).
- Re-run with `python -m benchmarks.search.demo_replay --both --demo <stored demo.json> --family path --lane ax
--image <digest> ...`. Stored demos live under `benchmarks/search/private/runs/*/<pair>/demo/demo.json` (gitignored).
- **Classifier note:** keep exploit payloads on disk. Hand Codex file-path tasks that treat demo contents as opaque;
- Classifier note: keep exploit payloads on disk. Hand Codex file-path tasks that treat demo contents as opaque;
print outcomes / phase timings / error *classes*, never request strings, env values, or step bodies. See memory
`search-with-proof-direction` and the classifier false-positive note. Code changes never tripped it; echoing
payloads or step-by-step attack prose did.
@@ -95,9 +95,9 @@ image (pip 3.13 vs venv 3.12 → use `sys.executable`) → checkout root on sys.
- `/workspace` ~3 GiB free per node until the volume move; scratch guard 2 GiB/task.
- Pilot spend to date ~$11 across 9 paid runs; the decisive replay was $0.
## Decision model vs search — current read
## Decision model vs search (current read)
- Decision/classification: measured, **promising as triage** (within-pair AUC 0.762; trained logistic ties at 0.721),
**capped as a blocker** (cannot prove 95% precision). Reframed as triage (`learned-decision-engine` Req 6.5).
- Search-with-proof: **not yet demonstrated**, but no observation so far argues against the idea — every failure was
- Search-with-proof: **not yet demonstrated**, but no observation so far argues against the idea. Every failure was
our execution, now cleared. The pair-6 demo/oracle question is the first verdict on the method itself.
+3 -3
View File
@@ -16,13 +16,13 @@ each confirm one fact; the loop ends when the goal is demonstrably met.
Three things follow for OpenUltraSAST:
1. **An objective success condition.** A finding is demonstrated when a test or proof of concept fails on the
1. An objective success condition. A finding is demonstrated when a test or proof of concept fails on the
vulnerable code and passes on the fixed code (or, for a push, fails on head and passes on base where applicable).
A demonstrated finding is right by construction; it needs no calibration and no training data, so it generalises
to unseen repositories by construction.
2. **Exploration with tools.** Agents that read other files, follow callers, build and run the code, instead of a
2. Exploration with tools. Agents that read other files, follow callers, build and run the code, instead of a
single-shot judgement of one excerpt.
3. **Search state per repository and push.** Facts and intents accumulate on a board instead of every question
3. Search state per repository and push. Facts and intents accumulate on a board instead of every question
starting from zero.
## Shape
@@ -73,7 +73,7 @@ Other files already modified/untracked at session start remain uncommitted.
- Requested suite, with `-rs` for skip reasons:
`/home/mc/Source/OpenUltraSAST/.venv/bin/pytest -q tests/test_search_*.py tests/test_ax_batch.py tests/test_plane_literals.py -rs`
— **181 passed, 3 skipped, exit 0**, 24.94 seconds.
Result: **181 passed, 3 skipped, exit 0**, 24.94 seconds.
- Final executor/storage regression and module audit tests after tightening stderr
truncation: **34 passed, exit 0**, including all **6 module audit tests**.
- Shared dispatcher consumer `tests/test_unseen_calibrate.py`: **10 passed, exit 0**.
@@ -9,12 +9,12 @@ kube-ax cluster. BLOCK becomes "demonstrated", not "scored above a threshold". C
## Boundary Context
- **In scope**: the board (facts, intents, hints) in the memory store; reason and explore steps as AX Tasks; the
- In scope: the board (facts, intents, hints) in the memory store; reason and explore steps as AX Tasks; the
demonstration contract and its verification; triage-to-search budgeting; a pilot on known fix pairs; a G1 arm on the
unseen pool; model access through an operator-managed scoped key.
- **Out of scope**: copying Cairn code (AGPL-3.0); network actions against anything but the code under analysis;
- Out of scope: copying Cairn code (AGPL-3.0); network actions against anything but the code under analysis;
changing the existing quick rules, engine or classifier (they are inputs here).
- **Adjacent expectations**: `unseen-repo-evaluation` scores this arm; the AX batch dispatcher (`benchmarks/ax/batch.py`)
- Adjacent expectations: `unseen-repo-evaluation` scores this arm; the AX batch dispatcher (`benchmarks/ax/batch.py`)
and the kube-ax constraints (2 heavy lanes, `ousast-engine-` names, egress to public HTTPS) apply;
`learned-decision-engine` Req 6.4 BLOCK rule is superseded for this arm by "demonstrated".
@@ -13,7 +13,7 @@ directed state-space **search**, then a DAST-style **proof (verification)**.
between the vulnerable and fixed revisions: the owned effect appears on the vulnerable side and
not on the fixed side (3/3 vs 0/3). Only this differential earns `demonstrated`.
## Trust boundary — what runs where
## Trust boundary (what runs where)
The brain runs on the VM; the agent's tool executions and the DAST proof run as isolated tasks in
the kube-ax cluster. Secrets never enter the cluster.