spec+docs(search-with-proof): record first cluster demonstration + fixed-side negative [skip ci]

Feed the milestone back into the spec and steering. HANDOFF: the open "first demonstrated"
question is RESOLVED -- xiaomusic (path, prefix-check-bypass) demonstrated on the vulnerable
side (3/3 affected, 0/3 safe) and the blind fixed-side search produced 0 false proofs; pair
total $1.22. Records the six-fix chain that enabled it (parity, verify-time-only, fair-share
budget, served-root guidance, oracle prefix-bypass coverage, control-signal feedback) plus the
keystone. tasks.md: implementation note; pilot (6.2/6.3) in progress. spec.json: pilot status.
safety-net steering: proof-based BLOCK mechanism reached first execution-backed demonstration
(feasibility-pilot scope, not release readiness; pilot exit + 73-negative qualification open;
classifier stays triage). search README: status note. Payload-free; demos/boards stay gitignored.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-07 13:25:52 +02:00
co-authored by Claude Opus 4.8
parent ecd1372568
commit 8de0c2732b
6 changed files with 114 additions and 7 deletions
@@ -1,6 +1,12 @@
{
"record": "2026-10-07 FIRST CLUSTER DEMONSTRATION for search-with-proof",
"pair": {"index": 13, "repo": "hanxi/xiaomusic", "family": "path", "class": "prefix-check-bypass (startswith without separator)", "language": "python"},
"pair": {
"index": 13,
"repo": "hanxi/xiaomusic",
"family": "path",
"class": "prefix-check-bypass (startswith without separator)",
"language": "python"
},
"image": "ghcr.io/norandom/ousast-search-task@sha256:e2c8b460...",
"side": "vulnerable",
"outcome": "demonstrated",
@@ -18,5 +24,23 @@
"served-root guidance: wire the served dir to ${served_root}",
"oracle prefix-bypass coverage: sibling canary for startswith-without-separator vulns",
"control-signal feedback: served-root hint so the agent self-corrects wiring"
]
}
],
"both_sides": {
"vulnerable": {
"outcome": "demonstrated",
"settled_usd": 0.3686,
"verify_tasks": 18,
"model_calls": 55
},
"fixed_blind_negative": {
"outcome": "inconclusive",
"false_proof": false,
"settled_usd": 0.5632,
"verify_tasks": 18,
"model_calls": 85,
"note": "Agent searched the FIXED (safe) revision blind (no vuln knowledge); authored demos but the verifier found no effect -> no false proof (Req 6.3 negative)."
},
"pair_total_settled_usd": 1.224,
"verdict": "True positive on the vulnerable revision, zero false proof on the fixed revision."
}
}
+9
View File
@@ -131,3 +131,12 @@ manifest; without a subtype verification returns `no_oracle`. No subtype is gues
from repository names. The selected manifest's pair 1 currently lacks this metadata.
Live transport, provider authentication/cost, real project buildability, and AX
oracle operation must still be measured; an offline probe cannot establish them.
## Status (2026-10-07)
The feasibility pilot achieved its first cluster `demonstrated` finding (xiaomusic, path /
prefix-check-bypass): vulnerable side demonstrated, fixed side blind search produced no
false proof. See `benchmarks/measurements/2026-10-07-search-first-cluster-demonstration/`.
Outcomes and spend are aggregates only; stored demos and boards stay in the gitignored
`benchmarks/search/private/` area. Pilot exit (≥5 demonstrations / 0 false proofs per
family) and the ≥73-negative BLOCK qualification remain open.