mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
spec+docs(search-with-proof): record first cluster demonstration + fixed-side negative [skip ci]
Feed the milestone back into the spec and steering. HANDOFF: the open "first demonstrated" question is RESOLVED -- xiaomusic (path, prefix-check-bypass) demonstrated on the vulnerable side (3/3 affected, 0/3 safe) and the blind fixed-side search produced 0 false proofs; pair total $1.22. Records the six-fix chain that enabled it (parity, verify-time-only, fair-share budget, served-root guidance, oracle prefix-bypass coverage, control-signal feedback) plus the keystone. tasks.md: implementation note; pilot (6.2/6.3) in progress. spec.json: pilot status. safety-net steering: proof-based BLOCK mechanism reached first execution-backed demonstration (feasibility-pilot scope, not release readiness; pilot exit + 73-negative qualification open; classifier stays triage). search README: status note. Payload-free; demos/boards stay gitignored. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
ecd1372568
commit
8de0c2732b
@@ -1,6 +1,12 @@
|
||||
{
|
||||
"record": "2026-10-07 FIRST CLUSTER DEMONSTRATION for search-with-proof",
|
||||
"pair": {"index": 13, "repo": "hanxi/xiaomusic", "family": "path", "class": "prefix-check-bypass (startswith without separator)", "language": "python"},
|
||||
"pair": {
|
||||
"index": 13,
|
||||
"repo": "hanxi/xiaomusic",
|
||||
"family": "path",
|
||||
"class": "prefix-check-bypass (startswith without separator)",
|
||||
"language": "python"
|
||||
},
|
||||
"image": "ghcr.io/norandom/ousast-search-task@sha256:e2c8b460...",
|
||||
"side": "vulnerable",
|
||||
"outcome": "demonstrated",
|
||||
@@ -18,5 +24,23 @@
|
||||
"served-root guidance: wire the served dir to ${served_root}",
|
||||
"oracle prefix-bypass coverage: sibling canary for startswith-without-separator vulns",
|
||||
"control-signal feedback: served-root hint so the agent self-corrects wiring"
|
||||
]
|
||||
}
|
||||
],
|
||||
"both_sides": {
|
||||
"vulnerable": {
|
||||
"outcome": "demonstrated",
|
||||
"settled_usd": 0.3686,
|
||||
"verify_tasks": 18,
|
||||
"model_calls": 55
|
||||
},
|
||||
"fixed_blind_negative": {
|
||||
"outcome": "inconclusive",
|
||||
"false_proof": false,
|
||||
"settled_usd": 0.5632,
|
||||
"verify_tasks": 18,
|
||||
"model_calls": 85,
|
||||
"note": "Agent searched the FIXED (safe) revision blind (no vuln knowledge); authored demos but the verifier found no effect -> no false proof (Req 6.3 negative)."
|
||||
},
|
||||
"pair_total_settled_usd": 1.224,
|
||||
"verdict": "True positive on the vulnerable revision, zero false proof on the fixed revision."
|
||||
}
|
||||
}
|
||||
@@ -131,3 +131,12 @@ manifest; without a subtype verification returns `no_oracle`. No subtype is gues
|
||||
from repository names. The selected manifest's pair 1 currently lacks this metadata.
|
||||
Live transport, provider authentication/cost, real project buildability, and AX
|
||||
oracle operation must still be measured; an offline probe cannot establish them.
|
||||
|
||||
## Status (2026-10-07)
|
||||
|
||||
The feasibility pilot achieved its first cluster `demonstrated` finding (xiaomusic, path /
|
||||
prefix-check-bypass): vulnerable side demonstrated, fixed side blind search produced no
|
||||
false proof. See `benchmarks/measurements/2026-10-07-search-first-cluster-demonstration/`.
|
||||
Outcomes and spend are aggregates only; stored demos and boards stay in the gitignored
|
||||
`benchmarks/search/private/` area. Pilot exit (≥5 demonstrations / 0 false proofs per
|
||||
family) and the ≥73-negative BLOCK qualification remain open.
|
||||
|
||||
Reference in New Issue
Block a user