mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00

Files

T

Marius CiepluchandClaude Fable 5.1 70553cb7c6 measurements: first search-with-proof demonstrated (glance ceiling proof, $0 local) [skip ci]

Outcome-only record of the first end-to-end demonstrated verdict and the two
masking defects (regenerated npm lock pinning an incompatible mime; agent demo
targeting the wrong file). Payloads stay in the gitignored private area.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

2026-10-06 17:54:47 +02:00

2.6 KiB

Raw Blame History

Search-with-proof: first `demonstrated` (ceiling proof), 2026-10-06

First end-to-end demonstrated verdict for the search-with-proof verifier. No model, no cluster, $0. Run locally in-process via openultrasast.search.verify.verify (non-task bubblewrap lane, userns, node v18.19.1).

Pair: glance (jarofghosts/glance), family path, index.js Glance$serveRequest.
Verdict: demonstrated — owned effect in 3/3 affected runs, 0/3 safe runs (zero false proof on the fixed side).
Vulnerable side: canary observed 3/3 (parent traversal reaches the out-of-root nonce file; app reflects it).
Fixed side: 0/3 (the fix commit's parent-escape guard returns 403; the traversal cannot escape).
Wall ~65 s for both sides x 3 runs; ready ~2 s/side.

What this proves

The execution harness, the path oracle, and the differential verifier are correct and CAN produce a demonstrated verdict on a real third-party app. The method's ceiling is reachable. Every prior pilot inconclusive on this pair was an instrument artifact, not a method verdict.

Two defects that had masked this (both isolated, neither is the method)

Broken build. [email protected] depends on an unpinned mime >=1.2.6; the checkout's regenerated lockfileVersion-3 pins [email protected], where .lookup() was removed. glance crashes serving ANY file (TypeError: mime.lookup is not a function). Directory listing (readiness GET /) survives; the first file serve kills the process, so the nonce never reaches the response. Fixed here by installing a compatible [email protected] before the run.
Wrong demo target. The stored agent demo traversed to a filename that is not the canary, so it 404s even on the vulnerable side. The path oracle is the only owned oracle that hides its target from the demo author. The corrected demo targets the conventional canary file one level above the served root (same convention the known-good probe uses).

Reproduction

Demo schema + driver stored (gitignored) at benchmarks/search/private/ceiling-proof-2026-10-06/. Prepared checkouts need their npm symlinks stripped (the verifier refuses links) and a compatible mime@1. Payloads stay on disk per the classifier-safety note; never echoed.

Follow-on for honest qualification at scale (not covered by this $0 proof)

Keystone: positive-control in verify.py so a broken app yields could_not_run, never inconclusive.
Build-repro: distrust generated npm locks; pin/override incompatible transitive deps; audit all 20 pairs.
Demo-authoring: document the path-oracle target convention so the search AGENT can author a hitting demo.

2.6 KiB Raw Blame History

Search-with-proof: first demonstrated (ceiling proof), 2026-10-06