Files
Open_Ultra_SAST/benchmarks
Marius CiepluchandClaude Opus 4.8 7c220307d4 measurements: path-family pilot after keystone + target-parity (0 demos, parity validated) [skip ci]
3 path searches (php/ts/py), vulnerable side, cluster verify lane, $2.95 of $3.
Payload-free aggregate; raw boards/demos stay gitignored. Finding: the parity fix
produced directed, target-aware search (the agent reasons about reaching the named
observable and inspects the app's serving guard), but no demonstration landed -- all
three spent their per-pair allowance before a differential, and two wrote zero verify
demos (budget consumed in reason/explore). Next levers: allowance/budget split toward
demo-writing, entrypoint discovery for php, and the explore-vs-verify path layout the
py agent flagged.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-06 19:35:42 +02:00
..