Files
Open_Ultra_SAST/benchmarks/measurements
Marius CiepluchandClaude Opus 5 d93dee9f1a test(cpg): prove transport equivalence for every shipped query kind (M2 11.2)
A session is only usable if it returns exactly what the disposable invocation
returns. On a real NodeGoat graph, taint, dominance and configuration all produce
byte-identical payloads through both transports, with parameters built by each
arbiter's own request function through the driver's family lookup rather than a
hand-written shape.

The failure shapes that would make a session dangerous rather than merely slow are
correct too. An empty answer stays present and distinguishable from an unavailable
one, which is the distinction that once turned a failed scan into a clean bill of
health. An unknown query and an absent graph are both unavailable. Two graphs in
one session each answer about the graph the request names.

Amortized over five taint requests the session took 22.5 s against 72.0 s
disposable, all five answered and identical to each other.

Two instrument defects are recorded rather than hidden. The first run created a
fresh backend per comparison, so every session served one request and paid full
startup, measuring the session's worst case and making it look slower; that is why
the amortized comparison exists. The switched-graph check crashed on a scratch
directory it never created, so it produced no verdict at all.

These timings are orientation, not the latency gate. Task 11.5 measures against the
frozen seven-class profile, and the transport stays off by default until then.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:37:47 +02:00
..