Files
Open_Ultra_SAST/benchmarks
Marius CiepluchandClaude Opus 5 99abe12062 measure(php): quantify what remains once the malformed-CPG bug is excluded
The graph bug is fixed and was never what these runs were hitting. joernio/joern#6281, this
project's own report, is a PHP `global` inside a closure giving a node two AST parents, which made
the dataflow overlay throw and cost the whole graph. Filed against 4.0.623, fixed by #6270 in
v4.0.625, which the image has pinned since before any measurement recorded under task 13.2.
Verified today on the real trigger rather than a transcription of the repro, which does not
reproduce: PMPro's includes/fields.php builds and applies the default overlays cleanly on 4.0.625
and on 4.0.631, files=2 methods=158, no ReachingDefPass assertion.

What remains is runtime. A taint batch of 2,526 questions over the 673-file checkout does not
complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed
nothing but how long the failure took: query time went from 363 s to 1,874 s, and zero questions
completed in both. The eight-way split retry did not rescue it either, finishing its parts in
about three minutes against a deadline with 3,300 seconds still on it, which is worth its own
look.

The engine pin stays at 4.0.625. An image on 4.0.631, today's release, was built and measured and
is indistinguishable on this subject: 627.7 s against 619.1 s, 363.4 s against 367.3 s of query
time, zero completed in both. There is no demonstrated benefit to weigh against re-verifying every
JavaScript and Python measurement on a new engine. Should that change, the bump is three values
rather than a port: the upstream AstGenRunner.scala is byte-identical in 4.0.631 and its patch
anchor still appears exactly once, so only the jar path, the jar checksum and the source URL tag
move. Both are recorded.

Also closes task 14.5. Neither prototype pollution nor deserialization has a subject that can
answer, each for its own reason. Deserialization's mechanism exists on exactly one available
subject and that subject cannot be queried at all. Prototype pollution has no available subject
containing the mechanism, so its ability to answer is established on a control instead.

1,417 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 18:13:25 +02:00
..