Files
Open_Ultra_SAST/benchmarks
Marius CiepluchandClaude Opus 5 f3f41e2fa9 docs: the 637-file wall is the taint query, not memory and not vendored code
Follow-up measurements, because the obvious explanations were all wrong.

Not memory: at 2GB AND 4GB the CPG never finishes loading, with no
OutOfMemoryError either time, both runs stopping in the same place --
Braintree\Util. What costs is Joern's dataflow overlay, recomputed per import.

Not scope, though scope is real and worth fixing on its own terms: 429 of the
637 files are vendored SDKs (Braintree, Stripe, each with its own LICENSE) and
31 are tests. PMPro's own code is 177 files, 62k lines. Auditing a plugin by
scanning the payment SDK it bundles is wrong regardless of cost -- those
findings are not the plugin's bugs -- and it is the same idea shipped.py
already applies to Makefile.am, which PHP has no equivalent of.

Scoping to authored code helps and does not fix it:

    regions 4,463 -> 1,324    total 996.6s -> 843.7s    findings still 0

600.3s of that 843.7s is the taint query hitting QUERY_TIMEOUT_SECONDS (300s)
on each of two shards. Two timeouts, not work.

So the wall is the per-request cost of the taint query at 2,500 requests. The
next measurement is what the two-stage joins cost at scale -- 1.6x on a slice
(62s -> 97s) and plausibly the dominant term here. Raising the timeout before
knowing that would only buy a slower way to find out.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-10 09:06:03 +02:00
..