fix(cpg): exclude the files php2cpg miscompiles, and name them

Root cause of every failed whole-repository PHP scan, bisected from 637 files
to one file to one function to one construct:

    <?php
    function f($n) {
        h("p", function($p) { global $g; return $p . $g; });
    }

A `global` declaration inside a CLOSURE makes php2cpg 4.0.623 emit a node with
two AST parents, and Joern then refuses to apply its own dataflow overlay:

    java.lang.AssertionError: Iterable was expected to have exactly one
    element, but it has 2. Hint: trying to resolve astParent ... (which is
    probably a malformed cpg)

Controlled, because four earlier hypotheses were refuted by measurement and I
was not going to add a fifth:

    global inside a closure WITH use   -> ASSERTS
    global inside a closure, NO use    -> ASSERTS   (so `use` is irrelevant)
    use, global OUTSIDE the closure    -> ok        (so scope is the trigger)

The property that matters: one such file costs the WHOLE GRAPH, not itself.
That is why a 2,500-request taint batch and a 13-request dominance batch failed
identically in ~25s, and why I twice called this a per-request cost problem. It
never was. Nothing was ever served; the graph is rejected at load.

Rare enough to exclude rather than surrender to -- 1 of 637 files in Paid
Memberships Pro, 1 of 357 in WP Statistics, 0 of 169 in MW WP Form, about 0.2%
-- so the construct is detected by reading the source, those files are excluded
from the build, and they are reported as `unparsed` so the coverage section
names them instead of implying they were clean.

Repository-scale PHP goes from blocked to 99.8% scanned with a stated gap.

854 tests pass; gates byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-10 12:17:48 +02:00
co-authored by Claude Opus 5
parent 128950a586
commit 4a1e06b75b
2 changed files with 111 additions and 3 deletions
+39
View File
@@ -645,3 +645,42 @@ def test_a_silently_short_graph_is_reported_even_with_no_warnings(tmp_path: Path
backend.build(tmp_path, language="php")
assert "1 of 3 source files" in backend.last_failure, "three files went in, one came out, and it said so"
def test_a_file_php2cpg_miscompiles_is_excluded_and_named(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""A `global` inside a closure makes php2cpg emit a node with two AST parents, and Joern then refuses to
apply its dataflow overlay -- to the WHOLE graph, not that file. Every query over the repository fails at
load, so one such file costs everything.
Four lines reproduce it; `use` is irrelevant, and a `global` at function scope is fine. It appears in
about 0.2% of files (1 of 637 in Paid Memberships Pro, 1 of 357 in WP Statistics, 0 of 169 in MW WP
Form), so excluding it saves the other 99.8% -- and it is reported, not silently dropped.
"""
import subprocess
from openultrasast.cpg.backend import JoernBackend
(tmp_path / "ok.php").write_text("<?php\nfunction a() { global $g; return $g; }\n")
(tmp_path / "bad.php").write_text('<?php\nfunction f($n) {\n h("p", function($p) { global $g; return $p . $g; });\n}\n')
commands: list[list[str]] = []
def runner(command, **kwargs): # type: ignore[no-untyped-def]
if "-r" in command:
return subprocess.CompletedProcess(args=command, returncode=0, stdout="", stderr="")
commands.append(list(command))
if "--script" in command:
body = '---OUSAST-CPG-BEGIN---\n{"files": "1", "methods": "2"}\n---OUSAST-CPG-END---\n'
return subprocess.CompletedProcess(args=command, returncode=0, stdout=body, stderr="")
Path(command[command.index("-o") + 1]).write_text("cpg")
return subprocess.CompletedProcess(args=command, returncode=0, stdout="", stderr="")
monkeypatch.setenv("OPENULTRASAST_JOERN_PROBE", "on")
monkeypatch.setattr("shutil.which", lambda name: f"/opt/bin/{name}")
result = JoernBackend(runner=runner).build(tmp_path, language="php")
assert result is not None
assert [Path(name).name for name in result.unparsed] == ["bad.php"], "the miscompiled file is named"
build = next(c for c in commands if "--script" not in c)
assert "--exclude" in build and any("bad.php" in part for part in build), "and excluded from the build"
assert not any("ok.php" in part for part in build if part != str(tmp_path)), "a global at function scope is fine"