fix(cpg): retry a failed build through the language frontend

Scanning a 637-file WordPress plugin reported 1577 regions with 0 scanned
and `cpg_build_failed`. The build was not slow or out of memory -- it
failed in 42 seconds -- and `php2cpg` run directly on the same tree
produced a 4.3MB CPG with no errors at all.

The difference is that `joern-parse` detects the language and then shells
out to the frontend in a separate process. On this tree it got as far as
applying overlays and then threw, while still writing an unusable cpg.bin.
Joern's own output recommends the direct route for a large codebase:

  "Invoking CPG generator in a separate process... If you are importing a
   large codebase (and/or running into memory issues), please try:
   invoke the frontend: php2cpg -J-Xmx3072m /tmp/src -o cpg.bin"

So a failed `joern-parse` now retries through `<language>2cpg`, with the
driver passing the dominant region language. A language with no mapping
fails closed exactly as before -- no guessing at a frontend name.

This is the documented fallback rather than a workaround, and it was
diagnosed rather than assumed: the standalone build was run first, and only
after it succeeded on the identical tree was the retry added.

Gates byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-09 16:19:32 +02:00
co-authored by Claude Opus 5
parent 922ed84053
commit 9c50e17c11
3 changed files with 101 additions and 4 deletions
+47
View File
@@ -256,3 +256,50 @@ def test_the_operator_can_raise_the_heap(tmp_path: Path, monkeypatch: pytest.Mon
JoernBackend(runner=runner).build(tmp_path)
assert "-J-Xmx6144m" in commands[0]
def test_a_failed_joern_parse_retries_through_the_language_frontend(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""Measured on a 637-file WordPress plugin: `joern-parse` threw after applying overlays while `php2cpg`
on the same tree produced a 4.3MB CPG with no errors. Joern's own message recommends the direct route for
a large codebase, so this is the documented fallback rather than a workaround."""
from openultrasast.cpg.backend import JoernBackend
tried: list[str] = []
def runner(command, **kwargs): # type: ignore[no-untyped-def]
class _Result:
stderr = ""
stdout = ""
tried.append(Path(command[0]).name)
if Path(command[0]).name == "joern-parse":
_Result.returncode = 1
return _Result()
_Result.returncode = 0
Path(command[command.index("-o") + 1]).write_text("cpg")
return _Result()
monkeypatch.setenv("OPENULTRASAST_JOERN_PROBE", "on")
monkeypatch.setattr("shutil.which", lambda name: f"/opt/joern/{name}")
result = JoernBackend(runner=runner).build(tmp_path, language="php")
assert result is not None, "the frontend succeeded where joern-parse did not"
assert tried == ["joern-parse", "php2cpg"]
def test_an_unknown_language_has_nothing_to_retry_with(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""No guessing at a frontend name: a language with no mapping fails closed, as it did before."""
from openultrasast.cpg.backend import JoernBackend
def runner(command, **kwargs): # type: ignore[no-untyped-def]
class _Result:
returncode = 1
stderr = ""
stdout = ""
return _Result()
monkeypatch.setenv("OPENULTRASAST_JOERN_PROBE", "on")
assert JoernBackend(runner=runner).build(tmp_path, language="cobol") is None
assert JoernBackend(runner=runner).build(tmp_path) is None