mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 20:49:28 +00:00
fix(cpg): retry a failed build through the language frontend
Scanning a 637-file WordPress plugin reported 1577 regions with 0 scanned and `cpg_build_failed`. The build was not slow or out of memory -- it failed in 42 seconds -- and `php2cpg` run directly on the same tree produced a 4.3MB CPG with no errors at all. The difference is that `joern-parse` detects the language and then shells out to the frontend in a separate process. On this tree it got as far as applying overlays and then threw, while still writing an unusable cpg.bin. Joern's own output recommends the direct route for a large codebase: "Invoking CPG generator in a separate process... If you are importing a large codebase (and/or running into memory issues), please try: invoke the frontend: php2cpg -J-Xmx3072m /tmp/src -o cpg.bin" So a failed `joern-parse` now retries through `<language>2cpg`, with the driver passing the dominant region language. A language with no mapping fails closed exactly as before -- no guessing at a frontend name. This is the documented fallback rather than a workaround, and it was diagnosed rather than assumed: the standalone build was run first, and only after it succeeded on the identical tree was the retry added. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
922ed84053
commit
9c50e17c11
@@ -110,7 +110,12 @@ def scan_repository(
|
||||
|
||||
taxonomy = load_families()
|
||||
build_started = time.monotonic()
|
||||
cpg = backend.build(root)
|
||||
# The dominant region language, so a failed `joern-parse` can retry through that frontend directly.
|
||||
counts: dict[str, int] = {}
|
||||
for region in regions:
|
||||
counts[region.language] = counts.get(region.language, 0) + 1
|
||||
dominant = max(counts, key=lambda name: (counts[name], name)) if counts else ""
|
||||
cpg = _build(backend, root, dominant)
|
||||
build_seconds = round(time.monotonic() - build_started, 2)
|
||||
if cpg is None:
|
||||
return ModelScanResult(
|
||||
@@ -223,6 +228,14 @@ def scan_repository(
|
||||
)
|
||||
|
||||
|
||||
def _build(backend: Any, root: Path, language: str) -> Any:
|
||||
"""Build through the backend, passing the language when the backend can use it."""
|
||||
try:
|
||||
return backend.build(root, language=language)
|
||||
except TypeError: # a backend from before the retry existed, including every test double
|
||||
return backend.build(root)
|
||||
|
||||
|
||||
def _prefetched(rows: list[object]): # type: ignore[no-untyped-def]
|
||||
"""A `run` that ignores the question and returns rows already fetched in the batch."""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user