mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 12:09:27 +00:00
fix(cpg): a graph the frontend emptied must not read as a clean repository
Scanning a two-file WordPress slice returned 64 regions examined, zero
findings and zero degradations. The graph it examined had no methods in it.
php2cpg 4.0.623 feeds several files to one `php-parse` process and reads its
stdout and stderr merged. php-parse writes a `====> File <next>:` banner to
stderr the moment it starts the next file, while the previous file's JSON --
megabytes of it -- is still draining from a block-buffered stdout. The banner
lands inside the JSON, ujson reports
expected json value or } got "=" at index 5011645
and the frontend drops that file. Reproducible on real plugin code, and the
size that matters is one file's AST dump, not the repository's:
class.memberorder.php alone (1834 lines) -> 142KB CPG, no failures
the same file + a two-line file -> 5.7KB CPG, BOTH dropped
One oversized file empties the whole graph. `joern-parse` then does not
propagate any of the frontend's warnings: 13 lines of output, exit status 0,
"Successfully wrote graph". This is what made a 637-file plugin unscannable
and reported it as clean -- it was never a scale limit.
Two fixes, because the failure arrives on two different paths:
* `_unparsed_files` reads the per-file warnings the frontend does emit, and
the scan reports them as `files_unparsed`. This covers the direct-frontend
path where the warnings are visible.
* The batched query now carries a `__census__` of the graph -- method and
file counts -- under a key no request id can collide with, and a scan over
a graph with no methods degrades as `cpg_empty`. This covers joern-parse,
which shows nothing. It rides the batch rather than costing its own
invocation, because avoiding a JVM start is the entire reason batching
exists.
Neither invents a verdict. There is no rung for "the graph was empty" because
the ladder labels verdicts and none was produced, which is exactly why it has
to be a degradation.
pmpro two-file slice before: 0 findings, degradations=[]
after: 0 findings, degradations=[cpg_empty]
Gates byte-identical; 840 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
37a6594703
commit
e0442d146c
@@ -8,6 +8,7 @@ Nothing here imports a JVM binding, and every test runs with Joern absent.
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
@@ -341,3 +342,29 @@ def test_a_batch_goes_through_a_file_not_the_command_line(tmp_path: Path) -> Non
|
||||
assert staged, "the batch is staged to a file"
|
||||
written = Path(staged[0].split("=", 1)[1])
|
||||
assert written.is_file() and len(written.read_text()) > 200_000, "and the file holds the whole payload"
|
||||
|
||||
|
||||
def test_a_file_the_frontend_dropped_is_carried_off_the_build() -> None:
|
||||
"""A Joern frontend fails a file without failing the build, and exits 0 with the rest of the graph.
|
||||
|
||||
php2cpg 4.0.623 does exactly this: one oversized file corrupts the batched parser stream, that file is
|
||||
logged at WARN and dropped, and the CPG comes back missing it. A scan over the remainder that says
|
||||
nothing is a clean bill of health over code nobody looked at.
|
||||
"""
|
||||
warning = (
|
||||
"2026-09-09 17:46:21.892 WARN AstCreationPass Failed to process '/src/class.memberorder.php'\n"
|
||||
"2026-09-09 17:46:21.892 WARN SymbolSummaryPass Failed to process '/src/class.memberorder.php'\n"
|
||||
"2026-09-09 17:46:21.893 WARN AstCreationPass Failed to process '/src/rest-api.php'\n"
|
||||
)
|
||||
from openultrasast.cpg.backend import _unparsed_files
|
||||
|
||||
completed = subprocess.CompletedProcess(args=[], returncode=0, stdout="", stderr=warning)
|
||||
assert _unparsed_files(completed) == ("/src/class.memberorder.php", "/src/rest-api.php")
|
||||
|
||||
|
||||
def test_a_build_with_nothing_dropped_reports_no_unparsed_files() -> None:
|
||||
from openultrasast.cpg.backend import _unparsed_files
|
||||
|
||||
completed = subprocess.CompletedProcess(args=[], returncode=0, stdout="Successfully wrote graph", stderr="")
|
||||
assert _unparsed_files(completed) == ()
|
||||
assert _unparsed_files(None) == ()
|
||||
|
||||
Reference in New Issue
Block a user