mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-19 13:10:03 +00:00
fix(cpg): a graph the frontend emptied must not read as a clean repository
Scanning a two-file WordPress slice returned 64 regions examined, zero
findings and zero degradations. The graph it examined had no methods in it.
php2cpg 4.0.623 feeds several files to one `php-parse` process and reads its
stdout and stderr merged. php-parse writes a `====> File <next>:` banner to
stderr the moment it starts the next file, while the previous file's JSON --
megabytes of it -- is still draining from a block-buffered stdout. The banner
lands inside the JSON, ujson reports
expected json value or } got "=" at index 5011645
and the frontend drops that file. Reproducible on real plugin code, and the
size that matters is one file's AST dump, not the repository's:
class.memberorder.php alone (1834 lines) -> 142KB CPG, no failures
the same file + a two-line file -> 5.7KB CPG, BOTH dropped
One oversized file empties the whole graph. `joern-parse` then does not
propagate any of the frontend's warnings: 13 lines of output, exit status 0,
"Successfully wrote graph". This is what made a 637-file plugin unscannable
and reported it as clean -- it was never a scale limit.
Two fixes, because the failure arrives on two different paths:
* `_unparsed_files` reads the per-file warnings the frontend does emit, and
the scan reports them as `files_unparsed`. This covers the direct-frontend
path where the warnings are visible.
* The batched query now carries a `__census__` of the graph -- method and
file counts -- under a key no request id can collide with, and a scan over
a graph with no methods degrades as `cpg_empty`. This covers joern-parse,
which shows nothing. It rides the batch rather than costing its own
invocation, because avoiding a JVM start is the entire reason batching
exists.
Neither invents a verdict. There is no rung for "the graph was empty" because
the ladder labels verdicts and none was produced, which is exactly why it has
to be a degradation.
pmpro two-file slice before: 0 findings, degradations=[]
after: 0 findings, degradations=[cpg_empty]
Gates byte-identical; 840 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
37a6594703
commit
e0442d146c
@@ -73,7 +73,13 @@
|
||||
def field(name: String): String = req.obj.get(name).map(_.str).getOrElse("")
|
||||
id -> ujson.Arr(rowsFor(field("settings"), field("function"), field("file")): _*)
|
||||
}
|
||||
println(ujson.write(ujson.Obj.from(answers)))
|
||||
// A census of the graph, under a key no request id can collide with (ids are numbers). A frontend can
|
||||
// fail every file and STILL exit 0 with a valid, empty CPG -- `joern-parse` does not even propagate the
|
||||
// per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this.
|
||||
// It rides the batch rather than costing its own invocation, because JVM startup is what this whole
|
||||
// batching design exists to avoid.
|
||||
val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString))
|
||||
println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census))))
|
||||
} else {
|
||||
println(ujson.write(ujson.Arr(rowsFor(settings, function, file): _*)))
|
||||
}
|
||||
|
||||
@@ -185,7 +185,13 @@
|
||||
): _*
|
||||
)
|
||||
}
|
||||
println(ujson.write(ujson.Obj.from(answers)))
|
||||
// A census of the graph, under a key no request id can collide with (ids are numbers). A frontend can
|
||||
// fail every file and STILL exit 0 with a valid, empty CPG -- `joern-parse` does not even propagate the
|
||||
// per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this.
|
||||
// It rides the batch rather than costing its own invocation, because JVM startup is what this whole
|
||||
// batching design exists to avoid.
|
||||
val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString))
|
||||
println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census))))
|
||||
} else {
|
||||
println(ujson.write(ujson.Arr(rowsFor(operations, dischargers, function, file, operationRequires, dischargersByKind): _*)))
|
||||
}
|
||||
|
||||
@@ -280,7 +280,13 @@
|
||||
): _*
|
||||
)
|
||||
}
|
||||
println(ujson.write(ujson.Obj.from(answers)))
|
||||
// A census of the graph, under a key no request id can collide with (ids are numbers). A frontend can
|
||||
// fail every file and STILL exit 0 with a valid, empty CPG -- `joern-parse` does not even propagate the
|
||||
// per-file warnings -- so "no rows" and "no graph" are indistinguishable to the driver without this.
|
||||
// It rides the batch rather than costing its own invocation, because JVM startup is what this whole
|
||||
// batching design exists to avoid.
|
||||
val census = ujson.Arr(ujson.Obj("methods" -> cpg.method.size.toString, "files" -> cpg.file.size.toString))
|
||||
println(ujson.write(ujson.Obj.from(answers.toSeq :+ ("__census__" -> census))))
|
||||
} else {
|
||||
println(
|
||||
ujson.write(ujson.Arr(rowsFor(sources, sinks, sanitizers, function, parameterSources, file, callDepth, boundedSinks): _*))
|
||||
|
||||
Reference in New Issue
Block a user