feat(facts): the [[layout]] fact -- vendored trees are out, test paths are not product

Two decisions the maintainer made, as rows rather than constants. A vendored tree (vendor/, node_modules/,
third_party/) is somebody else's unit: out of the targets in preprocess and out of the graph through the
frontend's --exclude, which the scan hands to the backend because the CPG seam imports only the standard
library. A test path (tests/, test-*.php, *Test.php) stays in scope -- a test that reaches a sink still
names the sink -- but its regions are not shipped and order after the product's. Measured before the row:
207 of WP Statistics's 357 PHP files were vendored, and 14 of the 67 regions ahead of MW WP Form's CVE
were test files. The position benchmark applies both, so it measures the order the scan spends.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-11 14:27:50 +02:00
co-authored by Claude Fable 5.1
parent 9142cb1fb4
commit b821ac6e4e
9 changed files with 593 additions and 347 deletions
+8 -4
View File
@@ -278,7 +278,7 @@ class JoernBackend:
return has_cpg()
def build(self, root: Path, *, language: str = "") -> CpgResult | None:
def build(self, root: Path, *, language: str = "", exclude: Sequence[str] = ()) -> CpgResult | None:
"""Build a CPG for ``root``. ``None`` on any failure — the caller degrades, never guesses.
``language`` lets a failed ``joern-parse`` retry through the frontend directly, which is what Joern
@@ -312,7 +312,7 @@ class JoernBackend:
# A frontend whose failures are visible and retryable is worth more than one whose are not.
if language.lower() in _PREFER_FRONTEND:
self.last_failure = ""
shards, unparsed = self._build_sharded(root, scratch, language)
shards, unparsed = self._build_sharded(root, scratch, language, exclude=exclude)
if not shards and self.last_failure:
# A build that FAILED BY NAME is not one to retry through the other launcher: the graph the
# census could not load would be rebuilt the same and never asked again.
@@ -450,7 +450,9 @@ class JoernBackend:
)
return ""
def _build_sharded(self, root: Path, scratch: Path, language: str) -> tuple[tuple[Path, ...], tuple[str, ...]]:
def _build_sharded(
self, root: Path, scratch: Path, language: str, exclude: Sequence[str] = ()
) -> tuple[tuple[Path, ...], tuple[str, ...]]:
"""Build the tree, and give the files it refuses a CPG of their OWN rather than losing them.
A frontend that drops a file drops it from the only graph there is, and every question about that
@@ -465,7 +467,9 @@ class JoernBackend:
Returns the CPGs to query and the files that defeated even a shard of their own.
"""
# Excluded from the start, because one of them costs the entire graph rather than itself.
defective = self._files_with_frontend_defect(root, language)
# `exclude` is the caller's: the vendored trees the `[[layout]]` facts put out of scope. This seam
# imports only the standard library, so it is handed the names rather than reading the table.
defective = (*exclude, *self._files_with_frontend_defect(root, language))
if defective:
logger.warning(
"excluding %d file(s) under %s that php2cpg miscompiles (a `global` inside a closure): %s",