fix(cpg): a query that could not be asked is not an empty answer

contributor-scan 5.7, and it was the wrong question.

**The 117k-line pmpro CPG does not load.** Importing it throws in a
ForkJoinParallelCpgPassWithAccumulator during overlay application, so every
query against it returns nothing. It is not a call-graph question at all.

Isolated by running the ARBITER against two prebuilt CPGs rather than
writing a sixth ad-hoc Scala probe, four of which had already failed on my
own mistakes:

  small wpdb CPG   get_order_by_code       -> 1 model_entailed
  pmpro CPG        getMemberOrderByCode    -> 0

Both have their source and sink in ONE function, both were asked with
parameter sources and depth 3, same arbiter and same spec. The CPG is the
only difference.

That clears the facts, which is worth stating: `$wpdb` sinks work, the
parameter-source rule works, and a PHP parameter reaching `$wpdb->get_var`
through a concat is entailed. Whether php2cpg resolves `new X()` and
`$this->method()` is still unmeasured -- it cannot be measured on a CPG
that will not load -- and is recorded as 5.10 along with finding the size
boundary.

**And the defect underneath it, which is the fourth quiet failure found on
this one checkout.** `query_batch` returned `{}` for a failure and `{}` for
a genuine empty answer, so a caller could not tell "could not ask" from
"nothing found". Every query on this scan failed at load and it reported
500 regions examined, no findings, and no degradation: a vulnerable plugin,
clean.

A failed batch now returns None, unparseable output counts as a failure
rather than an empty answer, and the driver records a `query_failed`
degradation naming the kind and the request count.

This is the failure mode the rung ladder structurally cannot catch. Every
other honesty mechanism here labels a VERDICT -- suspicion,
regions_unjudged, regions_unasked, stated family limits -- and when no
verdict is produced there is nothing to label.

Gates byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-09 17:15:51 +02:00
co-authored by Claude Opus 5
parent db273dde53
commit 21fdb554aa
8 changed files with 133 additions and 20 deletions
+4 -1
View File
@@ -147,7 +147,10 @@ def test_a_failed_batch_yields_no_rows_for_any_request(monkeypatch: pytest.Monke
stderr = "boom"
backend = JoernBackend(runner=lambda c, **k: _Bad())
assert backend.query_batch(Path("/tmp/c.bin"), "taint", {"r1": {}}) == {}
assert backend.query_batch(Path("/tmp/c.bin"), "taint", {"r1": {}}) is None, (
"None means COULD NOT ASK; {} would mean the engine answered and found nothing, and a caller that "
"cannot tell them apart reports a failed scan as a clean repository"
)
def test_a_batch_result_that_omits_a_request_returns_nothing_for_it() -> None:
+26
View File
@@ -364,3 +364,29 @@ def test_two_families_at_one_site_stay_two_findings() -> None:
]
assert len(_deduplicated(both)) == 2
def test_a_query_the_engine_could_not_answer_is_recorded_not_ignored() -> None:
"""contributor-scan 5.7. A 117k-line PHP CPG threw while loading, so every query returned nothing -- and
the scan reported 500 regions examined, no findings, and no degradation. A failed query and a clean
repository looked identical.
This is the failure mode the rung ladder cannot catch: every other honesty counter labels a VERDICT, and
here none was produced to label.
"""
from openultrasast.cpg.backend import CpgResult
from openultrasast.model.scan import scan_repository
class _Backend:
def available(self) -> bool:
return True
def build(self, root, **kwargs): # type: ignore[no-untyped-def]
return CpgResult(cpg_path=Path("cpg.bin"), run=lambda q, p: [], run_batch=lambda q, r: None)
result = scan_repository(Path("/repo"), (_region(),), backend=_Backend(), client=None, model="")
failures = [d for d in result.degradations if d.get("reason") == "query_failed"]
assert failures, "a query that could not be asked must be reported"
assert failures[0]["kind"] == "taint"
assert result.findings == ()