fix(scan): prune a tier-zero question unless ITS partition is the incomplete one

The mechanism already existed. A question whose evidence vector says no sink of its family is in
reach cannot produce a flow, and tier-zero pruning defers it. That was switched off for the whole
scan whenever any of twelve degradation reasons appeared, and a multi-language repository always
carries two of them, so the measured plugin arbitrated every question it could ask. Third instance
this week of one blast radius, and the invariant written into the design on Monday named it.

A prune blocker that names a partition now withholds pruning from that partition questions and from
nobody else. cross_partition_semantics_unresolved leaves the blocker set entirely, for the reason it
left the integrity gaps: a PHP family sink is PHP, so a second language cannot be why its vector saw
none.

Measured on Paid Memberships Pro. At 25 regions arbitration falls from 127 questions to 34 with 93
recorded deferred as tier_zero, and 19 findings before and after, identical family by family. At 500
it falls from 2,526 to 406 with 2,120 deferred.

It is not sufficient, and that is the useful half. The 500-region run still fails in the same way at
the same cost, because 406 questions at roughly one and a half seconds each still exceed the
300-second per-batch ceiling. This lever moved the volume; the ceiling is a separate limit.

The census now records what a scan skipped and why, per family, so a cheaper scan and a narrower one
stop looking identical: a family whose every question was pruned reports no sink of this family in
reach rather than region offered, no question asked.

1,420 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-22 08:06:15 +02:00
co-authored by Claude Opus 5
parent b0500ecfa0
commit 3b6e642433
4 changed files with 105 additions and 22 deletions
+37 -22
View File
@@ -357,28 +357,38 @@ def _scan_repository_impl(
# Authoritative selection is made once, from the actual final ranker order.
# Stable question IDs are the IDs sent to every execution request below.
scope_limits = (
replace(limits, prune_tier0=False)
if any(
d.get("reason")
in {
"files_unparsed",
"cpg_empty",
"partition_file_census_incomplete",
"partition_file_census_unavailable",
"cpg_sharded",
"cross_partition_semantics_unresolved",
"vendor_semantics_unresolved",
"symlink_context_unresolved",
"frontend_unsupported",
"source_unreadable",
"ambiguous_frontend_path",
"typescript_property_support_unvalidated",
}
for d in degradations
)
else limits
)
#
# A tier-zero vector says no sink of this family is in reach, and a question with no sink cannot produce a
# flow. Pruning those is what makes a large population affordable: arbitration costs one to two seconds a
# question and a 673-file plugin asks 2,526 of them, while the structural pass that produced the vectors
# answered 17,700 requests in 44 s.
#
# It is switched off where the graph might be missing the very sink the vector did not see -- an unparsed
# file, an excluded tree, a shard that cannot see its neighbour. That is right, and it was applied to the
# whole scan, which is not: a census gap belongs to ONE partition, and a JavaScript file missing from the
# JavaScript graph is no reason to arbitrate every tier-zero question a PHP family asks. On the measured
# plugin that single distinction is the difference between 2,526 arbitrated questions and the few hundred
# that could say anything.
#
# `cross_partition_semantics_unresolved` leaves this set entirely, for the reason it left the integrity
# gaps: a PHP family's sink is PHP, so the existence of a second language cannot be why its vector saw none.
prune_blockers = {
"files_unparsed",
"cpg_empty",
"partition_file_census_incomplete",
"partition_file_census_unavailable",
"cpg_sharded",
"vendor_semantics_unresolved",
"symlink_context_unresolved",
"frontend_unsupported",
"source_unreadable",
"ambiguous_frontend_path",
"typescript_property_support_unvalidated",
}
blocking = [d for d in degradations if d.get("reason") in prune_blockers]
# A blocker that names no partition is about the whole read and stops pruning everywhere.
scope_limits = replace(limits, prune_tier0=False) if any(not d.get("census_language") for d in blocking) else limits
unprunable_languages = frozenset(str(d["census_language"]) for d in blocking if d.get("census_language"))
scope, work = _scope_work(
ordered_regions,
evidence_by_pair,
@@ -389,6 +399,7 @@ def _scan_repository_impl(
change_context=change_context,
context_gaps=context_gaps,
execution_budget=execution_budget,
unprunable_languages=unprunable_languages,
)
outcomes: dict[str, QuestionOutcome] = {}
@@ -649,6 +660,9 @@ def _scope_work(
change_context: ChangeContext | None = None,
context_gaps: set[QuestionIdentity] | None = None,
execution_budget: ExecutionBudget | None = None,
# Languages whose graph might be missing the sink their vectors did not see. Pruning is withheld from
# their questions and from nobody else's.
unprunable_languages: frozenset[str] = frozenset(),
) -> tuple[ScopeDecision, list[tuple[str, ScanRegion, ArbiterSpec]]]:
"""Consume the ranker's final order once; the returned work IS selected scope."""
selected: list[RankedQuestion] = []
@@ -707,6 +721,7 @@ def _scope_work(
if index >= limits.max_regions
else "tier_zero"
if limits.prune_tier0
and identity.language not in unprunable_languages
and identity not in (context_gaps or set())
and isinstance(spec, TaintSpec)
and vector is not None