mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-20 05:29:28 +00:00
perf(model): one Joern invocation per query kind, not one per region
JVM startup dominated a scan: a call per region per family put a ten-line file at 275s over three regions, and would have put a thousand regions at roughly fifty hours. The queries themselves are milliseconds once the CPG is loaded, so the cost was almost entirely process starts. scan_repository() is now collect -> batch -> arbitrate. The three .sc scripts accept a `requests` JSON map and answer all of them in one run, keeping the single-request form so the committed measurements stay reproducible byte-for-byte. Arbiters are untouched: each is handed a CpgResult whose `run` returns its own prefetched rows. The batcher builds parameters by calling request_params() extracted from taint.py, dominance.py and config_value.py rather than reconstructing them. A second copy that drifts from the arbiter's is the exact shape of the five family-assumption bugs the predecessor spec spent a session finding; there is now one definition per family and the batcher borrows it. Regions are also deduplicated by (path, function), collapsing the nameless entry-point rows that were producing `model:injection:?:10`. Measured on the same ten-line file: 275s -> 106s, three duplicate findings -> one `model:injection:ping:10`, rung unchanged at model_entailed. Gates byte-identical. Server mode is recorded as task 2.3, deliberately sequenced after the repository measurement: whether the remaining ~100s floor is worth a process model, a weakened subprocess seam and a Scala-evaluating HTTP endpoint depends on a build-time-to-query-time ratio only 2.2 produces. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
e67f3ee2f0
commit
c35426e186
@@ -64,8 +64,11 @@ def regions_for(entries: Sequence[object], targets: Sequence[object]) -> tuple[S
|
||||
if language:
|
||||
by_path[str(getattr(target, "path", ""))] = language
|
||||
|
||||
regions: list[ScanRegion] = []
|
||||
covered: set[str] = set()
|
||||
# One region per (path, function). The mapper can emit several entry points for a single file -- on a
|
||||
# ten-line Flask file it produced three, one named and two nameless -- and turning each into a region
|
||||
# scanned the same code repeatedly, multiplying Joern invocations for no new coverage. Where several
|
||||
# entries agree on a region, the highest access rank wins, because that is the one worth looking at first.
|
||||
best: dict[tuple[str, str | None], ScanRegion] = {}
|
||||
for entry in entries:
|
||||
path = str(getattr(entry, "path", ""))
|
||||
language = by_path.get(path)
|
||||
@@ -74,17 +77,18 @@ def regions_for(entries: Sequence[object], targets: Sequence[object]) -> tuple[S
|
||||
families = _families(language, has_handler=True)
|
||||
if not families:
|
||||
continue
|
||||
covered.add(path)
|
||||
regions.append(
|
||||
ScanRegion(
|
||||
path=path,
|
||||
function=(getattr(entry, "function_name", None) or None),
|
||||
language=language,
|
||||
families=families,
|
||||
rank=_ACCESS_RANK.get(str(getattr(entry, "access_level", "")), 0.5),
|
||||
source="entry_point",
|
||||
)
|
||||
)
|
||||
function = getattr(entry, "function_name", None) or None
|
||||
rank = _ACCESS_RANK.get(str(getattr(entry, "access_level", "")), 0.5)
|
||||
key = (path, function)
|
||||
current = best.get(key)
|
||||
if current is None or rank > current.rank:
|
||||
best[key] = ScanRegion(path=path, function=function, language=language, families=families, rank=rank, source="entry_point")
|
||||
|
||||
# A nameless entry point IS the file. Where a named region already covers that file it adds nothing but
|
||||
# a second full-file pass, so it is dropped -- unless the file has no named region at all.
|
||||
named_paths = {path for path, function in best if function is not None}
|
||||
regions = [region for (path, function), region in best.items() if function is not None or path not in named_paths]
|
||||
covered = {region.path for region in regions}
|
||||
|
||||
for path, language in by_path.items():
|
||||
if path in covered:
|
||||
|
||||
Reference in New Issue
Block a user