mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 19:59:26 +00:00
feat(model): the envelope checkout, and no to Joern server mode
contributor-scan 2.2 for libpng and the 2.3 decision it exists to inform. 89,082 lines of C across 93 files, under a 2GB heap cap: CPG build 16.1s (25s cold) Joern queries 250.0s arbitration 122.0s (105 calls, $0.0122) whole scan 446s peak RSS 911MB regions 121, all arbitrated, 32 unasked CVE-2025-66293 out_of_scope, as the recipe declares Memory is not the constraint a fleet runner would hit: 911MB leaves room for the rest of the job on a 2-vCPU/7GB runner. The memory pressure seen while building this was concurrent JVMs on one desktop, not repository size. **Server mode: no.** The query phase splits taint 243.3s / config 10.7s over two invocations, and fixed overhead measured on the same CPG is ~13s per invocation. So ~26s of the 250s is JVM startup and ~224s is CPGQL evaluation: a warm server removes 6% of the scan and none of the part that costs. The driver now records query seconds per kind, because a single total cannot tell startup from evaluation and the two answer this oppositely. The small repository pointed the other way. vampi's 34.3s over three kinds is roughly three startups and almost no evaluation, which made a warm JVM look worth having; the ratio inverts with size. That inversion is why the task was sequenced after a real repository rather than decided on a 520-line file. On whether one whole-repository CPG is the only shape for a horizontal fleet: no, and it is also not where the pressure is. The build parallelises and costs 16s; the query does not shard and costs 250s, so splitting a repository across runners divides the cheap phase -- and after 2.4 a shard boundary hides exactly the cross-module flows that matter. Scoping to a contributor's diff cuts the expensive phase instead. 911MB for 89k lines extrapolates to roughly 5GB at a million lines, which would exceed a standard runner, so the ceiling is real and further out than this checkout can locate. Two defects recorded rather than fixed, both invisible at vampi scale: 2.9, all 26 entailed findings are the same site reported 26 times, because regions that share a sink each emit their own; and 2.10, 22 of the 30 highest-ranked regions are libpng's `contrib/` example programs rather than the library. Also ignores Joern's `workspace/`, which a maintainer querying a CPG by hand from the repository root will create. Gates byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
d15985f695
commit
68b4e551f3
@@ -890,6 +890,7 @@ def _model_payload(result: object, *, unjudged_paths: tuple[str, ...] = ()) -> d
|
||||
"seconds": getattr(result, "seconds", 0.0),
|
||||
"build_seconds": getattr(result, "build_seconds", 0.0),
|
||||
"query_seconds": getattr(result, "query_seconds", 0.0),
|
||||
"query_seconds_by_kind": dict(getattr(result, "query_seconds_by_kind", {}) or {}),
|
||||
"arbitrate_seconds": getattr(result, "arbitrate_seconds", 0.0),
|
||||
}
|
||||
|
||||
|
||||
@@ -83,6 +83,9 @@ class ModelScanResult:
|
||||
# a single figure cannot answer any of them.
|
||||
build_seconds: float = 0.0
|
||||
query_seconds: float = 0.0
|
||||
# Per query kind, because the total cannot say whether the cost is JVM startup (which a warm server
|
||||
# would remove) or CPGQL evaluation (which it would not). That is the whole of the server-mode decision.
|
||||
query_seconds_by_kind: Mapping[str, float] = field(default_factory=dict)
|
||||
degradations: tuple[Mapping[str, object], ...] = ()
|
||||
|
||||
@property
|
||||
@@ -141,8 +144,10 @@ def scan_repository(
|
||||
# per family put a ten-line file at four minutes and a thousand regions at roughly fifty hours -- while
|
||||
# the queries themselves are milliseconds once the CPG is loaded.
|
||||
rows_by_id: dict[str, list[object]] = {}
|
||||
per_kind: dict[str, float] = {}
|
||||
query_started = time.monotonic()
|
||||
for kind, requests in _grouped(work).items():
|
||||
kind_started = time.monotonic()
|
||||
batch = getattr(cpg, "run_batch", None)
|
||||
if callable(batch):
|
||||
rows_by_id.update(batch(kind, requests) or {})
|
||||
@@ -150,6 +155,7 @@ def scan_repository(
|
||||
for rid, params in requests.items():
|
||||
answered = cpg.run(kind, params)
|
||||
rows_by_id[rid] = list(answered) if isinstance(answered, list) else []
|
||||
per_kind[kind] = round(time.monotonic() - kind_started, 2)
|
||||
query_seconds = round(time.monotonic() - query_started, 2)
|
||||
|
||||
# Phase 3: arbitrate from the rows already in hand. The arbiters are unchanged: each is handed a
|
||||
@@ -212,6 +218,7 @@ def scan_repository(
|
||||
seconds=round(time.monotonic() - started, 2),
|
||||
build_seconds=build_seconds,
|
||||
query_seconds=query_seconds,
|
||||
query_seconds_by_kind=per_kind,
|
||||
degradations=tuple(degradations),
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user