mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
docs(g2): engine-arm wall diagnosed -- ~200s fixed overhead, change-scoping insufficient
Records the full G2 engine-detection finding on main (code attempt parked on branch engine-change-scoped-evidence-pass, unmerged). The engine arm has ~200s intrinsic fixed overhead -- ~48s CPG build + ~150s first-query graph-load+census (measured uncontended: build 48s, evidence 243.79s over 1015 pairs, query 0) -- so a 300s budget cannot reach the flow queries. Pair-count change-scoping trims only the ~86s per-pair tail; the warm-session env times out at startup contended and spares only subsequent batches. Real fix is clean-host engine work (reliable session startup + graph preload, or census-cost reduction). Parked to finish G0. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
15992d3090
commit
df09fe4030
@@ -83,6 +83,45 @@ detection is being lost, not in Joern's capability.
|
|||||||
pass already returns, tier those in a second small pass — exact reach-into-unchanged, more code.
|
pass already returns, tier those in a second small pass — exact reach-into-unchanged, more code.
|
||||||
- (iii) **Per-pair speedup:** attack the ~235 ms/pair (warm server / batching) rather than the pair count —
|
- (iii) **Per-pair speedup:** attack the ~235 ms/pair (warm server / batching) rather than the pair count —
|
||||||
helps every scan but is a deeper engine change and does not by itself bound a huge repo.
|
helps every scan but is a deeper engine change and does not by itself bound a huge repo.
|
||||||
|
|
||||||
|
**CORRECTION (2026-10-11, after implementing (i) + four live re-verifies — the cost model was wrong).**
|
||||||
|
Scope + bound was built and tested (change-first ordering, adaptive budget-fitted portioning, `_context_pass`
|
||||||
|
scoped to changed files, honest `evidence_pass_bounded` degradation; branch `engine-change-scoped-evidence-pass`).
|
||||||
|
It correctly cut the tiered pair count — but `query_seconds` stayed **0** every time. The live numbers show why:
|
||||||
|
|
||||||
|
| pairs tiered | run_batch calls | evidence s | build s |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1015 | 1 | 238 | 52 |
|
||||||
|
| 512 | 1 | 219 | 68 |
|
||||||
|
| 64 | 2 | 220 | 68 |
|
||||||
|
| 4 (probe) | 1 | 160 | 127 |
|
||||||
|
|
||||||
|
Evidence time is **flat regardless of pair count** → the cost is **per-*call* fixed** (~150–220 s: loading the CPG
|
||||||
|
into the query engine + the census the evidenceOnly batch computes), ~85 ms/pair on top. Pair-count scoping
|
||||||
|
therefore cannot free budget, and portioning re-pays the fixed cost per call (so (i) can be counterproductive).
|
||||||
|
Build is also large and swings 52→127 s — two engines thrashing the 7 GB host (the record-07 baseline shares it).
|
||||||
|
The real levers are now (iii)-shaped, not (i): **warm/persistent query engine or a cached census** to kill the
|
||||||
|
~150 s per-call fixed cost, and/or a **bigger engine-arm budget** than 300 s; plus a **clean (uncontended)
|
||||||
|
measurement**, because contention alone may be inflating build + fixed cost 2–3×. Scope+bound is NOT merged.
|
||||||
|
|
||||||
|
**UNCONTENDED measurement (2026-10-11, baseline paused via SIGSTOP + container kill, then resumed).** Shipped
|
||||||
|
engine, one evidence call over 1,015 pairs: `build=48 s`, `evidence=243.79 s`, `query=0`. The evidence pass is
|
||||||
|
**the same ~244 s uncontended as ~238 s contended** → the cost is **INTRINSIC, not contention**; contention only
|
||||||
|
inflated `build` (48 → 127 s) and added run-to-run noise. The model is `evidence ≈ 150 s fixed (CPG load into the
|
||||||
|
query engine + the census the evidenceOnly batch computes) + ~93 ms/pair` (244 = 150 + 1015×0.093). Consequences:
|
||||||
|
- **~200 s of fixed overhead** (`build` + evidence-load) precedes any analysis, so a 300 s budget leaves only
|
||||||
|
~100 s for everything else. This is why `query=0` even on a clean host.
|
||||||
|
- **Change-scoping still helps, but only by cutting the per-pair tail** (~1,015×93 ms ≈ 95 s): one evidence call
|
||||||
|
over ~30 changed pairs is ~150 s + 3 s instead of 244 s, which would free ~90 s — possibly enough for analysis,
|
||||||
|
but marginal.
|
||||||
|
- **It must be ONE call, not portioned.** The ~150 s is a first-call load, so the adaptive probe attributed it to
|
||||||
|
its 4 probe pairs (per-pair≈37 s), concluded the budget fit ~0 more, and tiered only 4 — the implemented
|
||||||
|
scope+bound is doubly wrong for this cost model. The correct shape: change-first candidates, count-capped, ONE
|
||||||
|
evidence call, `deadline=None`.
|
||||||
|
- **Pre-push viability:** the ~200 s fixed floor means the engine arm cannot be a fast (≤30 s) pre-push check on a
|
||||||
|
repo this size without a warm/persistent session or a cached census. A larger budget (e.g. 600 s) makes the
|
||||||
|
benchmark arm run but is not a realistic pre-push deadline. The lever that matters for the product is killing
|
||||||
|
the ~150 s fixed cost, not scoping pairs.
|
||||||
- [ ] **2. Seed analysis from the change.** Prioritize (or restrict first-pass to) the changed declarations plus
|
- [ ] **2. Seed analysis from the change.** Prioritize (or restrict first-pass to) the changed declarations plus
|
||||||
their call/data neighborhood over repo-wide static rank, within budget.
|
their call/data neighborhood over repo-wide static rank, within budget.
|
||||||
- [ ] **3. Soundness.** Keep cross-file dataflow INTO the changed function (don't scope to literal changed lines);
|
- [ ] **3. Soundness.** Keep cross-file dataflow INTO the changed function (don't scope to literal changed lines);
|
||||||
|
|||||||
Reference in New Issue
Block a user