From 5be63bdfea035649d25580ffe7f01d49b786244a Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Fri, 11 Sep 2026 09:10:14 +0200 Subject: [PATCH] docs(specs): the whole-repository scan completes -- 72 minutes, CVE-2023-23488 found, 250 findings pmpro, 637 files, the product's own build path after 60963f5: build 97 s with overlays, evidence 76 s, taint 4,105 s for 348 real requests (11.8 s each), arbitrate 99 s, peak child RSS 2.4 GB. The CVE is at class.memberorder.php:936 among 250 entailed findings across 40 files -- the number the flow-aware-ranking spec now has to concentrate. callDepth measured on the same graph: a 1.4x time term and a 100x payload term (3,838 rows for ten requests at depth 3 against 38 at depth 1). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG --- .kiro/specs/contributor-scan/tasks.md | 23 +++++++++++++++++++++++ .kiro/specs/flow-aware-ranking/brief.md | 20 ++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/.kiro/specs/contributor-scan/tasks.md b/.kiro/specs/contributor-scan/tasks.md index 101c488..0780fcf 100644 --- a/.kiro/specs/contributor-scan/tasks.md +++ b/.kiro/specs/contributor-scan/tasks.md @@ -896,6 +896,29 @@ the build as a data table and reported in `unparsed`; (3) the hook half's seeds are the callback's own file. The payload diff between a raw and a saved-overlay pmpro graph was checked request by request: identical (sink, line, source kind, sanitized) sets, fewer repeated flow paths on the saved one. + - **`callDepth`, finally measured.** Same ten pairs, both halves on, the saved-overlay pmpro graph; the + fixed cost is now ~18 s (JVM plus an 8 s reload) and is included: + + | callDepth | 10 requests | per request, JVM incl. | rows | + |---|---|---|---| + | 0 | 107.5 s | 10.8 s | 3 | + | 1 | 124.9 s | 12.5 s | 38 | + | 3 | 150.5 s | 15.0 s | **3,838** | + + Depth is a 1.4× time term, not the 5× the sharding-era numbers implied -- but it is a **100× payload + term**: at depth 3 the same ten pairs return 3,838 flow rows, most of them the same (sink, source kind, + sanitized) evidence repeated once per path through the reachable set. Across 339 real requests that is + on the order of 130,000 rows to ship and parse, and the reporter collapses them only after they have + been paid for. The next cost to take out is in the query's row emission, not its graph work: one row + per (sink, source kind, sanitized), with a path count, would be the same evidence at 1% of the bytes. + - **The whole-repository scan completes, 2026-09-11.** pmpro, 637 files, the product's own build path + (overlays at build, seeds scoped, size guard), no judge, `OPENULTRASAST_CPG_QUERY_TIMEOUT=7200`: + build 97 s, evidence 76 s, **taint 4,105 s for 348 real requests = 11.8 s each**, arbitrate 99 s, + **total 72 minutes**, peak child RSS 2.4 GB, `tier_counts = {0: 2152, 1: 60, 2: 20, 3: 218, 4: 50}`. + **CVE-2023-23488 found at `class.memberorder.php:936:getMemberOrderByCode`**, among 250 entailed + findings across 40 files. First completion of this scan on this repository. The 2,400 s default is + still too low for it and stays -- a per-push scan of 4,463 regions is the flow-aware-ranking spec's + problem, and 250 findings is the number that spec now has to concentrate. - **Open, and the maintainer's call, not mine:** `includes/vendor/` is 207 of WP Statistics's 357 PHP files and is not excluded by `preprocess.IGNORED_DIRS`, so vendored third-party code is both scanned and in the graph. Whether a project's vendored code is in scope is policy; the size guard above is diff --git a/.kiro/specs/flow-aware-ranking/brief.md b/.kiro/specs/flow-aware-ranking/brief.md index e812614..2eed56a 100644 --- a/.kiro/specs/flow-aware-ranking/brief.md +++ b/.kiro/specs/flow-aware-ranking/brief.md @@ -99,6 +99,20 @@ A phase that cannot show that is not an improvement, it is a story. requests pruned (86%)**, evidence pass 94.5 s. The remaining **339 requests still hit the 2,400 s ceiling** — more than 7 s each. + **The scan finishes, 2026-09-11.** Not because of anything in this spec: contributor-scan 5.13 found + that every query batch had been recomputing Joern's dataflow overlay at load since PHP builds went + through php2cpg (54 s fixed cost per batch; an OutOfMemoryError on WP Statistics), that the hook half's + seeds were repository-wide, and that two vendored data tables were 60% of one graph. With the overlays + applied once at build time, the seeds scoped to the callback's file, and files over 1 MB excluded as + data, the same whole-repository scan on the 637-file plugin completes: **build 97 s (overlays included), + evidence 76 s, taint 4,105 s for 348 real requests (11.8 s each), arbitrate 99 s, total 72 minutes**, + `tier_counts = {0: 2152, 1: 60, 2: 20, 3: 218, 4: 50}`, peak child RSS 2.4 GB. **CVE-2023-23488 is + found at `class.memberorder.php:936`**, among 250 entailed findings across 40 files. The 250 is the next + fact to reckon with -- it is what "500 regions when 50 would do" looks like as output -- and it is phase + 2's material. `callDepth`, measured on the same graph, is a 1.4× time term and a 100× payload term + (3,838 flow rows for ten requests at depth 3 against 38 at depth 1); collapsing rows per evidence key + is the next cost out. + Phase 0 reported 1,148 ms per request after memoisation. That was an average over a 250-request set of which 88% were tier 0 and returned instantly; the ~30 real requests in it cost about 9.5 s each. Pruning removes exactly the cheap questions and keeps the expensive ones, so it cannot make those faster. The @@ -153,6 +167,12 @@ removes 53% of requests, and 1,177 × 6.63 s is still **2.2 hours** — nowhere requests issued, measured", and completion belongs to phase 2. Stating otherwise would have sent someone looking for a bug in phase 1 when phase 1 had done exactly what it can. +*Amended 2026-09-11:* completion arrived, and it belonged to neither phase 1 nor phase 2 -- it belonged to +the instrument (contributor-scan 5.13). Pruning was and is exact; what made 339 requests exceed 2,400 s was +a dataflow overlay recomputed per batch and a seed set computed repository-wide, and the scan now completes +in 72 minutes with the CVE found. The paragraph above stands as written because its reasoning was right on +the numbers it had; the numbers were the instrument's. + **Ranking's job is concentration, not inclusion.** The brief implies the CVE is missed because the ranker excluded it. It did not: CVE-2023-23488's region sits at position 390 of 4,463, inside the 500-region budget, and was examined. It goes unreported because the budget is spent on 500 regions when 50 would do, and 500