perf(cpg): one taint row per evidence key, with the path count

Ten real pmpro pairs at callDepth 3 returned 3,838 flow rows. Every field the driver reads is a property
of the sink or of (source kind, source, sanitized); only `length` varies per path, and the driver ranks
by the shortest. Grouped on that key with the minimum length and a `paths` count: 1,155 rows carrying
3,838 paths, same time. The specs are corrected too -- the first write-up called the depth-3 rows
"mostly the same evidence repeated", and 3.3x is what the measurement says; the other 30x is distinct
sources.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-11 09:14:25 +02:00
co-authored by Claude Fable 5.1
parent 5be63bdfea
commit 0f35578d5b
3 changed files with 45 additions and 30 deletions
+7 -6
View File
@@ -905,12 +905,13 @@
| 1 | 124.9 s | 12.5 s | 38 | | 1 | 124.9 s | 12.5 s | 38 |
| 3 | 150.5 s | 15.0 s | **3,838** | | 3 | 150.5 s | 15.0 s | **3,838** |
Depth is a 1.4× time term, not the 5× the sharding-era numbers implied -- but it is a **100× payload Depth is a 1.4× time term, not the 5× the sharding-era numbers implied -- and a 100× payload term,
term**: at depth 3 the same ten pairs return 3,838 flow rows, most of them the same (sink, source kind, 3,838 rows against 38. **How much of that was duplication, measured:** the query now emits one row per
sanitized) evidence repeated once per path through the reachable set. Across 339 real requests that is (sink, source kind, source, sanitized) with the shortest `length` and a `paths` count, and the same ten
on the order of 130,000 rows to ship and parse, and the reporter collapses them only after they have requests at depth 3 return **1,155 rows carrying 3,838 paths** -- a 3.3× collapse, exact by
been paid for. The next cost to take out is in the query's row emission, not its graph work: one row accounting, at the same time (15.9 s/request). The other 30× is depth 3 finding more distinct sources
per (sink, source kind, sanitized), with a path count, would be the same evidence at 1% of the bytes. per sink, which is evidence, not repetition; the first write-up of this table called it "mostly the
same evidence repeated" and that was a guess the measurement did not support.
- **The whole-repository scan completes, 2026-09-11.** pmpro, 637 files, the product's own build path - **The whole-repository scan completes, 2026-09-11.** pmpro, 637 files, the product's own build path
(overlays at build, seeds scoped, size guard), no judge, `OPENULTRASAST_CPG_QUERY_TIMEOUT=7200`: (overlays at build, seeds scoped, size guard), no judge, `OPENULTRASAST_CPG_QUERY_TIMEOUT=7200`:
build 97 s, evidence 76 s, **taint 4,105 s for 348 real requests = 11.8 s each**, arbitrate 99 s, build 97 s, evidence 76 s, **taint 4,105 s for 348 real requests = 11.8 s each**, arbitrate 99 s,
+3 -2
View File
@@ -110,8 +110,9 @@ A phase that cannot show that is not an improvement, it is a story.
found at `class.memberorder.php:936`**, among 250 entailed findings across 40 files. The 250 is the next found at `class.memberorder.php:936`**, among 250 entailed findings across 40 files. The 250 is the next
fact to reckon with -- it is what "500 regions when 50 would do" looks like as output -- and it is phase fact to reckon with -- it is what "500 regions when 50 would do" looks like as output -- and it is phase
2's material. `callDepth`, measured on the same graph, is a 1.4× time term and a 100× payload term 2's material. `callDepth`, measured on the same graph, is a 1.4× time term and a 100× payload term
(3,838 flow rows for ten requests at depth 3 against 38 at depth 1); collapsing rows per evidence key (3,838 flow rows for ten requests at depth 3 against 38 at depth 1). Collapsing rows per evidence key
is the next cost out. takes 3.3× of that out (1,155 rows carrying 3,838 paths, exact); the rest is depth 3 naming more
distinct sources per sink, which is evidence the ranker gets to use.
Phase 0 reported 1,148 ms per request after memoisation. That was an average over a 250-request set of Phase 0 reported 1,148 ms per request after memoisation. That was an average over a 250-request set of
which 88% were tier 0 and returned instantly; the ~30 real requests in it cost about 9.5 s each. Pruning which 88% were tier 0 and returned instantly; the ~30 real requests in it cost about 9.5 s each. Pruning
+35 -22
View File
@@ -619,28 +619,41 @@
boundShape.matcher(c).find() && argNames.exists(a => mentionsToken(c, List(a))) boundShape.matcher(c).find() && argNames.exists(a => mentionsToken(c, List(a)))
) )
flows.map { flow => // ONE ROW PER EVIDENCE KEY, not per path. Every field the driver reads is a property of the sink or of
val elements = flow.elements.map(_.code).l // (source kind, source, sanitized); only `length` varies per path, and the driver ranks by the shortest.
val sanitized = sanitizerNames.nonEmpty && flow.elements.l.exists(node => sanitizesHere(node)) // Emitting every path was a 100x payload term: ten real requests at callDepth 3 returned 3,838 rows
val sourceKind = sourceKindOf(flow.elements.l.headOption) // for 38 distinct keys, and a whole-repository scan would ship on the order of 130,000 rows to have
ujson.Obj( // the reporter collapse them after they were paid for. `paths` keeps the count. Task 5.13.
"sink" -> sink.code.take(200), flows
"sourceKind" -> sourceKind, .map { flow =>
"sinkLine" -> sink.lineNumber.getOrElse(-1).toString, val elements = flow.elements.map(_.code).l
"sinkMethod" -> sink.method.name, val sanitized = sanitizerNames.nonEmpty && flow.elements.l.exists(node => sanitizesHere(node))
// Where the sink actually is. With callDepth > 0 that need not be the region's own file, and a val sourceKind = sourceKindOf(flow.elements.l.headOption)
// finding reported against the handler's file when the bug is in another module is unactionable. ((sourceKind, elements.headOption.getOrElse("").take(200), sanitized), elements.size)
"sinkFile" -> sink.method.filename, }
"source" -> elements.headOption.getOrElse("").take(200), .groupBy(_._1)
"sanitized" -> sanitized, .toList
"length" -> elements.size, .sortBy(_._1)
"sinkArity" -> arity, .map { case ((sourceKind, source, sanitized), paths) =>
"sinkArg0Literal" -> arg0Literal, ujson.Obj(
"bounded" -> bounds.nonEmpty, "sink" -> sink.code.take(200),
"bound" -> bounds.headOption.getOrElse("").take(120), "sourceKind" -> sourceKind,
"inLabeledScope" -> (function.isEmpty || nestedInLabeled(sink.method) || reachableMethods.contains(sink.method.fullName)) "sinkLine" -> sink.lineNumber.getOrElse(-1).toString,
) "sinkMethod" -> sink.method.name,
} // Where the sink actually is. With callDepth > 0 that need not be the region's own file, and a
// finding reported against the handler's file when the bug is in another module is unactionable.
"sinkFile" -> sink.method.filename,
"source" -> source,
"sanitized" -> sanitized,
"length" -> paths.map(_._2).min,
"paths" -> paths.size,
"sinkArity" -> arity,
"sinkArg0Literal" -> arg0Literal,
"bounded" -> bounds.nonEmpty,
"bound" -> bounds.headOption.getOrElse("").take(120),
"inLabeledScope" -> (function.isEmpty || nestedInLabeled(sink.method) || reachableMethods.contains(sink.method.fullName))
)
}
} }
rows rows