measure(php): quantify what remains once the malformed-CPG bug is excluded

The graph bug is fixed and was never what these runs were hitting. joernio/joern#6281, this
project's own report, is a PHP `global` inside a closure giving a node two AST parents, which made
the dataflow overlay throw and cost the whole graph. Filed against 4.0.623, fixed by #6270 in
v4.0.625, which the image has pinned since before any measurement recorded under task 13.2.
Verified today on the real trigger rather than a transcription of the repro, which does not
reproduce: PMPro's includes/fields.php builds and applies the default overlays cleanly on 4.0.625
and on 4.0.631, files=2 methods=158, no ReachingDefPass assertion.

What remains is runtime. A taint batch of 2,526 questions over the 673-file checkout does not
complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed
nothing but how long the failure took: query time went from 363 s to 1,874 s, and zero questions
completed in both. The eight-way split retry did not rescue it either, finishing its parts in
about three minutes against a deadline with 3,300 seconds still on it, which is worth its own
look.

The engine pin stays at 4.0.625. An image on 4.0.631, today's release, was built and measured and
is indistinguishable on this subject: 627.7 s against 619.1 s, 363.4 s against 367.3 s of query
time, zero completed in both. There is no demonstrated benefit to weigh against re-verifying every
JavaScript and Python measurement on a new engine. Should that change, the bump is three values
rather than a port: the upstream AstGenRunner.scala is byte-identical in 4.0.631 and its patch
anchor still appears exactly once, so only the jar path, the jar checksum and the source URL tag
move. Both are recorded.

Also closes task 14.5. Neither prototype pollution nor deserialization has a subject that can
answer, each for its own reason. Deserialization's mechanism exists on exactly one available
subject and that subject cannot be queried at all. Prototype pollution has no available subject
containing the mechanism, so its ability to answer is established on a control instead.

1,417 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

.kiro/specs/pre-push-safety-net/status.md

@@ -188,6 +188,14 @@ server.js now carries both its permissive session and its disabled template auto
 with its own site; the escaping defect had been detected on every run since task 14.3 and
 reported on none, because an earlier setting in the same function won the single verdict.
 
+The PHP record is corrected. The malformed-CPG bug this project reported upstream, a `global`
+inside a closure giving a node two AST parents, was fixed in Joern 4.0.625 and the image has
+pinned that version throughout; it is verified today on the real trigger file, on 4.0.625 and
+4.0.631 alike. What remains under task 13.2 is runtime alone: a 2,526-question batch over a
+673-file plugin does not finish in thirty minutes, and a six-fold larger ceiling only made the
+failure take longer. The engine pin stays where it is, because the newer release measures the
+same here.
+
 Closing the second gap surfaced four defects, three of which were false positives it would
 otherwise have shipped: a permissive-value set shared across settings whose booleans have
 opposite polarity, a dotted setting matched on its trailing segment alone, a flow query asked of
@@ -202,4 +210,5 @@ Evidence: [family detection census](../../../benchmarks/measurements/2026-09-19-
 [access control across the module split](../../../benchmarks/measurements/2026-09-20-access-control-module-split.json),
 [collection read obligation](../../../benchmarks/measurements/2026-09-20-collection-read-obligation.json),
 [sibling scope and identity](../../../benchmarks/measurements/2026-09-20-sibling-scope-and-identity.json),
-[every permissive setting](../../../benchmarks/measurements/2026-09-20-every-permissive-setting.json).
+[every permissive setting](../../../benchmarks/measurements/2026-09-20-every-permissive-setting.json),
+[PHP runtime after the graph fix](../../../benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json).

.kiro/specs/pre-push-safety-net/tasks.md

@@ -458,6 +458,7 @@ Preserve all completed task evidence below.
   - _Two harness defects found and fixed on the way, each of which had made an earlier run look like a transfer failure:_ the manifest labels a case by what it is while the id says what the comparison does, and filtering on one while keying expectations by the other ran a single case; and the per-query ceiling is taken as the minimum with the remaining transaction time, so raising only the deadline could never have helped.
   - _Retried 2026-09-18 with batch splitting, still not established._ A batch that fails whole is now retried in eight parts, which a probe made look promising: the evidence pass answers all 4,425 PMPro regions in one batch in 29.7 s with a 681 KB response, and cost is fixed rather than per request, so neither count nor size is near a limit. That probe sent `evidenceOnly=true` and therefore measured the structural pass, not the failing one; recorded because those numbers would otherwise read as evidence that PMPro's taint work is cheap. With a 4500-second deadline and a 900-second query ceiling the run was stopped after 26 minutes with no case completed. Splitting also makes an unrecoverable batch slower to abandon, since each part burns its own share of the ceiling.
   - _Correction 2026-09-20: the malformed-CPG bug is fixed and was never what these runs were hitting._ joernio/joern#6281, this project's own report, is a PHP `global` inside a closure producing a node with two AST parents, which made the dataflow overlay throw and cost the WHOLE graph. It was filed against 4.0.623 and fixed by #6270 in **v4.0.625**, which is the version the image has pinned since before any measurement here. Verified today on the real trigger: PMPro's `includes/fields.php`, the file the original bisection found, builds and applies the default overlays cleanly on 4.0.625 and on 4.0.631. So what remains under this task is a RUNTIME failure and must not be re-diagnosed as the graph bug.
+  - _Quantified 2026-09-20, and it is a runtime failure._ A taint batch of 2,526 questions over the 673-file checkout does not complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed nothing but how long the failure took: query time went from 363 s to 1,874 s and zero questions completed in both. The eight-way split retry did not rescue it, finishing its parts in about three minutes against a deadline with 3,300 seconds still on it, which is worth its own look. Joern 4.0.625 and 4.0.631 are indistinguishable here, 627.7 s against 619.1 s. The engine pin stays at 4.0.625: the newer release was built and measured and is equivalent on this subject, so there is no benefit to weigh against re-verifying every JavaScript and Python measurement. Evidence: `benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json`.
   - **Blocker still unidentified; investigation paused 2026-09-19 to return to vulnerability work.** Six hypotheses were measured and eliminated: batch size (17,700 requests answer in 44 s with a 2.6 MB response), dataflow cost (50 execution requests in 24 s), call depth 3 with parameter sources (24.7 s for 20), argument length (a 16 KB hook table against a 128 KB cap), hook callback resolution (1 s difference with 233 hooks), and build or sharding (79 s, no shards). Preparation measures 273.5 s. Every reproduction of the failing 22,125-request batch outside the push path answers in under a minute, and the difference from the production invocation is not known. The exception has never been read, because the backend logged a stack trace's tail and the corrected extractor reports its head, which on this engine is JVM startup noise. Evidence: `benchmarks/measurements/2026-09-19-php-blocker-open.json`.
   - _Boundary: Evaluation harness, full-run integration_
   - _Depends: 13.1_
@@ -536,9 +537,12 @@ Preserve all completed task evidence below.
   - _Boundary: shipped obligation facts, contributor-scan dominance_
   - _Depends: 14.4_
   - _Requirements: 6.2, 8.1_
-- [ ] 14.5 Ask the prototype and deserialization families somewhere they can answer
+- [x] 14.5 Ask the prototype and deserialization families somewhere they can answer
   - **Named by the 14.1 census.** Prototype pollution has been asked on one repository and its only report was a false positive; deserialization was never asked at all, because VAmPI holds no such sink and NodeGoat is not a language where the family is declared. Neither has established anything anywhere.
   - Find or record a subject where the family has a sink, and report what it does. A negative on a subject that contains the mechanism is a result; silence on a subject that does not contain it is not.
+  - _Verified 2026-09-20:_ 1,417 tests passed, eighteen skipped; Ruff/format/mypy passed. **Neither family has a subject that can answer, and each for its own reason, which is the result rather than a failure to produce one.**
+  - _Deserialization: the mechanism exists on exactly one subject and that subject cannot be queried._ Across four WordPress plugins and 1,164 PHP files there is one deserialization site, `maybe_unserialize` at `includes/fields.php:1111` in Paid Memberships Pro. The vocabulary held only the bare `unserialize`, which never reached it and correctly so: the matcher is word-bounded and the character before the token inside `maybe_unserialize` is a word character. That spelling is now declared, so the family has a sink to ask about — and the census asked 487 deserialization questions on that plugin and completed none, because no query over that repository completes at all. That is task 13.2's runtime failure, now quantified there, not a statement about this family.
+  - _Prototype pollution: no available subject contains the mechanism._ The only first-party matches on the Node subject are `app.set("view engine", "html")` and its sibling, Express settings whose arguments are literals, and the other JavaScript available is minified browser bundles with no request source. So the family's ability to answer is established on a control instead: a merge of the request body into an object is reported, and the Express setting beside it is not. Declared and never demonstrated has become demonstrated on a control and never on a real subject, which is a different and honest claim.
   - _Boundary: Evaluation harness_
   - _Depends: 14.1_
   - _Requirements: 6.2, 8.1_

benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json

@@ -0,0 +1,83 @@
+{
+  "schema_version": 1,
+  "experiment": "php-runtime-after-the-graph-fix-v1",
+  "date": "2026-09-20",
+  "task": "pre-push-safety-net 13.2",
+  "claim_scope": "What remains of the PHP blocker once the malformed-CPG bug is excluded. Measured on Paid Memberships Pro, 673 files.",
+  "the_graph_bug_is_fixed": {
+    "issue": "joernio/joern#6281, this project's own report",
+    "defect": "a PHP `global` inside a closure gave a node two AST parents, so the dataflow overlay threw and one file cost the whole graph",
+    "filed_against": "4.0.623",
+    "fixed_in": "4.0.625, by joernio/joern#6270",
+    "the_image_pins": "4.0.625, since before any measurement recorded under this task",
+    "verified_today": "Not on a transcription of the repro, which does not reproduce, but on the real trigger: PMPro's includes/fields.php, the file the original bisection found, whose add_filter callback carries a `global $submenu_file`. It builds and applies the default overlays cleanly on BOTH 4.0.625 and 4.0.631: files=2, methods=158, no ReachingDefPass assertion.",
+    "consequence": "What remains under this task is a runtime failure and must not be re-diagnosed as the graph bug."
+  },
+  "what_remains_is_runtime": {
+    "subject": {
+      "files": 673,
+      "regions": 4461,
+      "php_regions": 4425,
+      "questions_asked": 2526
+    },
+    "runs": {
+      "pmpro": {
+        "questions_total": 2526,
+        "questions_completed": 0,
+        "seconds": 627.7,
+        "build_seconds": 144.04,
+        "query_seconds": 363.41,
+        "degradations": [
+          "cross_partition_semantics_unresolved",
+          "partition_file_census_incomplete",
+          "process_timeout",
+          "query_failed",
+          "scratch_cleanup_incomplete",
+          "shard_query_incomplete"
+        ]
+      },
+      "pmpro-joern631": {
+        "questions_total": 2526,
+        "questions_completed": 0,
+        "seconds": 619.1,
+        "build_seconds": 134.54,
+        "query_seconds": 367.32,
+        "degradations": [
+          "cross_partition_semantics_unresolved",
+          "partition_file_census_incomplete",
+          "process_timeout",
+          "query_failed",
+          "scratch_cleanup_incomplete",
+          "shard_query_incomplete"
+        ]
+      },
+      "pmpro-631-longceiling": {
+        "questions_total": 2526,
+        "questions_completed": 0,
+        "seconds": 2096.5,
+        "build_seconds": 113.24,
+        "query_seconds": 1874.15,
+        "degradations": [
+          "cross_partition_semantics_unresolved",
+          "partition_file_census_incomplete",
+          "process_timeout",
+          "query_failed",
+          "scratch_cleanup_incomplete",
+          "shard_query_incomplete"
+        ]
+      }
+    },
+    "reading": "A taint batch of 2,526 questions over a 673-file PHP graph does not complete in thirty minutes. Raising the ceiling six-fold, from 300 to 1,800 seconds, changed nothing except how long the failure took: 363 seconds of query time became 1,874, and zero questions completed in both. The eight-way split retry did not rescue it either, finishing its parts in about three minutes against a deadline with 3,300 seconds still on it, which is worth a look on its own.",
+    "engine_comparison": "4.0.625 and 4.0.631 are indistinguishable on this subject: 627.7s against 619.1s, 363.4s against 367.3s of query time, zero completed in both. The six releases between them change nothing here."
+  },
+  "on_the_engine_pin": {
+    "decision": "leave the image pinned at 4.0.625",
+    "why": "4.0.631 was built and measured and is equivalent on this subject, so there is no demonstrated benefit to weigh against re-verifying every JavaScript and Python measurement on a new engine.",
+    "what_a_bump_would_cost": "Three values in ops/frontend-retention/install.py, not a port. The upstream AstGenRunner.scala is byte-identical in 4.0.631, sha256 9963f949360d7e17baed1bd8875b717b2855745cf7260eef8a87d17533a54268, and its patch anchor still appears exactly once. Only the jar path, the jar checksum (f770d929d192a7da40932ee8c22df76ca635af216f643acf2f46a0fbe5747ef1 for io.joern.jssrc2cpg-4.0.631.jar) and the source URL tag need changing."
+  },
+  "limits": [
+    "One PHP subject. The other three plugins available hold no deserialization site and were not run.",
+    "The split retry's fast failure is unexplained and may be hiding a second cause.",
+    "Both engines were measured with the same region budget, 500, and the same driver."
+  ]
+}