mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
fix(census): stop hiding findings behind a question-outcome count, and correct the PHP record
PHP is not silent. Paid Memberships Pro establishes findings, including a true positive, and this project's own instrument is why it looked otherwise. The census harness ordered its verdict on completion before findings, so a family with eighteen findings and no completed question reported "asked but no question completed". Findings are now checked first: a question-outcome count is bookkeeping and cannot hide the thing being censused. NodeGoat and VAmPI were never affected, because they complete every question and their verdicts reached the findings check. The scan summary now also carries the total findings and the outcome status counts, so the two numbers can be seen disagreeing instead of one silently winning. The second mistake was mine rather than the harness's: only the 500-region runs had been looked at, which are the ones that time out. What PHP actually detects on PMPro. At 25 regions, 19 findings in seven minutes with no timeout; at 100 regions, 43 findings in twelve minutes with no timeout. Among them adminpages/ discountcodes.php:749, where sanitize_text_field($_REQUEST["s"]) reaches $wpdb->get_results($sqlQuery, OBJECT) through `$sqlQuery .= "WHERE code LIKE '%$s%' "` with no prepare. That is the class of CVE-2023-23488, which php.toml's own comment names as the reason those WordPress sanitizers are kept out of the sanitizer list. The prediction and the measurement agree. The precision cost beside it is measured too: output encoding reports eighteen findings at 25 regions and the three sampled are all properly escaped, two with esc_attr and one coerced with intval. Those escapers are absent from a sanitizer list that is flat across families, because none of them stops SQL injection. Keeping the injection finding costs this family its precision. The scale limit is roughly one to two seconds per question. 25 and 100 regions complete cleanly; 500 regions asks 2,526 questions, needs more than the 1,800-second ceiling and is killed. Four things are measured NOT to be the problem: the malformed-CPG bug (fixed in 4.0.625, verified on the real trigger file), the dataflow overlay (42 s on the full 637-file graph), the engine version (4.0.625 and 4.0.631 indistinguishable), and the bundled Stripe, Braintree and Twocheckout SDKs under includes/lib that the vendored list does not cover -- that was my hypothesis for the cost and the overlay measures 42 s with them and 37 s without, so it is refuted. What remains is bookkeeping, not detection: questions_completed is zero in every PHP run including the ones that find things, with cross_partition_semantics_unresolved and partition_file_census_incomplete present throughout. The admission policy reads those outcomes. 1,417 tests pass, eighteen skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5
parent
99abe12062
commit
1f51c82b51
@@ -459,6 +459,12 @@ Preserve all completed task evidence below.
|
||||
- _Retried 2026-09-18 with batch splitting, still not established._ A batch that fails whole is now retried in eight parts, which a probe made look promising: the evidence pass answers all 4,425 PMPro regions in one batch in 29.7 s with a 681 KB response, and cost is fixed rather than per request, so neither count nor size is near a limit. That probe sent `evidenceOnly=true` and therefore measured the structural pass, not the failing one; recorded because those numbers would otherwise read as evidence that PMPro's taint work is cheap. With a 4500-second deadline and a 900-second query ceiling the run was stopped after 26 minutes with no case completed. Splitting also makes an unrecoverable batch slower to abandon, since each part burns its own share of the ceiling.
|
||||
- _Correction 2026-09-20: the malformed-CPG bug is fixed and was never what these runs were hitting._ joernio/joern#6281, this project's own report, is a PHP `global` inside a closure producing a node with two AST parents, which made the dataflow overlay throw and cost the WHOLE graph. It was filed against 4.0.623 and fixed by #6270 in **v4.0.625**, which is the version the image has pinned since before any measurement here. Verified today on the real trigger: PMPro's `includes/fields.php`, the file the original bisection found, builds and applies the default overlays cleanly on 4.0.625 and on 4.0.631. So what remains under this task is a RUNTIME failure and must not be re-diagnosed as the graph bug.
|
||||
- _Quantified 2026-09-20, and it is a runtime failure._ A taint batch of 2,526 questions over the 673-file checkout does not complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed nothing but how long the failure took: query time went from 363 s to 1,874 s and zero questions completed in both. The eight-way split retry did not rescue it, finishing its parts in about three minutes against a deadline with 3,300 seconds still on it, which is worth its own look. Joern 4.0.625 and 4.0.631 are indistinguishable here, 627.7 s against 619.1 s. The engine pin stays at 4.0.625: the newer release was built and measured and is equivalent on this subject, so there is no benefit to weigh against re-verifying every JavaScript and Python measurement. Evidence: `benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json`.
|
||||
- _Corrected 2026-09-20: PHP is not silent, and this project's own instrument is why it looked silent._ The census harness ordered its verdict completion-first, so a family with eighteen findings and no completed question reported "asked but no question completed" — and only the 500-region runs, the ones that time out, had been looked at. Fixed: findings are checked first, because a question-outcome count is bookkeeping and cannot hide the thing being censused. NodeGoat and VAmPI were never affected, since they complete every question.
|
||||
- _What PHP actually detects on PMPro._ At 25 regions, 19 findings in seven minutes with no timeout; at 100 regions, 43 findings in twelve minutes with no timeout. Among them a true positive at `adminpages/discountcodes.php:749`: `sanitize_text_field($_REQUEST["s"])` reaching `$wpdb->get_results($sqlQuery, OBJECT)` through `$sqlQuery .= "WHERE code LIKE '%$s%' "` with no `prepare`. That is the class of CVE-2023-23488, which `php.toml`'s own comment names as the reason those WordPress sanitizers are excluded — the prediction and the measurement agree.
|
||||
- _And a measured precision cost beside it._ Output encoding reports eighteen findings at 25 regions and the three sampled are all properly escaped, two with `esc_attr` and one coerced with `intval`. Those escapers are deliberately absent from a sanitizer list that is flat across families, because none of them stops SQL injection; keeping the injection finding above costs this family its precision, and that is the documented tension's price.
|
||||
- _The scale limit, quantified._ Roughly one to two seconds per question. 25 and 100 regions complete cleanly; 500 regions asks 2,526 questions, needs more than the 1,800-second ceiling and is killed. Nothing pathological, and the retry's fast failure is still unexplained.
|
||||
- _What is NOT the problem, each measured rather than assumed:_ the malformed-CPG bug (fixed in 4.0.625, verified on the real trigger file); the dataflow overlay (42 s on the full 637-file graph); the engine version (4.0.625 and 4.0.631 indistinguishable); and the bundled Stripe, Braintree and Twocheckout SDKs under `includes/lib`, which the vendored list does not cover — that was the hypothesis and the overlay costs 42 s with them and 37 s without, so it is refuted.
|
||||
- **The remaining real gap is bookkeeping, not detection:** `questions_completed` is zero in every PHP run including the ones that find things, with `cross_partition_semantics_unresolved` and `partition_file_census_incomplete` present throughout. Findings and question outcomes disagree in public, and the admission policy reads the outcomes. Group 12 raised completion for JavaScript and Python and PHP was never measured. Evidence: `benchmarks/measurements/2026-09-20-php-detection-corrected.json`.
|
||||
- **Blocker still unidentified; investigation paused 2026-09-19 to return to vulnerability work.** Six hypotheses were measured and eliminated: batch size (17,700 requests answer in 44 s with a 2.6 MB response), dataflow cost (50 execution requests in 24 s), call depth 3 with parameter sources (24.7 s for 20), argument length (a 16 KB hook table against a 128 KB cap), hook callback resolution (1 s difference with 233 hooks), and build or sharding (79 s, no shards). Preparation measures 273.5 s. Every reproduction of the failing 22,125-request batch outside the push path answers in under a minute, and the difference from the production invocation is not known. The exception has never been read, because the backend logged a stack trace's tail and the corrected extractor reports its head, which on this engine is JVM startup noise. Evidence: `benchmarks/measurements/2026-09-19-php-blocker-open.json`.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 13.1_
|
||||
|
||||
Reference in New Issue
Block a user