mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
docs(overview): record the CVE, and what scoping is actually for
The scoreboard now carries a real finding: CVE-2023-23488 at class.memberorder.php:936, the exact line, in unmodified plugin code, with 16 of 26 unprepared $wpdb queries reported in that file and zero of the 26 using ->prepare. And the reading the maintainer's instinct produced: **the first real CVE was found by cutting, not by scanning more.** Two of 637 files load in 9 seconds and report the bug; the full checkout does not load at all. That two-file slice IS the contributor case -- a developer editing class.memberorder.php gets precisely that scan -- so scoping is not a workaround for a scale problem, it is the thesis, and the scale problem is what happens when the thesis is ignored. The distinction recorded is not whether to cut but who decides and whether the decision survives: a build declaration, a diff or a subtree is chosen by the project or the developer and is recoverable from the artifact, while a model judging "this looks like boilerplate" gives the same answer when right and tells you nothing when it drops the file holding the bug. Every failure found on this checkout was of that second shape. Also corrects "one language is demonstrated" to two, with the honest limit attached: PHP works on a real plugin slice and not at repository scale. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
96e7476c6d
commit
f13934c663
@@ -56,14 +56,18 @@ invocation per query kind.
|
||||
| libpng envelope (89k lines of C) | build 16s, query 250s, scan 446s, **911 MB** | `2026-09-09-repo-libpng-envelope.json` |
|
||||
| libpng valid findings | **zero** | same |
|
||||
| report proportionality | libpng 198 sections → **39** | `2026-09-09-report-proportionality.json` |
|
||||
| **CVE-2023-23488, production plugin code** | **found at `class.memberorder.php:936`**, the exact line | `2026-09-09-pmpro-cve-found.json` |
|
||||
| — precision on that file | 16 of 26 unprepared `$wpdb` queries; **0 of 26 use `->prepare`** | same |
|
||||
|
||||
## What is not true yet
|
||||
|
||||
- **The C story does not work.** `c/memory` has produced zero true positives and three distinct classes of
|
||||
false one. It models string functions; libpng overflows through arithmetic, and its library code never
|
||||
calls a sink we model. This is structural, not tuning.
|
||||
- **One language is demonstrated.** Python. JavaScript has fact tables and pair coverage but no repository.
|
||||
PHP, Ruby and Go have **zero families** — Joern would parse them and the engine would say nothing.
|
||||
- **Two languages are demonstrated.** Python end to end, and PHP on a real plugin slice where it found a
|
||||
real CVE. Ruby and Go still have **zero families** — Joern parses them and the engine says nothing.
|
||||
- **Not at repository scale for PHP.** The 637-file checkout does not load; the 2-file slice does. That
|
||||
boundary is unmeasured (task 5.10).
|
||||
- **The suspicion band is still too loud.** vampi emits 61 report sections for a 520-line application, 38 of
|
||||
them a candidate the graph could not decide with a judge's opinion attached.
|
||||
- **The evidence base is two repositories.** One has never produced a true positive.
|
||||
@@ -71,6 +75,26 @@ invocation per query kind.
|
||||
controlling control structure, so every guard question runs on CFG dominance instead.
|
||||
- **`execution_confirmed` has no production caller.** The rung exists and nothing reaches it.
|
||||
|
||||
## What scoping is for, which is not what it looks like
|
||||
|
||||
The first real CVE was found by **cutting**, not by scanning more. Two of Paid Memberships Pro's 637 files —
|
||||
the ones holding both ends of the bug — load in 9 seconds, query in 111, and report CVE-2023-23488 at the
|
||||
exact line. The full checkout does not load at all.
|
||||
|
||||
That two-file slice **is the contributor case**. A developer editing `class.memberorder.php` gets precisely
|
||||
that scan. So scoping is not a workaround for a scale problem; it is the thesis, and the scale problem is
|
||||
what happens when the thesis is ignored.
|
||||
|
||||
The distinction that matters is not whether to cut but **who decides and whether the decision is recorded**:
|
||||
|
||||
| | decided by | recoverable afterwards |
|
||||
| --- | --- | --- |
|
||||
| **scoping** — a build declaration, a diff, a subtree | the project or the developer | yes, and it is in the artifact |
|
||||
| **triage** — "this looks like boilerplate" | a model | no; if it drops the file with the bug, the scan reports clean |
|
||||
|
||||
Both give the same answer when the model is right. Only one of them tells you when it was wrong, and every
|
||||
failure found on this checkout was of exactly that shape.
|
||||
|
||||
## Why a corpus of benchmarks is not enough on its own
|
||||
|
||||
This is the finding worth teaching, and it is not an opinion — it is what the gates say.
|
||||
|
||||
Reference in New Issue
Block a user