docs(flow-aware-ranking): brief -- rank pairs by evidence, in phases that each must prove themselves

A taint request costs 6.63s, a 637-file plugin yields 4,463 regions, and 83.8%
of them share one rank ordered alphabetically. The ranker is not weak, it is
looking at the wrong thing.

The evidence that this is tractable, from one plugin:

    getMemberOrderByCode  (holds CVE-2023-23488)  rank 0.30
        $wpdb->get_var("... WHERE code = '" . $code . "' LIMIT 1")
    getMemberOrderByID    (sibling)               rank 0.30
        $wpdb->get_row("... WHERE id = '$id' LIMIT 1")
    pmpro_rest_api_get_membership_level           rank 0.80
        no SQL sink at all

A region with no sink outranks two holding raw concatenated SQL. The signal is
in the sink call's own text -- interpolated versus bound -- and nothing in the
ranker reads it.

And the detail that decides whether a keyword rule suffices: `esc_sql` IS in
scope for the CVE's function, applied to a different statement. "A sanitizer is
present, rank it lower" demotes exactly the right answer. Knowing which call it
wraps is reading, not matching -- which is the case for a model rather than a
table.

Unit: (region, family) PAIRS, tiered by cheap evidence. Tier 0 (no sink of that
family in scope) is the only tier that excludes rather than orders, so it alone
must be exact; above it, being wrong costs position rather than the finding.
That bound is what makes a model safe here -- a ranker cannot manufacture a
finding.

Six phases, each shipping only if it beats the previous one on a repository it
was not developed against: baseline, exact pruning (the phase that makes a scan
finish), heuristic tiers, the LLM ranker, breaking the closed loop, then prompt
optimisation. Phase 2 carries an explicit possibility that the model is not
needed for tiering -- and knowing that is worth more than assuming it. Phase 5
comes last because an optimiser run against a censored metric optimises the
censorship.

Records what the prompt must carry, and that the question is never "is this
vulnerable" -- that is the arbiter's job and the one thing a ranker must not be
asked.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-10 16:06:55 +02:00
co-authored by Claude Opus 5
parent 62052b9525
commit 7e397beca6
3 changed files with 136 additions and 1 deletions
+25 -1
View File
@@ -897,7 +897,31 @@
method-recogniser fixes moved it. The reason it is not reported is that the taint query TIMED OUT
(`query_failed`, all 2,500 requests), so no region got a verdict, well ranked or not. Ranking is a real
problem; it is not this miss.
- The fix is three layers, and the LLM is only the third:
- **Reframed 2026-09-10, and this is the shape the whole thing should take.** Pruning impossible questions
(5.14) is not a separate optimisation, it is TIER 0 of the ranker. The two are one mechanism at different
resolutions, and once that is seen the unit of ranking changes:
> **Rank (region, family) PAIRS, not regions — and rank them by evidence that a taint flow could exist.**
Today the ranker orders regions by declared access and then asks all five families of each, so a region
is examined for `deserialization` in a repository that never calls `unserialize`. The expensive question
is "does a source reach a sink of family F in this scope", and it costs 6.63 seconds. Every cheap
approximation of that question is available for free, from the fact tables and a text scan:
| tier | evidence | what it means |
|---|---|---|
| **0** | no sink of F in scope | **never ask** -- provably empty, cannot lose a finding |
| 1 | sink present, no source in scope or reachable | ask last |
| 2 | sink and source present, a sanitizer between them | may still corroborate |
| 3 | sink and source, no sanitizer, entry point or declared-public | ask first |
Tier 0 must be EXACT -- it is the only tier that excludes rather than orders, so it may only hold pairs
that provably cannot yield a flow. Everything above it is ordering, where being wrong costs position
rather than the finding.
- That makes the ranker a **cheap conservative approximation of the query it schedules**, which is the
right relationship: the ranker's job is to spend an expensive budget where the cheap evidence is
strongest, and its errors are bounded by construction because it cannot exclude anything above tier 0.
- The three layers below then describe HOW a tier is decided, not what the unit is:
- **Facts, not code.** Framework vocabulary moves into the semantic tables the way `[[dispatch]]` already
did today. An `[[access]]` fact declares hook-name patterns and their access level, so
`wp_ajax_nopriv_*` is a row rather than an `if`. Covers every framework somebody has written facts for,