Commit Graph
7 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Fable 5.1 7411f3568a bench(ranking): the phase-2 baseline on Joern 4.0.625 -- pmpro 82, wpstatistics 173, mwwpform 68, vampi 16
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-11 14:39:22 +02:00
Marius CiepluchandClaude Fable 5.1 9142cb1fb4 docs(flow-aware-ranking): v6 measured -- identical to v5, and why; the layout fact table proposed
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-11 14:15:12 +02:00
Marius CiepluchandClaude Fable 5.1 2de873274f feat(model): shipped joins the evidence vector as a within-tier weight; v5 baseline
v5 (parameter-fed carried confirmed by the exact flow): pmpro 69, wpstatistics 172, mwwpform 68,
vampi 16. Reading the recorded head of each order: every region ahead of a CVE is in its tier on a tied
or higher score, and 10 of the 67 ahead of MW WP Form's are test-mail.php. The build's own word on what
is product was on the region and not in the vector; it is now, at -2.0 for an unshipped file. Orders
only, never promotes.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-11 13:45:00 +02:00
Marius CiepluchandClaude Fable 5.1 983eda5886 feat(model): v4 measured; a parameter-fed field becomes strong carried only when the exact flow confirms it
v4 (provenance split): pmpro 69 and wpstatistics 172 unchanged -- their tier-4 members are source-fed
through same-file helpers that read the request; mwwpform 68 -> 248, because its CVE's region is fed by
a constructor parameter, the shape the split demotes and the exact flow had put at 30. v5 asks the flow
for exactly the parameter-fed fields a sink-bearing region reads, memoised per (family, file, field),
and reports carriedBy for diagnosis.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-11 13:27:49 +02:00
Marius CiepluchandClaude Fable 5.1 d1f37abadd feat(model): carried with provenance -- source-fed promotes a tier, parameter-fed only orders
Structural carried (v3) measured: pmpro 391 -> 69, wpstatistics 504 -> 172, mwwpform 513 -> 68 -- and
tier 4 swollen to 555 and 569 regions, because "assigned from a parameter" was folded in with "assigned
from a source" and every constructor does the former. The query now reports carriedKind; the vector
promotes on "source" and gives "parameter" a within-tier weight of 0.75.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-11 13:11:00 +02:00
Marius CiepluchandClaude Fable 5.1 1741b6bed6 feat(model): phase 2 measured (weights v1) and carried reported by the evidence branch
budget_at_recall by a real evidence pass over every region: pmpro 391 -> 116, wpstatistics 504 -> 223,
mwwpform 513 -> 292, vampi 16 -> 16. The gate (< 50) is not met. pmpro's CVE is inside its 121 tier-4
regions; WP Statistics's and MW WP Form's are one tier low because `carried` -- stage one of the
two-stage join -- was not reported as evidence. taint.sc now reports it where a sink exists, and the
vector reads the query's answer when given.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-11 11:47:06 +02:00
Marius CiepluchandClaude Fable 5.1 88ada8e48c feat(ranking): phase 0 -- the baseline, the evidence instrument, and a 7x taint speedup found on the way
flow-aware-ranking phase 0: measure before changing anything.

THE BASELINE, committed as benchmarks/ranking/baselines/2026-09-10-access-rank.json
from benchmarks/ranking/position.py, which sorts regions exactly as the scan does:

    pmpro         4,463 regions  83.8% at 0.30   CVE at 390   budget_at_recall 391
    mwwpform        983 regions  81.7% at 0.30   CVE at 512   budget_at_recall 513
    wpstatistics  1,673 regions  75.3% at 0.30   CVE at 503   budget_at_recall 504
    vampi            25 regions                  BOLA at 15, 10; SQLI reached transitively

That corrects the brief's own cross-assessment: "concentration, not inclusion"
was measured on pmpro alone. On the other two plugins the CVE sits OUTSIDE a
500-region budget and is never examined; the slices found those CVEs only
because a slice has too few regions to exclude anything.

THE INSTRUMENT: `evidenceOnly` mode on the taint query -- what is in scope,
which sink calls, what shape, whether a sanitizer wraps THEIR argument, whether
a source is local or reachable -- returning before reachableByFlows is ever
called. model/evidence.py turns its rows into a vector per (region, family)
and a tier; scan.py records every pair's tier on the result and acts on none of
it. Every later phase is measured against a record made by scans that did not
know they were being measured.

    2,500 pairs in 98.9s incl. JVM load, ~40 ms per pair
    tier 0 -- no sink of that family in reach: 2,161 of 2,500 = 86%

Two earlier attempts ran past an hour each with no payload, and both were my
fault: repository-wide walks repeated per request -- the sink scan, the
reachable-method BFS, familyInRepo. Memoised across the batch.

AND THE SAME FIX REACHES THE DATAFLOW MODE. sinkCalls and reachableMethods do
not depend on the family, and were recomputed for all 2,500 requests. The
identical 250-request set with both joins on:

    2,026.3s -> 287.0s      8,105 -> 1,148 ms per request      same 5,779 rows

Most of what 5.13 measured as "the base taint query" was never dataflow. A 7x
speedup with nothing about the analysis changed; both corrections recorded in
the specs. Combined with 86% tier 0, the whole-repository taint query that
timed out at 40 minutes is ~339 real requests at ~1.1s: about six minutes.

Also: the module audit manifest is regenerated (91 load-bearing, 7 standalone,
0 orphaned) because evidence.py is now imported by the scan, and the
failed-query test is order-independent since the evidence pass records its own
failure before taint's.

869 tests pass; query compile test 4 passed; gates byte-identical.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-10 18:22:18 +02:00