docs(contributor-scan): 5.12 -- the ranker knows one framework, everything else is "unknown"

The rank SCALE is generic: `_ACCESS_RANK` maps abstract access levels to
numbers and knows no framework. The EVIDENCE feeding it does not --
`_wordpress_hook_access` holds the literals `wp_ajax_nopriv_` and `wp_ajax_`,
`_rest_route_access` holds `__return_true`. One framework, written in Python.

The consequence is not "slightly worse elsewhere", it is no ordering at all.
Anything unrecognised falls to review-required = 0.30 and ties break
alphabetically by path. That is where CVE-2023-23488's region sits -- position
~1036 of 4,463, below every REST handler -- so a 500-region budget never
reaches it. On a Laravel or Django codebase EVERY region lands in that tier:
thousands of regions ordered by filename.

Three layers, and the model is only the third:

  * facts, not code -- an [[access]] fact declares hook patterns and their
    access level, the way [[dispatch]] already replaced hardcoded
    add_action/apply_filters today;
  * generic features needing no framework knowledge -- modelled sinks in the
    file, a modelled source present, call-graph distance to the nearest sink,
    shipped, an unsanitized path to any sink. This is what would move
    CVE-2023-23488's region on its own, and it should exist BEFORE any model,
    or the model's contribution cannot be measured against anything;
  * the LLM for the residue the first two cannot separate.

Ranking is a safe place for a model because it cannot manufacture a finding:
it decides what is looked at, never what is reported. The arbiters still
decide, the gates stay byte-identical, and a misrank costs recall -- now
visibly, since regions_truncated reports 500 of 4,463.

Records the constraints that must be settled before spending model budget:
batched over a prefilter (4,463 calls a scan is not a CI tool),
leave-one-repository-out (this project already has the train-on-test scar),
a recorded ranking so baselines diff, and honest degradation with no model.

A model proposing FACTS for an unknown framework is a different, slower loop
needing human approval -- ranking at runtime is not policy, a fact table is.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

.kiro/specs/contributor-scan/tasks.md

@@ -805,6 +805,60 @@
   - _Requirements: 4.4, 6.1, 8.1, 9.1_
   - _Depends: 5.8_
 
+- [ ] 5.12 The ranker knows one framework, and everything else lands in "unknown"
+  - > "the ranker hardcodes wp nodes etc. which means it's not generalizeable. I'd like the LLM to be a
+    > flexible layer here to avoid hardcoded / overly specific ranker implementations"
+  - **What is actually hardcoded, precisely.** The rank SCALE is generic -- `_ACCESS_RANK` maps abstract
+    access levels to numbers and knows nothing about any framework. The EVIDENCE feeding it does not:
+    `mapping._wordpress_hook_access` is a Python function containing the literals `wp_ajax_nopriv_` and
+    `wp_ajax_`, and `_rest_route_access` contains `__return_true`. One framework, written in code.
+  - **The consequence is not "slightly worse on other frameworks", it is no ordering at all.** Anything the
+    classifier does not recognise falls to `review-required` = 0.30, and within a tie the order is
+    alphabetical by path -- arbitrary. On Paid Memberships Pro that is where CVE-2023-23488's region sits,
+    below every REST handler at 0.80, at position ~1036 of 4,463. With a 500-region budget the arbiter never
+    sees it. A Laravel, Symfony or Django codebase would have EVERY region in that tier: 4,463 regions
+    ordered by filename.
+  - The fix is three layers, and the LLM is only the third:
+    - **Facts, not code.** Framework vocabulary moves into the semantic tables the way `[[dispatch]]` already
+      did today. An `[[access]]` fact declares hook-name patterns and their access level, so
+      `wp_ajax_nopriv_*` is a row rather than an `if`. Covers every framework somebody has written facts for,
+      costs nothing at runtime, and is auditable.
+    - **Generic features that need no framework knowledge at all.** Every region can be scored on things the
+      CPG and the existing fact tables already know: how many modelled sinks its file contains, whether a
+      modelled source appears in it, its call-graph distance to the nearest sink, whether it is shipped,
+      whether any sink call in it lacks a sanitizer on any path. A method holding an unprepared `$wpdb`
+      query one hop from a parameter should outrank an empty getter WITHOUT anyone having written a
+      WordPress rule. This is the layer that would have moved CVE-2023-23488's region on its own, and it
+      should be built before any model is involved -- otherwise the LLM's contribution cannot be measured
+      against anything.
+    - **The LLM for the residue.** Regions the facts cannot classify and the features cannot separate. It
+      sees the feature vector plus a short excerpt and orders them.
+  - **Why ranking is a safe place for a model, and adjudication is not.** A ranker cannot manufacture a
+    finding. It decides what is LOOKED AT, never what is reported: the arbiters still entail or stay silent,
+    the three gates stay byte-identical, and a misranked region costs recall rather than producing a false
+    claim. The failure is also now visible rather than silent, because `regions_truncated` reports "500 of
+    4,463 examined" -- an unexamined region can no longer be mistaken for a clean one.
+  - Constraints that belong in the design before any model budget is spent:
+    - **Cost.** One call per region is 4,463 calls a scan and nobody runs that in CI. The model ranks a
+      prefiltered candidate set in batches, or it is not shipped.
+    - **Leak.** Four pinned CVEs is a tiny training set and this project has the scar already -- see the
+      closed-loop train-on-test finding, where the improve lever learned from holdout pairs and inflated its
+      own numbers. Optimisation must be leave-one-repository-out and the reported figure must come from a
+      plugin the optimiser never saw.
+    - **Reproducibility.** Two runs of one repository must agree, or a baseline diff means nothing. The
+      ranking is recorded in the manifest so a change in what was examined is auditable rather than
+      invisible.
+    - **Degradation.** With no model configured the scan still runs on layers one and two, and the report
+      says which ranker produced the order.
+  - A model could also PROPOSE facts for an unrecognised framework -- "this codebase registers handlers with
+    `Router::get(...)`" as a candidate `[[dispatch]]` row. That is a different and slower loop, and it must
+    go through human approval, because a fact is policy and the standing rule is that no policy is
+    LLM-authored. Ranking at runtime is not policy; a fact table is.
+  - Observable: the rank position of each pinned CVE's region, before and after, on a repository the
+    optimiser did not see. `getMemberOrderByCode` is at ~1036 of 4,463 today.
+  - _Requirements: 4.4, 9.1_
+  - _Depends: 5.8_
+
 ## Group 6 — Regression baselines
 
 - [ ] 6.1 Repository baseline and delta

src/openultrasast/cpg/backend.py

@@ -40,7 +40,12 @@ BEGIN = "---OUSAST-CPG-BEGIN---"
 END = "---OUSAST-CPG-END---"
 
 BUILD_TIMEOUT_SECONDS = 900  # a large tree takes minutes; the go/no-go records what it actually cost
+# A slice's questions answer in seconds; a repository's do not. Measured on a 637-file WordPress plugin:
+# 13 dominance requests answer in 50s (most of it loading the graph) and 2,500 taint requests exceed 300s.
+# The constant was chosen when a "scan" meant an excerpt, and a repository scan is not a slower slice -- it
+# is a different order of question, so the ceiling is configurable rather than a number to keep raising.
 QUERY_TIMEOUT_SECONDS = 300
+QUERY_TIMEOUT_ENV = "OPENULTRASAST_CPG_QUERY_TIMEOUT"
 # A JVM with no -Xmx takes a quarter of physical RAM for its heap. On a contributor's laptop that is the
 # difference between a scan running in the background and a scan the machine notices, and it made the
 # first repository measurements unreproducible: runs were killed under memory pressure at different
@@ -210,7 +215,7 @@ class JoernBackend:
 
     runner: Runner | None = None
     build_timeout: int = BUILD_TIMEOUT_SECONDS
-    query_timeout: int = QUERY_TIMEOUT_SECONDS
+    query_timeout: int = field(default_factory=lambda: _configured_timeout())
     heap_mb: int = 0  # 0 means read the environment, then fall back to CPG_HEAP_MB
     queries_dir: Path = field(default_factory=lambda: QUERIES_DIR)
     # Why the last build refused, so the driver can name it instead of reporting a bare `cpg_build_failed`.
@@ -712,6 +717,14 @@ def _hoist_shared(rendered: dict[str, dict[str, str]]) -> dict[str, str]:
     return shared
 
 
+def _configured_timeout() -> int:
+    """The query ceiling, from the environment or the default. Never zero or negative."""
+    raw = os.environ.get(QUERY_TIMEOUT_ENV, "").strip()
+    if raw.isdigit() and int(raw) > 0:
+        return int(raw)
+    return QUERY_TIMEOUT_SECONDS
+
+
 def _render(value: object) -> str:
     """CPGQL parameters are strings; a sequence becomes a comma-separated list the script splits."""
     if isinstance(value, str):