feat(model-grounded-detection): tasks 2.1-2.2, the Joern seam and the taint verdict

The whole engine integration is one subprocess boundary: joern-parse builds a
cpg.bin, joern --script runs a shipped CPGQL query that prints its JSON between
fence markers, because Joern's stdout carries a banner and a REPL prompt around
whatever a script emits. Nothing is imported in-process, so the core install
keeps dependencies = [] -- a test walks the seam's AST to assert it imports
only the standard library and names no JVM binding.

Fail closed everywhere. A build or query that errors, times out, or prints
something unparseable yields no verdict and the caller reports suspicion. The
taint verdict distinguishes this from absence deliberately: a query returning
None means "the engine could not decide", which is not "there is no flow", and
conflating them is how a missing engine silently becomes a clean bill of health.

The verdict is where the ladder earns its keep. An unsanitized source->sink
path is model_entailed -- the graph establishes it and the LLM's claim is no
longer load-bearing. A path a sanitizer may break is model_corroborated: the
flow is real, but whether the sanitizer suffices is the semantic residual the
LLM answers, checked against this verdict rather than averaged over runs. The
chosen flow is picked by a total order (shortest, then lexicographic), so two
runs over one CPG cannot disagree -- Req 6.4 asserted directly.

The query itself is the part our flat IR could not do: reachableByFlows walks
the PDG, so `q = "..." + request.args["x"]; execute(q)` is found, which is
exactly the one-hop limit that held measured entailment at 2.2%.

Capability detection reuses semantic/engines.joern_available rather than
growing a second notion of "installed". The audit manifest is regenerated; the
new modules are named by Req 4/5/6 rather than counted as orphans.

Task 2.3, the go/no-go measurement, needs Joern installed and has not run.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

src/openultrasast/cpg/__init__.py

@@ -0,0 +1,3 @@
+"""The Code Property Graph seam: the adopted engine the model layer arbitrates over."""
+
+from __future__ import annotations

src/openultrasast/cpg/backend.py

@@ -0,0 +1,168 @@
+"""The Joern subprocess seam — the single place this codebase touches a CPG engine (Req 4.1, 4.4, 4.5).
+
+Joern is JVM/Scala. Everything crosses one subprocess boundary: ``joern-parse`` builds a ``cpg.bin`` from a
+source tree, then ``joern --script`` runs a CPGQL script that prints a JSON payload. Nothing is imported
+in-process, so the core install keeps ``dependencies = []`` and an absent engine costs a recorded reason
+rather than a traceback.
+
+Two rules hold everywhere below:
+
+* **Fail closed.** A build or query that errors, times out, or prints something unparseable yields *no*
+  verdict. The model layer then reports ``suspicion``. A wrong verdict is worse than no verdict — that is the
+  whole reason this arbiter exists.
+* **Fence the payload.** Joern prints a banner, compilation notices and a REPL prompt on stdout. The script
+  brackets its JSON between markers so parsing can never swallow the noise around it.
+
+We do not build a control-flow or dataflow engine here (Req 4.4): the CFG, PDG and call graph our flat IR
+lacks are exactly what is being adopted.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+import shutil
+import subprocess
+import tempfile
+from collections.abc import Callable, Mapping
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Protocol
+
+logger = logging.getLogger(__name__)
+
+QUERIES_DIR = Path(__file__).resolve().parent / "queries"
+
+# The payload fence. Joern's stdout carries a banner and a prompt; the script prints these around its JSON.
+BEGIN = "---OUSAST-CPG-BEGIN---"
+END = "---OUSAST-CPG-END---"
+
+BUILD_TIMEOUT_SECONDS = 900  # a large tree takes minutes; the go/no-go records what it actually cost
+QUERY_TIMEOUT_SECONDS = 300
+
+Runner = Callable[..., subprocess.CompletedProcess[str]]
+
+
+def extract_payload(stdout: str) -> object | None:
+    """The JSON between the fence markers, or ``None`` when it is absent or malformed."""
+    start = stdout.find(BEGIN)
+    end = stdout.find(END, start + 1) if start >= 0 else -1
+    if start < 0 or end < 0:
+        return None
+    body = stdout[start + len(BEGIN) : end].strip()
+    try:
+        payload: object = json.loads(body)
+    except json.JSONDecodeError:
+        logger.debug("cpg query payload was not valid JSON")
+        return None
+    return payload
+
+
+@dataclass(frozen=True)
+class CpgResult:
+    """A built CPG and the way to query it. Opaque: only ``model/`` reads through ``run``."""
+
+    cpg_path: Path
+    run: Callable[[str, Mapping[str, object]], object | None]
+
+
+class CpgBackend(Protocol):
+    def available(self) -> bool: ...
+
+    def build(self, root: Path) -> CpgResult | None: ...
+
+
+@dataclass
+class NullBackend:
+    """No engine. Every verdict degrades to ``suspicion`` and the caller records ``cpg_unavailable``."""
+
+    reason: str = "cpg_unavailable"
+
+    def available(self) -> bool:
+        return False
+
+    def build(self, root: Path) -> CpgResult | None:
+        del root
+        return None
+
+
+@dataclass
+class JoernBackend:
+    """``joern-parse`` then ``joern --script``, both out of process and both timeout-bounded."""
+
+    runner: Runner | None = None
+    build_timeout: int = BUILD_TIMEOUT_SECONDS
+    query_timeout: int = QUERY_TIMEOUT_SECONDS
+    queries_dir: Path = field(default_factory=lambda: QUERIES_DIR)
+
+    def available(self) -> bool:
+        from .capability import has_cpg
+
+        return has_cpg()
+
+    def build(self, root: Path) -> CpgResult | None:
+        """Build a CPG for ``root``. ``None`` on any failure — the caller degrades, never guesses."""
+        parse = shutil.which("joern-parse")
+        if parse is None and os.environ.get("OPENULTRASAST_JOERN_PROBE", "").strip().lower() not in {"1", "on", "true", "yes"}:
+            return None
+        scratch = Path(tempfile.mkdtemp(prefix="ousast-cpg-"))
+        cpg_path = scratch / "cpg.bin"
+        command = [parse or "joern-parse", str(root), "--output", str(cpg_path)]
+        completed = self._run(command, timeout=self.build_timeout)
+        if completed is None or completed.returncode != 0 or not cpg_path.is_file():
+            detail = (completed.stderr or completed.stdout or "")[-400:] if completed is not None else "timeout"
+            logger.warning("cpg build failed for %s: %s", root, detail)
+            return None
+        return CpgResult(cpg_path=cpg_path, run=lambda query, params: self.query(cpg_path, query, params))
+
+    def query(self, cpg_path: Path, query: str, params: Mapping[str, object]) -> object | None:
+        """Run a shipped CPGQL script against a built CPG and return its fenced JSON payload."""
+        script = self.queries_dir / f"{query}.sc"
+        if not script.is_file():
+            logger.warning("no such cpg query: %s", script)
+            return None
+        command = [shutil.which("joern") or "joern", "--script", str(script), "--param", f"cpgFile={cpg_path}"]
+        for key, value in sorted(params.items()):
+            command += ["--param", f"{key}={_render(value)}"]
+        completed = self._run(command, timeout=self.query_timeout)
+        if completed is None or completed.returncode != 0:
+            detail = (completed.stderr or "")[-400:] if completed is not None else "timeout"
+            logger.warning("cpg query %s failed: %s", query, detail)
+            return None
+        return extract_payload(completed.stdout or "")
+
+    def _run(self, command: list[str], *, timeout: int) -> subprocess.CompletedProcess[str] | None:
+        run = self.runner if self.runner is not None else subprocess.run
+        try:
+            return run(command, capture_output=True, text=True, timeout=timeout, check=False)
+        except (OSError, subprocess.SubprocessError):
+            return None
+
+
+def _render(value: object) -> str:
+    """CPGQL parameters are strings; a sequence becomes a comma-separated list the script splits."""
+    if isinstance(value, str):
+        return value
+    if isinstance(value, (list, tuple, set, frozenset)):
+        return ",".join(sorted(str(item) for item in value))
+    return str(value)
+
+
+def resolve_cpg_backend(*, runner: Runner | None = None) -> CpgBackend:
+    """The backend for this machine: Joern when it is installed, else the null one."""
+    from .capability import has_cpg
+
+    return JoernBackend(runner=runner) if has_cpg() else NullBackend()
+
+
+__all__ = [
+    "BEGIN",
+    "END",
+    "CpgBackend",
+    "CpgResult",
+    "JoernBackend",
+    "NullBackend",
+    "extract_payload",
+    "resolve_cpg_backend",
+]

src/openultrasast/cpg/capability.py

@@ -0,0 +1,18 @@
+"""Is a CPG engine available? (Req 4.2, 4.3)
+
+One probe, not two: ``semantic/engines.joern_available`` already answers this and already honours the
+``OPENULTRASAST_JOERN_PROBE`` override that the test matrix uses to force the absent case. The model layer
+reuses it rather than growing a second, subtly different notion of "installed".
+"""
+
+from __future__ import annotations
+
+from ..semantic.engines import joern_available
+
+
+def has_cpg() -> bool:
+    """True when a CPG can be built. False means every verdict degrades to ``suspicion`` with a reason."""
+    return joern_available()
+
+
+__all__ = ["has_cpg"]

src/openultrasast/cpg/queries/taint.sc

@@ -0,0 +1,61 @@
+// Taint reachability for one flow family (model-grounded-detection Req 6.1).
+//
+// The question this answers is the one our flat IR could not: does an untrusted source reach the labeled sink
+// *through resolved dataflow*, and is there a sanitizer on that path. Joern's `reachableByFlows` walks the
+// PDG, so a value bound three statements upstream — `q = "..." + request.args["x"]; execute(q)` — is found,
+// which is exactly the one-hop limit that held our measured entailment at 2.2%.
+//
+// Parameters (all comma-separated where plural):
+//   cpgFile     the built cpg.bin
+//   sources     source matchers, e.g. request.args,request.form
+//   sinks       sink call names, e.g. execute,os.system
+//   sanitizers  sanitizer call names; a flow through one of these is corroborated, not entailed
+//   function    optional: restrict sinks to this enclosing method
+//
+// Output: a fenced JSON array of {sink, sinkLine, sinkMethod, source, sanitized, length}.
+// The fence exists because Joern prints a banner and a prompt around whatever a script emits.
+
+@main def exec(cpgFile: String, sources: String, sinks: String, sanitizers: String = "", function: String = "") = {
+  importCpg(cpgFile)
+
+  def split(raw: String): List[String] =
+    raw.split(",").map(_.trim).filter(_.nonEmpty).toList
+
+  val sourcePatterns    = split(sources)
+  val sinkNames         = split(sinks)
+  val sanitizerNames    = split(sanitizers)
+
+  // A source is any call or identifier whose code matches one of the source patterns. Matching on `code`
+  // rather than on a method name keeps `request.args["x"]` and `req.body.name` both reachable without
+  // per-framework special cases.
+  def sourceNodes = cpg.call.filter(c => sourcePatterns.exists(p => c.code.contains(p))).l ++
+                    cpg.identifier.filter(i => sourcePatterns.exists(p => i.code.contains(p))).l
+
+  // A sink is a call whose name (or full code) names one of the family's sinks, optionally inside `function`.
+  def sinkNodes = {
+    val all = cpg.call.filter(c => sinkNames.exists(n => c.name == n || c.code.startsWith(n) || c.methodFullName.contains(n))).l
+    if (function.isEmpty) all else all.filter(_.method.name == function)
+  }
+
+  def isSanitized(flowCode: List[String]): Boolean =
+    sanitizerNames.nonEmpty && flowCode.exists(code => sanitizerNames.exists(s => code.contains(s)))
+
+  val rows = sinkNodes.flatMap { sink =>
+    val flows = sourceNodes.reachableByFlows(sink).l
+    flows.map { flow =>
+      val elements = flow.elements.map(_.code).l
+      ujson.Obj(
+        "sink"       -> sink.code.take(200),
+        "sinkLine"   -> sink.lineNumber.getOrElse(-1).toString,
+        "sinkMethod" -> sink.method.name,
+        "source"     -> elements.headOption.getOrElse("").take(200),
+        "sanitized"  -> isSanitized(elements),
+        "length"     -> elements.size
+      )
+    }
+  }
+
+  println("---OUSAST-CPG-BEGIN---")
+  println(ujson.write(ujson.Arr(rows: _*)))
+  println("---OUSAST-CPG-END---")
+}