feat(model-grounded-detection): group 5, corpus calibration and honest reporting

5.1 inverts what the corpus is for. Previously a pair the detector missed was a
DETECTOR failure, to be averaged away over K runs and chased with a prompt
change. The arbiter is deterministic: it will miss that pair the same way every
time, and the only thing that moves it is better modelling. So a pair the model
cannot split is a NAMED GAP IN THE MODEL, recorded with its reason, and every
gap carries one even when the caller supplies none -- a bare gap teaches
nothing.

`covered` means the model reached a strictly higher rung on the vulnerable side
than on its fix. Flagging both sides equally is not coverage: it distinguishes
nothing, which is exactly what group 2 measured taint reachability doing on
every parameterised SQL pair. The harvest seam takes the fix commit and derives
the vulnerable side as its parent, with the fix diff as the oracle -- nothing
hand-labelled.

5.2 keeps three disciplines from the deleted publish layer, each because its
absence caused a real error here:

  * per-slice rows always -- `render` RAISES without them. A mixed headline
    once hid agent-written code scoring 44% against the development slice's 83%.
  * the overfitting gap beside the headline, and ABSENT rather than zero when a
    slice is missing: reading an unmeasured slice as "no overfitting" is the
    flattering failure, not a neutral one.
  * the rung on every finding, so a suspicion cannot be read as a result.

Run end to end against the committed ceiling artifact, the report says things
worth not hiding: agent-vfc 0/28 covered, vibe-py 40% against vfc-js 24%, an
overfitting gap of +16%, and gaps grouped by family with their reasons --
injection 32 (21 no verdict, 11 distinguished nothing), path 8 (all no
verdict), untrusted_destination 5 (4 entailed on BOTH sides).

713 passing, mypy/ruff clean, gates byte-identical, zero orphaned modules.
A full re-measure with the closure-scoping fix is in flight; these numbers are
the pre-fix baseline and will move.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

src/openultrasast/model/report.py

@@ -0,0 +1,83 @@
+"""Reporting that cannot flatter itself (model-grounded-detection, Req 9.3, 11.1).
+
+Three disciplines survive from the deleted publish/scoring layer, and each is here because its absence caused
+a real error in this project's history rather than because it is tidy:
+
+* **Per-slice rows, always.** A headline over mixed slices once hid that agent-written code scored 44.4%
+  while the hand-written development slice scored 83.3%. `render` refuses to produce anything without rows.
+* **The overfitting gap beside the headline.** The distance between the development slice and the deciding
+  real-world one is stated, never left for the reader to compute. When a slice is missing the gap is
+  *absent*, not zero — reading a missing measurement as "no overfitting" is the flattering failure.
+* **The rung on every finding.** A finding's strength is what established it; omitting the rung invites a
+  reader to treat a suspicion as a result.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Sequence
+from dataclasses import dataclass
+
+from ..findings import StaticFinding
+
+
+@dataclass(frozen=True)
+class SliceRow:
+    """One slice's coverage. `covered` is pairs the model *split*, not pairs it flagged."""
+
+    slice: str
+    pairs: int
+    covered: int
+
+    @property
+    def rate(self) -> float:
+        return self.covered / self.pairs if self.pairs else 0.0
+
+
+def overfitting_gap(rows: Sequence[SliceRow], *, development: str = "", deciding: str = "") -> float | None:
+    """Development-slice coverage minus the deciding slice's, or ``None`` when either is absent.
+
+    ``None`` rather than ``0.0`` on purpose: a slice that was never measured must not read as agreement
+    between them.
+    """
+    by_name = {row.slice: row for row in rows}
+    if development not in by_name or deciding not in by_name:
+        return None
+    return by_name[development].rate - by_name[deciding].rate
+
+
+def render(rows: Sequence[SliceRow], *, development: str = "", deciding: str = "") -> str:
+    """The report. Raises when there are no per-slice rows, because a bare aggregate is the thing to prevent."""
+    if not rows:
+        raise ValueError("a report needs per-slice rows: there is no headline without them")
+
+    lines = ["## Model coverage", "", "| slice | pairs | covered | coverage |", "|---|---|---|---|"]
+    for row in sorted(rows, key=lambda r: r.slice):
+        lines.append(f"| {row.slice} | {row.pairs} | {row.covered} | {row.rate:.0%} |")
+
+    total_pairs = sum(row.pairs for row in rows)
+    total_covered = sum(row.covered for row in rows)
+    lines += ["", f"Across every slice: {total_covered}/{total_pairs} = {total_covered / total_pairs:.0%} "
+                  "— read the rows above it, not this line."]
+
+    gap = overfitting_gap(rows, development=development, deciding=deciding)
+    if gap is not None:
+        lines += [
+            "",
+            f"**Overfitting gap: {gap:+.0%}** — `{development}` (development) minus `{deciding}`, "
+            f"which is the deciding real-world slice.",
+        ]
+    elif development or deciding:
+        lines += ["", f"**Overfitting gap: not measurable** — `{development}` or `{deciding}` is absent from this run."]
+    return "\n".join(lines) + "\n"
+
+
+def render_findings(findings: Sequence[StaticFinding]) -> str:
+    """Findings with the rung that established each one (Req 5.3)."""
+    lines = ["## Findings", "", "| finding | location | rung | severity |", "|---|---|---|---|"]
+    for finding in findings:
+        where = f"{finding.path}:{finding.line}" if finding.line else finding.path
+        lines.append(f"| {finding.title} | {where} | `{finding.rung}` | {finding.severity} |")
+    return "\n".join(lines) + "\n"
+
+
+__all__ = ["SliceRow", "overfitting_gap", "render", "render_findings"]