feat(learning-harness): one teacher rule across every learning path

Task 3.1. A teacher is a vendored pair, at a review tier a maintainer stands
behind, on the train split, and scorable. Every learning path now asks the same
module that one question, so the split cannot be enforced in one place and
forgotten in another. A refusal is data rather than an exception: the path
records which holdout pairs a candidate touched and moves on, so a leak is
visible in the run instead of silently accepted.

The exporter refuses to seed from a holdout pair, leave-one-out teaches from the
train split while still scoring every pair, and the mechanism lever refuses a
candidate a holdout pair taught, because such a shape can only recover the pair
that produced it.

Measured and committed: on the vibe-py holdout the lever's own number falls from
seven detected and a Youden of +0.059 to five detected and -0.059 once holdout
pairs stop teaching, and the candidate set shrinks from ten shapes to six. That
gap is the inflation this rule removes.

Two existing tests encoded the leak and now encode the truth. The documented
export-to-improve flow had a single holdout pair teaching itself, and now needs
a train pair to teach and a holdout pair to recover. The obligation export
claimed the corpus seeds at least one obligation shape, which was true only
because holdout pairs were teaching: every vibe-py absence pair that yields an
obligation lesson sits on the holdout side, so the honest number today is zero.
The derivation stays covered by fixtures; the corpus gap belongs to the absence
re-harvest and to a maintainer pass over the splits.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

src/openultrasast/learning/split.py

@@ -0,0 +1,64 @@
+"""One rule for who may teach, and one refusal for anything that touches the holdout (Req 5.1, 5.2).
+
+Every learning path in the tool asks this module the same question, so the split cannot be enforced in
+one place and forgotten in another. A teacher is a vendored pair, at a review tier a maintainer stands
+behind, on the train split, and scorable. Everything else may be measured; nothing else may teach.
+
+A refusal is data, not an exception: the path records which holdout pairs a candidate touched and moves
+on, so a leak is visible in the run rather than silently accepted.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Iterable, Sequence
+from dataclasses import dataclass
+
+GATING_TIERS = frozenset({"seeded", "reviewed"})
+TRAIN = "train"
+HOLDOUT = "holdout"
+REFUSED = "holdout_pair_refused"
+
+
+@dataclass(frozen=True)
+class Refusal:
+    """A candidate that a holdout pair taught, or that touches one."""
+
+    pairs: tuple[str, ...]
+    reason: str = REFUSED
+
+    def degradation(self) -> dict[str, object]:
+        return {
+            "stage": "learning",
+            "reason": self.reason,
+            "pairs": list(self.pairs),
+            "detail": f"refused: taught by or touching holdout pair(s) {', '.join(self.pairs)}",
+        }
+
+
+def teachers(cases: Sequence[object]) -> tuple[object, ...]:
+    """The pairs a learning path may learn from. Everything else is measured, never learned from."""
+    return tuple(case for case in cases if is_teacher(case))
+
+
+def is_teacher(case: object) -> bool:
+    if not bool(getattr(case, "vendored", True)):
+        return False
+    if str(getattr(case, "review_tier", "")) not in GATING_TIERS:
+        return False
+    if str(getattr(case, "split", TRAIN)) != TRAIN:
+        return False
+    return getattr(case, "unscorable", None) is None
+
+
+def holdout_names(cases: Sequence[object]) -> frozenset[str]:
+    return frozenset(str(getattr(case, "name", "")) for case in cases if str(getattr(case, "split", "")) == HOLDOUT)
+
+
+def refuse_if_holdout(taught_by: Iterable[str], cases: Sequence[object]) -> Refusal | None:
+    """None when nothing in ``taught_by`` is a holdout pair; a refusal naming them when something is."""
+    held = holdout_names(cases)
+    touched = tuple(sorted({name for name in taught_by if name in held}))
+    return Refusal(pairs=touched) if touched else None
+
+
+__all__ = ["GATING_TIERS", "HOLDOUT", "REFUSED", "TRAIN", "Refusal", "holdout_names", "is_teacher", "refuse_if_holdout", "teachers"]