feat(pair-corpus-honesty): score non-vendored pointer pairs through a local cache

Task 8.3 (Req 10.1-10.5).

- pairs.py: catalog rows with vendored = false carry their harvest recipe
  and no excerpts; evaluate_catalog harvests them into
  OPENULTRASAST_PAIR_CACHE (default ~/.cache/openultrasast/pairs) only
  when `pairs --pointers` or OPENULTRASAST_PAIRS_NETWORK=1 allows the
  network, otherwise records pointer_pair_skipped (a failed fetch is
  pointer_pair_fetch_failed with detail); a warm cache never scores
  without the switch. select_vendored for tests and gates.
- harvest.py: materialize_pointer harvests into the cache without the
  license requirement (nothing is redistributed); redaction applies.
- catalog_gen.py: emits pointer rows with recipe fields; training
  manifests skip them.
- pair_gate and improve/evolve.evaluate_profiles score vendored pairs
  only; cli `pairs --pointers`; .gitignore covers the cache path.
- tests: loader, skip/degradation, cache hit, fetch failure, warm cache
  with network off, profile gate exclusion, harvest and generator paths;
  conftest keeps every test offline whatever the shell exports.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

.gitignore

@@ -19,6 +19,8 @@ build/
 *.egg-info/
 
 **/.openultrasast/
+.cache/openultrasast/
+**/.cache/openultrasast/
 artifacts/
 reports/
 vector-store/

.kiro/specs/pair-corpus-honesty/tasks.md

@@ -169,7 +169,7 @@
   - _Boundary: ProfileGate_
   - _Depends: 8.1_
 
-- [ ] 8.3 Score non-vendored pointer pairs through a local cache
+- [x] 8.3 Score non-vendored pointer pairs through a local cache
   - Catalog rows with `vendored = false` carry the recipe; evaluation harvests both sides into the cache when `OPENULTRASAST_PAIRS_NETWORK=1` (or `pairs --pointers`), otherwise skips with a degradation; the cache is git-ignored; tests and gates select vendored pairs only.
   - Observable: a pointer row evaluates to the same outcome shape as a vendored row when network is on, is skipped with `pointer_pair_skipped` when off, and no cache file appears under the repository.
   - _Requirements: 10.1, 10.2, 10.3, 10.4, 10.5_
@@ -196,6 +196,9 @@
 - Review round 7 (2026-09-05) APPROVED tasks 7.1 and 7.2; REJECTED 8.1 on documentation only (REVIEW.md still described the candidates file; READMEs claimed the tier gate filter before 8.2 existed; TS zeros misattributed). Remediated: REVIEW.md/READMEs rewritten to tier semantics, note corrected, `catalog_gen._review_tier` precedence aligned with the loader (explicit tier wins), `training/manifest.jsonl` rows carry `review_tier`, design §LabelSchema candidates sentence superseded, roadmap agent-vfc line updated. Deferred to the engine specs: a `language_unsupported` loss counter (reviewer suggestion).
 - Task 8.2 (2026-09-05, RED first: `test_profile_gate_ignores_title_and_advisory_tiers` failed with `reason='profile_regression:agent'` on five title-tier pairs; then green): `evolve.evaluate_profiles` filters holdout pairs to `GATING_TIERS` via `select_tier` before scoring, so `advisory`/`title` pairs never gate and a regression on `reviewed` pairs still rejects the round.
 - Review round 8 (2026-09-05) APPROVED 8.1 and 8.2 (verify-completion: `pytest -q` 473 passed / 4 skipped, extra-free 460 / 17, ruff/mypy clean, three gates PASS; reviewer reproduced RED by removing the tier filter and checked the default holdout gate now covers `human` seeded pairs only, with the 16 title-tier agent rows excluded). Task 5.3 closed with 8.1 as re-scoped. Applied suggestion: the agent-vfc tier test accepts `reviewed` rows so promoting a row does not break it. Follow-up for corpus-seeded-mechanisms: report holdout profiles excluded by tier (`profiles_ungated`) so an operator sees why `agent`/`synthetic` do not gate.
+- Task 8.3 (2026-09-05, RED first: 6 failing tests captured — `pointer_recipe`/`select_vendored` missing, loader `KeyError: 'vuln'` on a `vendored = false` row (x2), `pairs --pointers` unknown flag (`SystemExit: 2`), `validate_recipe(require_license=)` unknown, `catalog_gen` skipped pointer recipes as missing excerpts; then green): `PairCase.vendored`/`recipe`, `pair_cache_dir` (`OPENULTRASAST_PAIR_CACHE`, default `~/.cache/openultrasast/pairs`), `network_allowed` (`--pointers` per run or `OPENULTRASAST_PAIRS_NETWORK=1`), `select_vendored`, `_pointer_ready` (cached -> score; network -> `harvest.materialize_pointer` via the shared library; else `pointer_pair_skipped`; a failed fetch is `pointer_pair_fetch_failed`), `evaluate_catalog(pointers=)`; `harvest.validate_recipe(require_license=False)` and `materialize_pointer` (cache only, redaction applies); `catalog_gen` emits `vendored = false` rows with recipe fields and no excerpts, training manifest skips them; `pair_gate` scores vendored local pairs only; `.gitignore` covers the cache path. Smoke with real network on two Real-Vuln LLM-repo recipes into a scratchpad cache: network off -> 0 outcomes, 2 x `pointer_pair_skipped`; network on -> both harvested to `<cache>/vibe-py/<name>/{vuln,fixed}.py` and scored (silent fix on both, vuln side undetected, unresolved 0); nothing written under the repository. Suite 479 passed / 4 skipped, extra-free 466 / 17, ruff/mypy clean, three gates PASS.
+- Review round 9 (2026-09-05) REJECTED 8.3 on two findings, both fixed RED-first (2 failing tests captured: warm cache + network off scored the pointer pair; `evaluate_profiles` kept an `agent` pointer pair): `_pointer_ready` consults `network_allowed` before the cache, so a warm cache only spares the fetch under `--pointers`/env and never turns a default run into a pointer run; `evolve.evaluate_profiles` applies `select_vendored` before the tier filter (design amendment: "tests and gates call `select_vendored`"; ProfileGate is the gate the design names, so the one-line filter is in boundary). Suggestions applied: fetch failures carry a `detail`; the harvest module load is memoized; pointer excerpt headers get `function`/`mechanism` from the expected row; `tests/conftest.py` autouse fixture clears `OPENULTRASAST_PAIRS_NETWORK` and points the cache at a temp dir for every test; full-catalog on-disk assertions use `select_vendored`.
+- Review round 10 (2026-09-05) APPROVED 8.3 (verify-completion: `pytest -q` 481 passed / 4 skipped; extra-free 468 / 17 inside a no-network namespace; ruff/format/mypy clean; three gates PASS; reviewer reproduced both RED tests by mutation, the warm-cache/network-off skip on the real cache, and that the default run creates no cache and touches no network). Non-blocking: top-level README/docs mention `--pointers` only in benchmarks/pairs/README.md (ride with 8.4); the cwd-relative harvest library path follows the DEFAULT_CATALOG convention.
 - ESCALATION (task 5.1 observable): vibe-py has `human` provenance only because the 40 LLM-generated Real-Vuln repos have no license (Req 5.4 wins). Spec owner decision needed: accept human-only vibe-py, or obtain a license statement upstream.
 - ESCALATION (Req 5.3): agent-vfc rows are unreviewed. They are NOT in the loaded catalog; `benchmarks/pairs/agent-vfc/catalog-candidates.toml` and `REVIEW.md` list them for a human reviewer. Task 5.3 stays open until reviewed.
 

benchmarks/pairs/README.md

@@ -67,6 +67,18 @@ Defaults per slice: local `reviewed`, vibe-py `seeded`, sast/vfc/vfc-js/github
 `advisory`, agent-vfc `title`. `pairs --json` reports `per_tier`; `evolve.evaluate_profiles` scores only `seeded`
 and `reviewed` pairs, so `advisory` and `title` pairs never gate the improve loop (task 8.2).
 
+## Pointer pairs (not vendored)
+
+A row with `vendored = false` carries its harvest recipe (`repo`, `parent`, `commit`,
+`path`, `mode`, `line`/`fix_path`/`fix_line`, ...) and no excerpt files (Req 10). The
+repository never redistributes that code: `pairs --pointers` (or
+`OPENULTRASAST_PAIRS_NETWORK=1`) harvests both sides into
+`OPENULTRASAST_PAIR_CACHE` (default `~/.cache/openultrasast/pairs/<slice>/<name>/`,
+git-ignored) and scores them exactly like a vendored pair; without network the pair is
+skipped with a `pointer_pair_skipped` degradation and a failed fetch is
+`pointer_pair_fetch_failed`, never an error. The default `pytest` run and every gate
+score vendored pairs only (`select_vendored`); the pointer run is the nightly command.
+
 ## Labels, profiles, and mechanisms
 
 Every `[[pair.expected]]` row names a `mechanism` from the closed vocabulary in

benchmarks/pairs/catalog_gen.py

@@ -52,6 +52,7 @@ def catalog_text(slice_name: str, recipes: list[dict[str, object]]) -> str:
     for recipe in recipes:
         vuln, fixed = excerpt_rel(recipe)
         lang = LANG_TO_PAIR.get(str(recipe.get("language", "c")), "c_cpp")
+        pointer = recipe.get("vendored") is False
         lines += [
             "[[pair]]",
             f'name = "{_q(recipe["name"])}"',
@@ -65,8 +66,13 @@ def catalog_text(slice_name: str, recipes: list[dict[str, object]]) -> str:
             f'commit = "{_q(recipe.get("commit", ""))}"',
             f'parent = "{_q(recipe.get("parent", ""))}"',
             f'commit_url = "{_q(recipe.get("commit_url", ""))}"',
-            f'vuln = "{vuln}"',
-            f'fixed = "{fixed}"',
+        ]
+        if pointer:
+            lines.append("vendored = false")  # Req 10.1: the recipe travels in the row, no excerpt in the repository
+            lines += _pointer_lines(recipe)
+        else:
+            lines += [f'vuln = "{vuln}"', f'fixed = "{fixed}"']
+        lines += [
             f'relpath = "{_q(recipe.get("relpath", recipe.get("path", "")))}"',
             "min_recall = 1.0",
             'fix_policy = "silent"',
@@ -114,9 +120,21 @@ def _review_tier(slice_name: str, recipe: dict[str, object]) -> str:
     return _DEFAULT_TIER.get(slice_name, "advisory")
 
 
+_POINTER_STR = ("host", "path", "fix_repo", "fix_path", "mode")
+_POINTER_INT = ("line", "fix_line", "line_start", "line_end")
+
+
+def _pointer_lines(recipe: dict[str, object]) -> list[str]:
+    out = [f'{key} = "{_q(recipe[key])}"' for key in _POINTER_STR if recipe.get(key)]
+    out += [f"{key} = {int(recipe[key])}" for key in _POINTER_INT if isinstance(recipe.get(key), int)]
+    return out
+
+
 def training_rows(slice_name: str, recipes: list[dict[str, object]]) -> list[dict[str, object]]:
     rows: list[dict[str, object]] = []
     for recipe in recipes:
+        if recipe.get("vendored") is False:
+            continue  # nothing vendored to train on
         vuln, fixed = excerpt_rel(recipe)
         base = {
             "pair": recipe["name"],
@@ -353,7 +371,12 @@ def generate(slice_name: str, *, prune: bool = False) -> tuple[Path, int]:
     recipes: list[dict[str, object]] = []
     for recipe in loaded:
         vuln, fixed = excerpt_rel(recipe)
-        if (slice_root / vuln).is_file() and (slice_root / fixed).is_file():
+        if recipe.get("vendored") is False:
+            if not recipe.get("function"):
+                print(f"skip {recipe['name']}: pointer recipe without a function label (nothing to derive from)")
+                continue
+            recipes.append(recipe)
+        elif (slice_root / vuln).is_file() and (slice_root / fixed).is_file():
             if not recipe.get("function"):
                 derived = derive_function(slice_root, recipe)
                 if not derived:

benchmarks/pairs/harvest.py

@@ -373,7 +373,7 @@ def provenance_header(recipe: dict[str, Any], *, side: str, sha: str, upstream_s
     return "/* " + "\n * ".join(lines) + "\n */\n"
 
 
-def validate_recipe(recipe: dict[str, Any]) -> None:
+def validate_recipe(recipe: dict[str, Any], *, require_license: bool = True) -> None:
     name = str(recipe.get("name", ""))
     if any(token in str(recipe).lower() for token in BLOCKED_TOKENS):
         raise RecipeError(f"{name}: embargoed or blocked dataset")
@@ -390,7 +390,7 @@ def validate_recipe(recipe: dict[str, Any]) -> None:
         raise RecipeError(f"{name}: line_range mode needs integer line_start and line_end")
     if mode == "enclosing" and not isinstance(recipe.get("line"), int):
         raise RecipeError(f"{name}: enclosing mode needs an integer line")
-    if not recipe.get("license"):
+    if require_license and not recipe.get("license"):
         raise RecipeError(f"{name}: license required")
 
 
@@ -514,8 +514,8 @@ def extract_pair(recipe: dict[str, Any], parent_src: str, fix_src: str) -> tuple
     return extract_function(parent_src, function, language), extract_function(fix_src, function, language)
 
 
-def harvest_recipe(recipe: dict[str, Any], root: Path) -> tuple[Path, Path]:
-    validate_recipe(recipe)
+def harvest_recipe(recipe: dict[str, Any], root: Path, *, require_license: bool = True) -> tuple[Path, Path]:
+    validate_recipe(recipe, require_license=require_license)
     if str(recipe.get("host", "github")) != "github":
         raise RecipeError(f"{recipe.get('name')}: live fetch currently implemented for host=github only")
     parent_src = fetch_url(github_raw_url(str(recipe["repo"]), str(recipe["parent"]), str(recipe["path"])))
@@ -531,6 +531,19 @@ def harvest_recipe(recipe: dict[str, Any], root: Path) -> tuple[Path, Path]:
     return vuln_path, fixed_path
 
 
+def materialize_pointer(recipe: dict[str, Any], cache_root: Path, *, slice_name: str) -> tuple[Path, Path]:
+    """Harvest a non-vendored pair into ``cache_root/<slice>/<name>/`` (Req 10.2).
+
+    The cache lives outside the repository; a missing upstream license is allowed here because nothing is redistributed,
+    the excerpts stay on the operator's machine. Redaction applies as for vendored excerpts.
+    """
+    root = Path(cache_root) / slice_name
+    vuln_path, fixed_path = excerpt_paths(recipe, root)
+    if vuln_path.is_file() and fixed_path.is_file():
+        return vuln_path, fixed_path
+    return harvest_recipe(recipe, root, require_license=False)
+
+
 def fetch_url(url: str, timeout: int = 60) -> str:
     host = urlparse(url).hostname or ""
     if host not in ALLOWED_HOSTS:

src/openultrasast/cli.py

@@ -156,6 +156,7 @@ def main(argv: list[str] | None = None) -> int:
     pairs.add_argument("--split", choices=("all", "train", "holdout"), default="all", help="filter pairs by declared split")
     pairs.add_argument("--hunter", action="store_true", help="also score the LLM tool hunter on overlay slices (needs a hunter model)")
     pairs.add_argument("--hunter-model", default=None, help="model id for --hunter (default: [models].hunter from openultrasast.toml)")
+    pairs.add_argument("--pointers", action="store_true", help="allow network for non-vendored pointer pairs this run (nightly; CI never)")
     pairs.add_argument("--json", action="store_true", help="print the pair scoreboard as JSON")
 
     subparsers.add_parser("mcp", help="run the narrow MCP server over stdio for OpenCode integration")
@@ -190,6 +191,7 @@ def main(argv: list[str] | None = None) -> int:
             split=args.split,
             hunter=args.hunter,
             hunter_model=args.hunter_model,
+            pointers=args.pointers,
         )
     if args.command == "mcp":
         from .mcp import serve  # lazy: keeps the import cycle (mcp -> cli) one-directional
@@ -787,6 +789,7 @@ def _pairs(
     split: str = "all",
     hunter: bool = False,
     hunter_model: str | None = None,
+    pointers: bool = False,
 ) -> int:
     if not catalog.exists() or not catalog.is_file():
         raise SystemExit(f"pair catalog is not a file: {catalog}")
@@ -799,7 +802,7 @@ def _pairs(
         client = tool_hunter.resolve_hunter_client()
         if model and client is not None:
             scan = make_hunter_scan(client, model)
-    result = evaluate_catalog(cases, hunter=scan)
+    result = evaluate_catalog(cases, hunter=scan, pointers=True if pointers else None)
     if json_out:
         print(json.dumps(result_payload(result), indent=2, sort_keys=True))
         return 0

src/openultrasast/improve/evolve.py

@@ -61,9 +61,10 @@ def evaluate_profiles(
 
     Only ``seeded`` and ``reviewed`` pairs gate (Req 9.3); ``advisory`` and ``title`` pairs are scored by ``pairs``, never here.
     """
-    from ..pairs import GATING_TIERS, PairCase, evaluate_catalog, select_tier
+    from ..pairs import GATING_TIERS, PairCase, evaluate_catalog, select_tier, select_vendored
 
-    cases = list(select_tier([case for case in pair_cases if isinstance(case, PairCase)], GATING_TIERS))
+    # Gates score vendored pairs only (Req 10.4); pointer pairs are for `pairs --pointers`, never for the improve gate.
+    cases = list(select_tier(select_vendored([case for case in pair_cases if isinstance(case, PairCase)]), GATING_TIERS))
     if not cases:
         return {}
     result = evaluate_catalog(cases, ruleset=rules, inventory_sidecar=False)

src/openultrasast/pair_gate.py

@@ -21,6 +21,7 @@ from .pairs import (
     evaluate_catalog,
     load_pair_catalog,
     select_slice,
+    select_vendored,
 )
 
 
@@ -33,7 +34,7 @@ class PairGateVerdict:
 
 def pair_gate(cases: Sequence[PairCase] | None = None) -> PairGateVerdict:
     catalog = tuple(cases) if cases is not None else load_pair_catalog(DEFAULT_CATALOG)
-    local = select_slice(catalog, "local")
+    local = select_vendored(select_slice(catalog, "local"))  # Req 10.4: gates score vendored pairs only
     if not local:
         return PairGateVerdict(passed=False, reasons=("no local pairs in catalog",), result=evaluate_catalog(()))
     result = evaluate_catalog(local)

src/openultrasast/pairs.py

@@ -13,6 +13,8 @@ evaluated but excluded from the achievable numbers.
 
 from __future__ import annotations
 
+import importlib.util
+import os
 import shutil
 import tempfile
 import tomllib
@@ -58,6 +60,29 @@ SPLITS = frozenset({"train", "holdout"})
 # Req 9: how a label was established. Only seeded and reviewed pairs steer the improve loop.
 REVIEW_TIERS = frozenset({"seeded", "advisory", "title", "reviewed"})
 GATING_TIERS = frozenset({"seeded", "reviewed"})
+# Req 10: pointer pairs are harvested into a cache outside the repository, never vendored.
+PAIR_CACHE_ENV = "OPENULTRASAST_PAIR_CACHE"
+PAIRS_NETWORK_ENV = "OPENULTRASAST_PAIRS_NETWORK"
+DEFAULT_PAIR_CACHE = Path.home() / ".cache" / "openultrasast" / "pairs"
+POINTER_REQUIRED = ("repo", "parent", "commit", "path", "mode")
+POINTER_FIELDS = POINTER_REQUIRED + (
+    "host",
+    "fix_repo",
+    "fix_path",
+    "line",
+    "fix_line",
+    "line_start",
+    "line_end",
+    "function",
+    "language",
+    "license",
+    "relpath",
+    "provenance",
+    "mechanism",
+    "cve",
+    "commit_url",
+)
+HARVEST_LIBRARY = Path("benchmarks/pairs/harvest.py")
 _DEFAULT_TIER = {
     "local": "reviewed",
     "github": "advisory",
@@ -99,6 +124,8 @@ class PairCase:
     known_limit: str | None = None
     review_tier: str = "advisory"
     reviewer: str = ""
+    vendored: bool = True
+    recipe: tuple[tuple[str, object], ...] = ()  # pointer pairs carry their harvest recipe instead of excerpts
 
 
 @dataclass(frozen=True)
@@ -222,14 +249,20 @@ def _load_catalog_file(path: Path) -> tuple[PairCase, ...]:
             raise CatalogError(f"pair {name}: unknown split {split!r}; expected train or holdout")
         known_limit = item.get("known_limit")
         review_tier, reviewer = _review_tier(item, slice_name, name)
+        vendored = item.get("vendored", True) is not False
+        recipe: tuple[tuple[str, object], ...] = ()
+        if vendored:
+            vuln_file, fixed_file = _resolve(root, str(item["vuln"])), _resolve(root, str(item["fixed"]))
+        else:
+            vuln_file, fixed_file, recipe = _pointer_files(item, slice_name, name)
         cases.append(
             PairCase(
                 name=name,
                 slice=slice_name,
                 language=str(item.get("language", "")),
                 origin=str(item.get("origin", "")),
-                vuln_file=_resolve(root, str(item["vuln"])),
-                fixed_file=_resolve(root, str(item["fixed"])),
+                vuln_file=vuln_file,
+                fixed_file=fixed_file,
                 relpath=str(item["relpath"]),
                 expected=expected,
                 min_recall=float(item.get("min_recall", 1.0)),
@@ -244,11 +277,90 @@ def _load_catalog_file(path: Path) -> tuple[PairCase, ...]:
                 known_limit=str(known_limit) if known_limit else None,
                 review_tier=review_tier,
                 reviewer=reviewer,
+                vendored=vendored,
+                recipe=recipe,
             )
         )
     return tuple(cases)
 
 
+def pair_cache_dir() -> Path:
+    """Cache root for pointer pairs (Req 10.2); outside the repository unless the operator points it elsewhere."""
+    return Path(os.environ.get(PAIR_CACHE_ENV) or DEFAULT_PAIR_CACHE).expanduser()
+
+
+def network_allowed(pointers: bool | None = None) -> bool:
+    """``pairs --pointers`` for one run, or OPENULTRASAST_PAIRS_NETWORK=1; CI never sets either (Req 10.3, 10.4)."""
+    if pointers is not None:
+        return pointers
+    return os.environ.get(PAIRS_NETWORK_ENV, "") == "1"
+
+
+def pointer_recipe(case: PairCase) -> dict[str, object]:
+    return dict(case.recipe)
+
+
+def select_vendored(cases: Sequence[PairCase]) -> tuple[PairCase, ...]:
+    """Tests and gates score vendored pairs only (Req 10.4)."""
+    return tuple(case for case in cases if case.vendored)
+
+
+def _pointer_files(item: dict[str, object], slice_name: str, name: str) -> tuple[Path, Path, tuple[tuple[str, object], ...]]:
+    missing = [key for key in POINTER_REQUIRED if not item.get(key)]
+    if missing:
+        raise CatalogError(f"pair {name}: vendored = false requires {', '.join(POINTER_REQUIRED)} (missing {', '.join(missing)})")
+    recipe = {key: item[key] for key in POINTER_FIELDS if key in item}
+    recipe["name"] = name
+    recipe["slice"] = slice_name
+    expected = item.get("expected")
+    if isinstance(expected, list) and expected and isinstance(expected[0], dict):  # header fields live on the expected row
+        for key in ("function", "mechanism"):
+            recipe.setdefault(key, expected[0].get(key, ""))
+    ext = Path(str(item["path"])).suffix or ".c"
+    folder = (pair_cache_dir() / slice_name / name).resolve()
+    return folder / f"vuln{ext}", folder / f"fixed{ext}", tuple(sorted(recipe.items()))
+
+
+@lru_cache(maxsize=1)
+def _harvest_library() -> object:
+    spec = importlib.util.spec_from_file_location("openultrasast_pair_harvest", HARVEST_LIBRARY.resolve())
+    if spec is None or spec.loader is None:
+        raise OSError(f"harvest library not found: {HARVEST_LIBRARY}")
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def _materialize_pointer(recipe: dict[str, object], cache_root: Path) -> tuple[Path, Path]:
+    """Harvest both sides of a pointer pair into ``cache_root`` through the shared harvest library (network)."""
+    module = _harvest_library()
+    vuln, fixed = module.materialize_pointer(  # type: ignore[attr-defined]
+        recipe, cache_root, slice_name=str(recipe["slice"])
+    )
+    return Path(vuln), Path(fixed)
+
+
+def _pointer_ready(case: PairCase, *, pointers: bool | None) -> dict[str, object] | None:
+    """None when the pair can be scored; otherwise the degradation entry (skipped or fetch failed).
+
+    The network switch is consulted first: a warm cache only spares the fetch under ``--pointers``/env, it never turns a
+    default run into a pointer run (Req 10.3).
+    """
+    if case.vendored:
+        return None
+    if not network_allowed(pointers):
+        return {"stage": "pairs", "reason": "pointer_pair_skipped", "pair": case.name}
+    if case.vuln_file.is_file() and case.fixed_file.is_file():
+        return None
+    try:
+        _materialize_pointer(pointer_recipe(case), pair_cache_dir())
+    except Exception as exc:  # noqa: BLE001 - a failed fetch is a recorded loss, never a crash of the whole run
+        return {"stage": "pairs", "reason": "pointer_pair_fetch_failed", "pair": case.name, "detail": f"{type(exc).__name__}: {exc}"[:200]}
+    if case.vuln_file.is_file() and case.fixed_file.is_file():
+        return None
+    return {"stage": "pairs", "reason": "pointer_pair_fetch_failed", "pair": case.name, "detail": "harvest wrote no excerpts"}
+
+
 def _review_tier(item: dict[str, object], slice_name: str, name: str) -> tuple[str, str]:
     """Tier from the row, else from ``reviewer`` (pending -> title, a name -> reviewed), else the slice default."""
     reviewer = str(item.get("reviewer", "") or "")
@@ -286,11 +398,24 @@ def evaluate_catalog(
     hunter: HunterScan | None = None,
     ruleset: tuple[PatternRule, ...] | None = None,
     inventory_sidecar: bool = True,
+    pointers: bool | None = None,
 ) -> PairEvalResult:
-    """Score every case. ``ruleset`` lets the improve loop score a candidate ledger; ``inventory_sidecar=False`` skips the second scan."""
+    """Score every case. ``ruleset`` lets the improve loop score a candidate ledger; ``inventory_sidecar=False`` skips the second scan.
+
+    Pointer pairs (``vendored = false``) are harvested into the cache when ``pointers`` (or OPENULTRASAST_PAIRS_NETWORK=1) allows the
+    network, otherwise skipped with a ``pointer_pair_skipped`` degradation (Req 10.2, 10.3).
+    """
+    degradations: list[dict[str, object]] = []
+    ready: list[PairCase] = []
+    for case in cases:
+        degradation = _pointer_ready(case, pointers=pointers)
+        if degradation is None:
+            ready.append(case)
+        else:
+            degradations.append(degradation)
+    cases = tuple(ready)
     outcomes = tuple(evaluate_pair(case, ruleset=ruleset) for case in cases)
     scorers: dict[str, dict[str, PairCorpusMetrics]] = {}
-    degradations: list[dict[str, object]] = []
     for slice_name in sorted({case.slice for case in cases if case.slice in OVERLAY_SLICES}):
         slice_cases = tuple(case for case in cases if case.slice == slice_name)
         scorers[slice_name] = {"overlay": _metrics(tuple(item for item in outcomes if item.slice == slice_name))}

tests/conftest.py

@@ -12,6 +12,13 @@ _HARNESSX_GUARD = (
 )
 
 
+@pytest.fixture(autouse=True)
+def _pointer_pairs_offline(monkeypatch: pytest.MonkeyPatch, tmp_path_factory: pytest.TempPathFactory) -> None:
+    """Req 10.4: the default test run never touches the network for pointer pairs, whatever the operator's shell exports."""
+    monkeypatch.delenv("OPENULTRASAST_PAIRS_NETWORK", raising=False)
+    monkeypatch.setenv("OPENULTRASAST_PAIR_CACHE", str(tmp_path_factory.mktemp("pair-cache")))
+
+
 @pytest.fixture
 def assert_cold_of_harnessx():  # type: ignore[no-untyped-def]
     """Run ``code`` in a fresh interpreter and assert it imported no ``harnessx`` module.

tests/test_improve.py

@@ -293,3 +293,32 @@ def test_profile_gate_ignores_title_and_advisory_tiers(tmp_path: Path) -> None:
     )
     assert not outcome.accepted and outcome.reason == "profile_regression:agent"
     assert outcome.per_profile_before["agent"]["pairs"] == 5.0  # the five title pairs are scored elsewhere, never here
+
+
+def test_profile_gate_scores_vendored_pairs_only(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    """Req 10.4 / design: gates call select_vendored; a pointer pair never reaches the improve gate, cached or not."""
+    from openultrasast.improve.evolve import evaluate_profiles
+    from openultrasast.pairs import PairCase
+    from openultrasast.ruleset import load_ruleset
+
+    monkeypatch.setenv("OPENULTRASAST_PAIRS_NETWORK", "1")
+    vendored = _pair(
+        tmp_path, "v", "def h(y):\n    return eval(y)\n", "def h(y):\n    return y\n", "python-unsafe-eval", "eval", provenance="human"
+    )
+    pointer_src = tmp_path / "cache" / "github" / "p"
+    pointer_src.mkdir(parents=True)
+    (pointer_src / "vuln.py").write_text("def h(y):\n    return eval(y)\n")
+    (pointer_src / "fixed.py").write_text("def h(y):\n    return y\n")
+    pointer = PairCase(
+        **{
+            **vendored.__dict__,
+            "name": "p",
+            "provenance": "agent",
+            "vendored": False,
+            "vuln_file": pointer_src / "vuln.py",
+            "fixed_file": pointer_src / "fixed.py",
+            "recipe": (("repo", "o/r"), ("parent", "a"), ("commit", "b"), ("path", "app.py"), ("mode", "name")),
+        }
+    )
+    profiles = evaluate_profiles([vendored, pointer], load_ruleset())
+    assert set(profiles) == {"human"}  # the agent pointer pair is scored by `pairs --pointers`, never by the gate

tests/test_pair_corpus.py

@@ -20,6 +20,7 @@ from openultrasast.pairs import (
     load_datasets,
     load_pair_catalog,
     select_slice,
+    select_vendored,
 )
 
 
@@ -36,7 +37,7 @@ def test_catalog_loads_local_and_github_slices() -> None:
     assert "openssl-cve-2014-0160" in names
     assert "firefox-cve-2020-15667" in names
     assert "chromium-cve-2019-5786" in names
-    assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in cases)
+    assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in select_vendored(cases))
 
 
 def test_pair_catalog_is_outside_stage1_smoke_gate() -> None:
@@ -459,9 +460,9 @@ def test_new_slices_load_offline_and_score_with_profiles() -> None:
     for slice_name, expected_provenance in (("vibe-py", "human"), ("vfc-js", "human")):
         cases = select_slice(catalog, slice_name)
         assert cases, slice_name
-        assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in cases)
-        assert all(case.license for case in cases), slice_name
-        assert all(case.provenance == expected_provenance for case in cases), slice_name
+        assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in select_vendored(cases))
+        assert all(case.license for case in select_vendored(cases)), slice_name  # pointer rows may be unlicensed (Req 10)
+        assert all(case.provenance == expected_provenance for case in select_vendored(cases)), slice_name
         assert {case.split for case in cases} <= {"train", "holdout"}
         header = cases[0].vuln_file.read_text()[:600]
         assert "Provenance:" in header and "license:" in header
@@ -678,7 +679,7 @@ def test_payload_reports_metrics_per_review_tier(tmp_path: Path) -> None:
 def test_default_catalog_has_the_new_slices_on_disk_and_off_the_stage1_list() -> None:
     catalog = load_pair_catalog()
     assert {"vibe-py", "vfc-js", "agent-vfc"} <= {case.slice for case in catalog}  # Req 8.1
-    assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in catalog)
+    assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in select_vendored(catalog))
     stage1 = {name for names in LANGUAGE_MANIFESTS.values() for name in names}
     assert not ({case.name for case in catalog} & stage1)
 
@@ -859,3 +860,106 @@ def test_cpp_qualified_method_range_is_found_by_its_unqualified_label() -> None:
     assert ranges is not None
     assert spans_named(ranges, "m") == ((1, 3),)
     assert function_at(ranges, 2) == "m"
+
+
+# --- task 8.3: pointer pairs (Req 10) -----------------------------------------
+
+
+_POINTER_ROW = (
+    '[[pair]]\nname = "ptr"\nslice = "vibe-py"\nlanguage = "python"\nvendored = false\nrepo = "o/r"\nparent = "aaa"\ncommit = "bbb"\n'
+    'path = "app.py"\nmode = "enclosing"\nline = 3\nrelpath = "app.py"\nprovenance = "agent"\nreview_tier = "seeded"\nlicense = ""\n\n'
+    '[[pair.expected]]\ncwe = "CWE-95"\nclass = "code injection"\npath = "app.py"\nfunction = "f"\n'
+    'sink = "eval"\nmechanism = "source_reaches_sink"\n'
+)
+
+
+def test_pointer_rows_load_without_excerpts_and_point_into_the_cache(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    from openultrasast.pairs import CatalogError, _load_catalog_file, pointer_recipe, select_vendored
+
+    monkeypatch.setenv("OPENULTRASAST_PAIR_CACHE", str(tmp_path / "cache"))
+    (tmp_path / "c.toml").write_text(_POINTER_ROW)
+    (case,) = _load_catalog_file(tmp_path / "c.toml")
+    assert case.vendored is False
+    assert case.vuln_file == (tmp_path / "cache" / "vibe-py" / "ptr" / "vuln.py").resolve()
+    assert pointer_recipe(case)["parent"] == "aaa" and pointer_recipe(case)["mode"] == "enclosing"
+    assert select_vendored([case]) == ()
+    (tmp_path / "bad.toml").write_text(_POINTER_ROW.replace('parent = "aaa"\n', ""))
+    with pytest.raises(CatalogError, match="parent"):
+        _load_catalog_file(tmp_path / "bad.toml")
+
+
+def test_pointer_pairs_skip_with_a_degradation_when_network_is_off(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    from openultrasast.pairs import _load_catalog_file, evaluate_catalog
+
+    monkeypatch.setenv("OPENULTRASAST_PAIR_CACHE", str(tmp_path / "cache"))
+    monkeypatch.delenv("OPENULTRASAST_PAIRS_NETWORK", raising=False)
+    (tmp_path / "c.toml").write_text(_POINTER_ROW)
+    result = evaluate_catalog(_load_catalog_file(tmp_path / "c.toml"))
+    assert result.outcomes == ()
+    assert {"stage": "pairs", "reason": "pointer_pair_skipped", "pair": "ptr"} in result.degradations
+    assert not (tmp_path / "cache").exists()
+
+
+def test_pointer_pairs_materialize_into_the_cache_and_score_like_vendored_pairs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    from openultrasast import pairs as pairs_mod
+
+    monkeypatch.setenv("OPENULTRASAST_PAIR_CACHE", str(tmp_path / "cache"))
+    monkeypatch.setenv("OPENULTRASAST_PAIRS_NETWORK", "1")
+    (tmp_path / "c.toml").write_text(_POINTER_ROW)
+    (case,) = pairs_mod._load_catalog_file(tmp_path / "c.toml")
+
+    def fake_materialize(recipe: dict[str, object], cache_root: Path) -> tuple[Path, Path]:
+        folder = cache_root / "vibe-py" / str(recipe["name"])
+        folder.mkdir(parents=True)
+        (folder / "vuln.py").write_text("from flask import request\n\ndef f():\n    return eval(request.args.get('x'))\n")
+        (folder / "fixed.py").write_text("def f():\n    return 1\n")
+        return folder / "vuln.py", folder / "fixed.py"
+
+    monkeypatch.setattr(pairs_mod, "_materialize_pointer", fake_materialize)
+    result = pairs_mod.evaluate_catalog([case])
+    assert len(result.outcomes) == 1 and result.outcomes[0].name == "ptr" and result.outcomes[0].detected_vuln
+    assert not any(item.get("reason", "").startswith("pointer_pair") for item in result.degradations)
+    # a second evaluation reuses the cache: no fetch
+    monkeypatch.setattr(pairs_mod, "_materialize_pointer", lambda *_: pytest.fail("cache hit expected"))
+    assert pairs_mod.evaluate_catalog([case]).outcomes[0].pair_correct
+    # a failing fetch is a recorded loss, never an exception
+    (tmp_path / "c2.toml").write_text(_POINTER_ROW.replace('name = "ptr"', 'name = "ptr2"'))
+    (case2,) = pairs_mod._load_catalog_file(tmp_path / "c2.toml")
+
+    def boom(recipe: dict[str, object], cache_root: Path) -> tuple[Path, Path]:
+        raise OSError("offline")
+
+    monkeypatch.setattr(pairs_mod, "_materialize_pointer", boom)
+    failed = pairs_mod.evaluate_catalog([case2])
+    assert failed.outcomes == () and any(
+        item["reason"] == "pointer_pair_fetch_failed" and item["pair"] == "ptr2" for item in failed.degradations
+    )
+
+
+def test_cli_pairs_pointers_flag_turns_network_on_for_one_run(monkeypatch: pytest.MonkeyPatch) -> None:
+    import io
+    from contextlib import redirect_stdout
+
+    monkeypatch.delenv("OPENULTRASAST_PAIRS_NETWORK", raising=False)
+    buf = io.StringIO()
+    with redirect_stdout(buf):
+        assert main(["pairs", "--slice", "local", "--pointers", "--json"]) == 0
+    assert json.loads(buf.getvalue())["overall"]["pairs"] == 3
+    assert "OPENULTRASAST_PAIRS_NETWORK" not in __import__("os").environ  # the flag is per run, not a persistent env change
+
+
+def test_warm_cache_never_scores_pointer_pairs_when_network_is_off(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    """Req 10.3: the cache only spares the fetch under --pointers; it never turns a default run into a pointer run."""
+    from openultrasast.pairs import _load_catalog_file, evaluate_catalog
+
+    monkeypatch.setenv("OPENULTRASAST_PAIR_CACHE", str(tmp_path / "cache"))
+    monkeypatch.delenv("OPENULTRASAST_PAIRS_NETWORK", raising=False)
+    (tmp_path / "c.toml").write_text(_POINTER_ROW)
+    (case,) = _load_catalog_file(tmp_path / "c.toml")
+    case.vuln_file.parent.mkdir(parents=True)
+    case.vuln_file.write_text("from flask import request\n\ndef f():\n    return eval(request.args.get('x'))\n")
+    case.fixed_file.write_text("def f():\n    return 1\n")
+    result = evaluate_catalog([case])
+    assert result.outcomes == ()
+    assert {"stage": "pairs", "reason": "pointer_pair_skipped", "pair": "ptr"} in result.degradations
+    assert evaluate_catalog([case], pointers=True).outcomes[0].pair_correct  # warm cache, network on: no fetch, scored

tests/test_pair_harvest.py

@@ -363,3 +363,64 @@ def test_catalog_gen_fails_loud_without_the_package_and_refuses_to_prune_an_empt
     monkeypatch.setitem(sys.modules, "openultrasast.semantic.functions", None)
     with pytest.raises((SystemExit, ImportError)):
         runpy.run_path(str(Path("benchmarks/pairs/catalog_gen.py")), run_name="catalog_gen_lib_blocked")
+
+
+# --- task 8.3: pointer pairs (Req 10) -----------------------------------------
+
+
+def test_materialize_pointer_writes_only_into_the_cache_root_and_needs_no_license(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    lib = _lib()
+    recipe = {
+        "name": "ptr",
+        "repo": "o/r",
+        "parent": "aaa",
+        "commit": "bbb",
+        "path": "app.py",
+        "mode": "enclosing",
+        "line": 3,
+        "license": "",
+    }
+    with pytest.raises(lib["RecipeError"], match="license"):
+        lib["validate_recipe"](recipe)
+    lib["validate_recipe"](recipe, require_license=False)
+    sources = {"aaa": "import os\n\ndef f(x):\n    return eval(x)\n", "bbb": "import os\n\ndef f(x):\n    return int(x)\n"}
+    fetched: list[str] = []
+
+    def fake_fetch(url: str, timeout: int = 60) -> str:
+        fetched.append(url)
+        return sources[url.split("/")[5]]
+
+    lib["fetch_url"] = fake_fetch
+    lib["harvest_recipe"].__globals__["fetch_url"] = fake_fetch
+    vuln, fixed = lib["materialize_pointer"](recipe, tmp_path / "cache", slice_name="vibe-py")
+    assert vuln == tmp_path / "cache" / "vibe-py" / "ptr" / "vuln.py" and fixed.is_file()
+    assert "def f(x):" in vuln.read_text() and "upstream_start" in vuln.read_text()
+    assert len(fetched) == 2 and not list(Path("benchmarks/pairs/vibe-py").glob("ptr*"))
+
+
+def test_catalog_gen_emits_pointer_rows_without_excerpts(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    import tomllib
+
+    gen = runpy.run_path(str(Path("benchmarks/pairs/catalog_gen.py")), run_name="catalog_gen_lib")
+    slice_root = tmp_path / "vibe-py"
+    slice_root.mkdir()
+    (slice_root / "recipes.toml").write_text(
+        '[[recipe]]\nname = "ptr"\nrepo = "o/r"\nparent = "aaa"\ncommit = "bbb"\npath = "app.py"\nfix_path = "trap.py"\n'
+        'mode = "enclosing"\n'
+        'line = 3\nfix_line = 9\nfunction = "f"\nlanguage = "python"\nrelpath = "app.py"\nlicense = "unlicensed"\ncwe = "CWE-95"\n'
+        'class = "code injection"\nmechanism = "source_reaches_sink"\nprovenance = "agent"\nvendored = false\nsplit = "holdout"\n'
+    )
+    gen["ROOT"] = tmp_path
+    gen["generate"].__globals__["ROOT"] = tmp_path
+    catalog, count = gen["generate"]("vibe-py")
+    assert count == 1
+    (row,) = tomllib.loads(catalog.read_text())["pair"]
+    assert row["vendored"] is False and "vuln" not in row and "fixed" not in row
+    assert row["parent"] == "aaa" and row["fix_path"] == "trap.py" and row["fix_line"] == 9 and row["mode"] == "enclosing"
+    assert row["expected"][0]["function"] == "f"
+    from openultrasast.pairs import _load_catalog_file
+
+    monkeypatch.setenv("OPENULTRASAST_PAIR_CACHE", str(tmp_path / "cache"))
+    (case,) = _load_catalog_file(catalog)
+    assert case.vendored is False and case.provenance == "agent"
+    assert not (slice_root / "training" / "manifest.jsonl").read_text().strip()  # pointer rows have no vendored files to train on