feat(learn): labels from ground truth with provenance (decision-engine task 4)

learn/sources.toml is the fail-closed source list: every read goes through one guard
that refuses an unlisted path or an [[excluded]] file name before opening it, and a
source whose evaluation record is missing cannot label. Population v3 is named only
as excluded, by file name, and is never opened (tested with a v3-named fixture and
open/read_text tracked).

learn/labels.py and `ousast learn labels` apply the design's rules: v2 declared
sites and v1 fix-range functions positive, the same function on the fixed pin
negative only when the fix changed it, introduce/repair delta units, benign control
pushes as conditional negatives; pairs vulnerable/fixed (unscorable and title tier
out, title as a sensitivity arm); dev-php recipe known functions; recorded
adjudications with their reasons. v2 rows are labels marked design_informed, as the
design says. Repository groups: normalised owner/name, merged by shared CVE/GHSA,
shared fix commits and [[alias]] rows.

assumed_benign (maintainer decision): ordinary commits of the population
repositories, >= 90 days from any fix, no later security commit touching their
files, weight 0.5, conditional, reported separately and never merged with verified
negatives. The security filter is multilingual (CJK, Korean, Russian, European and
other languages) plus CVE/GHSA/CWE references and dependency bumps; widened after
reading accepted pushes (validation, passwords, guards).

First snapshot, counts only (benchmarks/measurements/2026-09-30-decision-engine-
labels/labels.json): verified pin labels in positive/negative repository groups
injection 44/41, untrusted_destination 35/34, deserialization 20/20 (per-family),
output_encoding 15/15, path 15/14, access_control 12/12 (pooled), config_secrets 5/5,
memory 5/5 (insufficient). assumed_benign: 969 pushes in 27 groups from 36,000
commits scanned.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

src/openultrasast/cli.py

@@ -259,6 +259,16 @@ def _main(argv: list[str] | None) -> int:
     candidates.add_argument("--generator", choices=("ir", "ruleset"), default="ir", help="which pass enumerates the candidate sites")
     candidates.add_argument("--json", action="store_true")
 
+    learn = subparsers.add_parser("learn", help="maintainer: the learned decision engine's data (labels)")
+    learn_sub = learn.add_subparsers(dest="learn_command", required=True)
+    learn_labels = learn_sub.add_parser("labels", help="labels from the fail-closed sources in learn/sources.toml (host only)")
+    learn_labels.add_argument("--sources", type=Path, help="the source list (default: the packaged learn/sources.toml)")
+    learn_labels.add_argument("--root", type=Path, default=Path("."), help="the repository root the source paths are relative to")
+    learn_labels.add_argument("--cache", type=Path, help="clones and checkouts (default ~/.cache/openultrasast)")
+    learn_labels.add_argument("--out", type=Path, help="write the full rows to OUT/labels-<sha>.jsonl (local: they name functions)")
+    learn_labels.add_argument("--snapshot", type=Path, help="write the counts-only snapshot (the committable record) here")
+    learn_labels.add_argument("--include-title", action="store_true", help="the title-tier pairs as well (a sensitivity arm)")
+    learn_labels.add_argument("--no-assumed-benign", action="store_true", help="skip mining the assumed-benign history source")
     plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on this host's kind cluster")
     plane_sub = plane.add_subparsers(dest="plane_command", required=True)
     plane_run = plane_sub.add_parser("run", help="execute a Run manifest on ax; a rerun skips done tasks")
@@ -429,6 +439,8 @@ def _main(argv: list[str] | None) -> int:
         return _model_candidates(args)
     if args.command == "plane":
         return _plane(args)
+    if args.command == "learn":
+        return _learn_labels(args)
     if args.command == "mcp":
         from .mcp import serve  # lazy: keeps the import cycle (mcp -> cli) one-directional
 
@@ -1315,6 +1327,33 @@ def _load_scan_findings(run_dir: Path) -> list[StaticFinding]:
     return load_findings(run_dir / "findings.json")
 
 
+def _learn_labels(args: argparse.Namespace) -> int:
+    """``ousast learn labels``: exit 2 on a source outside the list, an excluded file or an unreadable clone."""
+    from .learn.labels import DEFAULT_CACHE, DEFAULT_SOURCES, LabelSourceError, build_labels, write_labels
+
+    try:
+        build, record = build_labels(
+            args.root, sources_path=args.sources or DEFAULT_SOURCES, cache=args.cache or DEFAULT_CACHE, include_title=args.include_title,
+            assumed_benign=not args.no_assumed_benign,
+        )  # fmt: skip
+    except LabelSourceError as exc:
+        print(f"learn labels: {exc}", file=sys.stderr)
+        return 2
+    if args.out is not None:
+        digest = write_labels(build, args.out / "labels.jsonl")
+        (args.out / "labels.jsonl").rename(args.out / f"labels-{digest[:16]}.jsonl")
+        record["labels_sha256"] = digest
+    text = json.dumps(record, indent=2, sort_keys=True) + "\n"
+    if args.snapshot is not None:
+        args.snapshot.parent.mkdir(parents=True, exist_ok=True)
+        args.snapshot.write_text(text, encoding="utf-8")
+    pins = record["verified_pin_labels"]["by_family"]
+    print(f"labels: {record['rows']} rows in {record['groups']} repository groups; per family (positive/negative groups, model):")
+    for family, entry in pins.items():
+        print(f"  {family}: {entry['positive_groups']}/{entry['negative_groups']} {entry['model']}")
+    return 0
+
+
 def _plane(args: argparse.Namespace) -> int:
     from .plane import reconciler  # lazy: the plane is a maintainer surface, not the scan path