feat(learn): compile, calibrate, evaluate (decision-engine 6.5)

learn/compile.toml: the pre-registered spec -- balanced Brier score of the
raw score on C_val (cells by family x label, repositories capped at 5,
unsure = 0.5, ties within 0.005 to fewer prompt tokens), seed, the
compile split (25%, boot 60/40, compile-split-only memory), bootstrap
(150 candidates, confidence >= 0.7, 4 sets of 3), search (6 proposals +
baseline, 20 errors shown, 60 validation candidates), k = 1, the
retrieval settings, and the $3 / 1,200-call ceiling.

learn/compile.py: bootstrap few-shot keeps only right, confident
rationales that cite a line and name no repository, path or CVE/GHSA id;
seeded demo sets (positive, negative, any; families mixed) with
other-framework alternates; one proposal call; 7 instructions scored with
set 1, the best with sets 2-4. The artifact is canonical JSON stored as
programs/<program_id>.json, program_id = sha256 of the artifact without
its id; it records the spec sha, memory/compile-groups/folds digests and
the response-cache digest, with spend priced from cached usage, so a
rerun from the cache is byte-identical at $0 (tested on FileStore and
MinioStore through the fake client; real MinIO gated as today). A ceiling
ends the compile `unfinished` with no artifact. packageable() keeps only
permissively licensed demonstrations (MIT, BSD, Apache-2.0, ISC, zlib),
swapping in a permissive alternate or dropping, under a new program id.

learn/calibrate.py: Platt scaling on logit(s), pure Python Newton, per
family / pooled with family intercepts / insufficient data by the floors;
cross-fitted (fold j's map is fitted on the other folds only); the check
(5-bin reliability, slope and intercept with repository-bootstrap CIs,
ECE <= 0.05), ADVISORY only when it fails.

learn/evaluate.py: predictions under each fold's boundary, families below
the floor never asked; operating points read from the calibrated,
cross-fitted predictions of the other folds (BLOCK: Wilson lower bound
>= 0.95, never on a majority unsure; ADVISORY: recall >= 0.90);
repository-cluster bootstrap intervals; strata with "insufficient data";
>5% parse failures is an instrument failure; explanations keep only
signals the record holds; version mismatch names the fallback reason;
the memory-size learning curve.

`ousast learn compile|evaluate|curve`: paid runs need --budget-usd (exit
3 at the ceiling), --replay-only runs from the cache. Nothing was spent:
every test and the dry run used scripted clients.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

src/openultrasast/learn/compile.toml

@@ -0,0 +1,44 @@
+# The pre-registered compile specification of the decision engine's program (learned-decision-engine design 4.3).
+# Committed before any compile runs: the metric, the seeds, the search space and the budget are fixed here, and the
+# compiled artifact records this file's sha256. Changing anything below after looking at a result is a new spec.
+version = 1
+seed = 0
+
+[metric]
+name = "balanced_brier_v1"   # Brier score of the raw score s on C_val, each (family, label) cell weighted equally
+repository_cap = 5           # each repository counts at most 5 effective rows per cell
+unsure_score = 0.5           # unsure and parse_failed answers score as s = 0.5
+tie_tolerance = 0.005        # within this, fewer prompt tokens wins, then the earlier configuration
+secondary = ["recall_at_precision_0.90", "unsure_share"]   # reported, never optimised
+
+[split]
+compile_fraction = 0.25      # of the repository groups, per primary family; never an evaluation fold
+boot_fraction = 0.6          # C_boot (demonstrations) vs C_val (scoring)
+retrieval = "compile_only"   # during the compile, memory is the compile split only: no evaluated group shapes it
+
+[bootstrap]
+max_candidates = 150
+min_confidence = 0.7
+demo_sets = 4
+demos_per_set = 3            # one positive, one negative, one of either; families mixed
+
+[search]
+proposals = 6                # instructions proposed in one call, scored with the hand-written baseline (7)
+errors_shown = 20            # summarised bootstrap errors in the proposal prompt (signals and verdicts, no code)
+val_candidates = 60
+
+[classify]
+model = "deepseek-flash"
+k = 1                        # k and lam are A/B variants (design section 6), not searched here
+max_output_tokens = 300
+
+[retrieval]
+n1 = 40
+k = 6
+lam = 0.5
+near_duplicate_cos = 0.98
+priors = "off"
+
+[budget]
+usd = 3.0                    # a compile that reaches it is `unfinished` and writes no artifact
+calls = 1200