bench(learn): injection first slice -- compiled AI classifier, out-of-repo evaluation (decision-engine 6.2, 6.6)

6.2: the 724 static examples' 616 distinct excerpts embedded with openai/text-embedding-3-small via
OpenRouter, 198,819 tokens, $0.0040 metered (ceiling $1).

6.6, family injection only, memory = every family (at most 2 contrast examples per prompt), input
profile v1: smoke fixed k = 5 (repeat prefix-cache hit share 0.96); one compile (program c1237a49,
proposed instruction 5, balanced Brier 0.128 on 13 C_val candidates vs 0.199 baseline); outer grouped
5-fold, 110 out-of-repository candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision
0.54 [0.44, 0.65], AUC 0.74; BLOCK not offered (ECE 0.064 > 0.05; no fold reached a Wilson lower bound
of 0.95); canary re-ask agreement 0.83. $0.91 metered at list price; DeepSeek balance $18.20 -> $17.90.

Instrument fixes found on the way, each with a test:
- memory build: pair sides join the record stored under their harvest unit pin (--units), and only the
  built profile's records are read (static and plane shared a key; 85 -> 724 examples);
- retrieval: stage 1 per side (family / contrast) and at most 2 contrast examples, never padded;
- program: the JSON answer contract is part of the signature, appended to proposed instructions
  (DeepSeek JSON mode refused a prompt without "json");
- compile: candidate_families for a one-family compile; compile.toml k = 5 from the smoke rule.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

.kiro/specs/learned-decision-engine/design.md

@@ -529,7 +529,8 @@ explicit rather than imputed:
   block). Prior features (`FeatureSpec.prior`) are dropped when `priors = off`. Weights are a compile-time
   setting only if an A/B shows a gain; the default is uniform.
 - Candidates in the eligible pool (after the boundary filter) are ranked by `D`, ties by `sha256(example_id +
-  seed)`; the first `n1 = 40` go to stage 2.
+  seed)`; the first `n1 = 40` of the candidate's family and the first `n1` of the other families go to stage 2
+  (per side since 2026-10-01, so a minority family reaches stage 2; Risks, "First paid slice").
 
 **Stage 2: code-embedding re-rank.** Cosine similarity of the candidate's excerpt embedding with each stage-1
 example's. The re-rank score is `r = lam * (1 - D) + (1 - lam) * cos`, with `lam` a compiled setting in {0, 0.5}
@@ -1205,6 +1206,27 @@ start only after the maintainer confirms their budget against the balance read j
   Pre-registering that and recording "inconclusive" is the intended behaviour. The failure mode to avoid is
   extending an experiment after looking.
 
+- **First paid slice, 2026-10-01 (tasks 6.2, 6.6; injection only).** Four instrument defects surfaced before any
+  number was believed, each fixed with a test:
+  - *The memory build joined no pair.* The harvest stores a pair side's record under its unit pin (the blob sha1 of
+    the side's excerpt), not under `label_pin`; `ousast learn memory build --units <harvest units.json>` joins them.
+    And a candidate's `static` and `plane` records share a key, so reading both let one overwrite the other and
+    silently drop about half the examples; the build now reads its own profile only. 85 -> 724 examples.
+  - *Contrast was unbounded.* The balance rule padded a short pick with other families (up to 5 of 6), and stage 1
+    took the 40 nearest of all families, so a minority family often never reached stage 2. Now stage 1 takes the
+    `n1` nearest of the candidate's family and the `n1` nearest others, and at most 2 contrast examples are shown
+    whenever the pool holds the family (maintainer: "allow 2 contrast"); never padded.
+  - *Proposed instructions dropped the answer contract*, and DeepSeek's JSON mode refuses a prompt without the word
+    "json" (HTTP 400, the first compile stopped there). The contract (`ANSWER_FORMAT`) is now part of the signature
+    and appended to any instruction that lacks it; the baseline prefix is byte-identical.
+  - The adjudication source fails the 10% excerpt gate: 9 of its 43 candidates are anonymous functions (`<lambda>N`,
+    `<module>`) that `function_span` cannot find by name; the clones were read. Open.
+  The memory is small: 724 static examples (only harvested candidates have feature records; the 97 advisory-fix pairs
+  have none yet), injection 140 (75/65), so `C_val` had 13 injection candidates and the compile metric is noisy.
+  The measured ranking signal is real but weak at the recall target: AUC 0.74 out of repository, ADVISORY at recall
+  0.90 has precision 0.54 (prevalence 0.53); the majority `vulnerable` verdict alone has precision 0.86, recall 0.43.
+  Canary re-ask agreement at temperature 0 was 0.83, below the 0.9 the adoption check requires.
+
 ## Maintainer decisions at design approval (2026-09-30)
 
 - Design approved as written.

.kiro/specs/learned-decision-engine/tasks.md

@@ -54,6 +54,8 @@ a model or embedding call.
       `plane/budget.py`, usage kept by `parse_embedding_response`; cache `embeddings/<model>/<excerpt_sha>.json`;
       `ousast learn memory embed`; scripted-client tests.
     - **(budget go-ahead)** embedding the store: ~5.0 M tokens, ~$0.10, ceiling $1 (OpenRouter account).
+      Done 2026-10-01: 616 distinct excerpts of the 724 static examples, 198,819 tokens, 20 calls, $0.0040 metered
+      (`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`).
     - _Requirements: 3.2_
   - [x] 6.3 Retrieval and the evaluation boundary (no model)
     - `learn/retrieve.py`: per-instrument Gower distance with explicit missing, `n1 = 40`, embedding re-rank
@@ -82,13 +84,21 @@ a model or embedding call.
     - Tests: `test_learn_compile.py` (`test_reproducible_from_cache` on both backends), `test_learn_calibrate.py`,
       `test_learn_decide.py`.
     - _Requirements: 3.3, 3.5, 5.2, 7.3, 7.4_
-  - [ ] 6.6 First paid slice **(budget go-ahead: ~$5 expected, ceiling $7)**
+  - [x] 6.6 First paid slice **(budget go-ahead: ~$5 expected, ceiling $7)**
     - Smoke run on 20 candidates (~$0.10): real token counts, prefix-cache hit share, parse rate; it fixes `k`
       (5 if samples 2..k hit the cache for >= 80% of their prompt tokens, else 1) before anything else is spent.
     - One compile per profile with data (`static` first; ~$2.2, ceiling $3).
     - Outer-fold evaluation, priors off, evaluable families only (675 candidates: ~$2.0 at k = 1, ~$3.0 at k = 5),
       with the 30-candidate canary set; report under `benchmarks/measurements/<date>-decision-engine-v1/` (counts
       and intervals only, no identities).
+    - Done 2026-10-01 for family **injection only** (the maintainer's scope for this slice; memory holds every family,
+      at most 2 contrast examples per prompt): smoke fixed `k = 5` (repeat hit share 0.96); program `c1237a49...`
+      (proposed instruction 5, balanced Brier 0.128 on 13 `C_val` candidates vs 0.199 baseline); 110 out-of-repository
+      candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision 0.54 [0.44, 0.65]; AUC of `s` 0.74 [0.65,
+      0.85]; BLOCK not offered (ECE 0.064 > 0.05, and no fold reached a Wilson lower bound of 0.95); canary agreement
+      0.83; $0.91 metered at list price (provider balance $18.20 -> $17.90). Record:
+      `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`. The other evaluable families
+      are not evaluated yet.
     - _Requirements: 3.1-3.5, 5.2, 7.3, 7.4_
   - [ ] 6.7 Second paid slice **(budget go-ahead: ~$7 expected, ceiling $10; needs a top-up)**
     - Leave-one-source-out (~$1.4-2.1), leave-one-framework-out where a framework reaches 10 groups ($0 today),