bench(learn): injection first slice -- compiled AI classifier, out-of-repo evaluation (decision-engine 6.2, 6.6)

6.2: the 724 static examples' 616 distinct excerpts embedded with openai/text-embedding-3-small via
OpenRouter, 198,819 tokens, $0.0040 metered (ceiling $1).

6.6, family injection only, memory = every family (at most 2 contrast examples per prompt), input
profile v1: smoke fixed k = 5 (repeat prefix-cache hit share 0.96); one compile (program c1237a49,
proposed instruction 5, balanced Brier 0.128 on 13 C_val candidates vs 0.199 baseline); outer grouped
5-fold, 110 out-of-repository candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision
0.54 [0.44, 0.65], AUC 0.74; BLOCK not offered (ECE 0.064 > 0.05; no fold reached a Wilson lower bound
of 0.95); canary re-ask agreement 0.83. $0.91 metered at list price; DeepSeek balance $18.20 -> $17.90.

Instrument fixes found on the way, each with a test:
- memory build: pair sides join the record stored under their harvest unit pin (--units), and only the
  built profile's records are read (static and plane shared a key; 85 -> 724 examples);
- retrieval: stage 1 per side (family / contrast) and at most 2 contrast examples, never padded;
- program: the JSON answer contract is part of the signature, appended to proposed instructions
  (DeepSeek JSON mode refused a prompt without "json");
- compile: candidate_families for a one-family compile; compile.toml k = 5 from the smoke rule.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

src/openultrasast/learn/program.py

@@ -87,16 +87,21 @@ SIGNATURE = Signature(
     )
 )
 
-BASELINE_INSTRUCTION = (
-    "You decide whether a function contains an exploitable security vulnerability of the given family.\n"
-    "You see the function's code (numbered lines), the signals static instruments emitted for it, and the roles\n"
-    "inferred for its calls. Signals are evidence, not verdicts: rules over-report, and a missing instrument is not a\n"
-    "negative. Similar labelled functions from other repositories follow as examples; reason about this code.\n"
+# The answer contract is part of the signature, not of the searched instruction: a proposed instruction that leaves it
+# out still gets it (``render_prefix``), so every prompt asks for JSON (DeepSeek's JSON mode refuses a prompt that
+# never says "json") and every answer can parse.
+ANSWER_FORMAT = (
     'Answer with one JSON object: {"verdict": "vulnerable" | "not_vulnerable" | "unsure", "family": <family or\n'
     '"none">, "confidence": <probability in [0, 1] that your verdict is right>, "rationale": <at most 3 sentences>,\n'
     '"cited_lines": [<line numbers of the code that support the verdict>]}. Cite at least one line. Say "unsure"\n'
     "when the code does not decide it."
 )
+BASELINE_INSTRUCTION = (
+    "You decide whether a function contains an exploitable security vulnerability of the given family.\n"
+    "You see the function's code (numbered lines), the signals static instruments emitted for it, and the roles\n"
+    "inferred for its calls. Signals are evidence, not verdicts: rules over-report, and a missing instrument is not a\n"
+    "negative. Similar labelled functions from other repositories follow as examples; reason about this code.\n" + ANSWER_FORMAT
+)
 
 
 # --- rendering ---------------------------------------------------------------------------------------------------------
@@ -232,7 +237,8 @@ ExcerptText = Callable[[str], str | None]
 
 def render_prefix(spec: ProgramSpec, demos: Sequence[Demo], examples: Mapping[str, Example], excerpt_text: ExcerptText) -> str:
     """The instruction and the demonstrations: identical for every candidate of one program and fold."""
-    parts = [spec.instruction.strip()]
+    instruction = spec.instruction.strip()
+    parts = [instruction if ANSWER_FORMAT in instruction else f"{instruction}\n{ANSWER_FORMAT}"]
     for number, demo in enumerate(demos, 1):
         example = examples[demo.example_id]
         code = excerpt_text(demo.excerpt_sha) or ""
@@ -481,7 +487,8 @@ class Program:
 
 
 __all__ = [
-    "BASELINE_INSTRUCTION", "PARSE_FAILED", "SIGNATURE", "UNSURE", "VERDICTS", "Answer", "Caller", "Candidate", "Decision", "Demo",
+    "ANSWER_FORMAT", "BASELINE_INSTRUCTION", "PARSE_FAILED", "SIGNATURE", "UNSURE", "VERDICTS", "Answer", "Caller", "Candidate",
+    "Decision", "Demo",
     "Program", "ProgramSpec", "Prompt", "ReplayMiss", "aggregate", "classify", "label_word", "line_numbers",
     "params_digest", "parse_answer", "render", "render_prefix", "render_roles", "render_signals", "request_key",
 ]  # fmt: skip