feat(learn): the classifier program and its response cache (decision-engine 6.4)

learn/program.py: the declared signature (code, signals, roles, family in;
verdict, family, confidence, rationale with cited lines out), the
baseline instruction, rendering as [system: instruction + demonstrations]
-- byte-identical for every candidate, so the provider's prefix cache can
hit -- then [user: retrieved examples with their labels, the candidate
without one]. Signals pass validate_x first (no EXCLUDED_FIELDS, missing
instruments in words). Answers are one JSON object; one that does not
parse or cites no line inside the excerpt is asked once more at
temperature 0 and otherwise becomes unsure/parse_failed, never
not_vulnerable. Classify(k): sample 1 at 0.0, the rest at 0.7; score s =
mean p_j; majority verdict, ties unsure; a majority unsure cannot BLOCK.
Caller caches each response as responses/<sha>.json keyed by (model,
Model parameters digest, messages, temperature, sample, json_object) and
replays at $0; spend is priced from the recorded usage once per distinct
request, so a replay reports the same usd. Program.decide asserts the
evaluation boundary on the rendered ids before any call.

MeteredClient.complete passes temperature/logprobs through (only when
set); DeepSeekChatClient sends temperature beside thinking=disabled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-01 10:13:31 +02:00
co-authored by Claude Opus 5.5
parent f5180a14ec
commit ea61dc1bb3
8 changed files with 764 additions and 13 deletions
+1 -1
View File
@@ -61,7 +61,7 @@ a model or embedding call.
- Tests: `test_learn_retrieve.py` including `test_boundary` (every rendered example and demonstration eligible,
for all fold kinds; `memory.seed` excludes the case's group), `test_learn_folds.py`.
- _Requirements: 3.2, 3.4, 7.1, 8.3_
- [ ] 6.4 The program (no model in tests)
- [x] 6.4 The program (no model in tests)
- `learn/program.py`: the signature, prompt rendering (instruction + demonstrations as a byte-identical prefix,
then retrieved examples, then the candidate), JSON parsing with one retry -> `unsure, parse_failed`,
`Classify(k)` with temperature 0 then 0.7, the score `s` and majority verdict; `responses/<sha>.json` cache