feat(learn): the classifier program and its response cache (decision-engine 6.4)

learn/program.py: the declared signature (code, signals, roles, family in;
verdict, family, confidence, rationale with cited lines out), the
baseline instruction, rendering as [system: instruction + demonstrations]
-- byte-identical for every candidate, so the provider's prefix cache can
hit -- then [user: retrieved examples with their labels, the candidate
without one]. Signals pass validate_x first (no EXCLUDED_FIELDS, missing
instruments in words). Answers are one JSON object; one that does not
parse or cites no line inside the excerpt is asked once more at
temperature 0 and otherwise becomes unsure/parse_failed, never
not_vulnerable. Classify(k): sample 1 at 0.0, the rest at 0.7; score s =
mean p_j; majority verdict, ties unsure; a majority unsure cannot BLOCK.
Caller caches each response as responses/<sha>.json keyed by (model,
Model parameters digest, messages, temperature, sample, json_object) and
replays at $0; spend is priced from the recorded usage once per distinct
request, so a replay reports the same usd. Program.decide asserts the
evaluation boundary on the rendered ids before any call.

MeteredClient.complete passes temperature/logprobs through (only when
set); DeepSeekChatClient sends temperature beside thinking=disabled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-01 10:13:31 +02:00
co-authored by Claude Opus 5.5
parent f5180a14ec
commit ea61dc1bb3
8 changed files with 764 additions and 13 deletions
+10 -1
View File
@@ -121,11 +121,20 @@ class MeteredClient:
tools: list[dict[str, object]],
timeout_seconds: int = 60,
json_object: bool = False,
temperature: float | None = None,
logprobs: bool = False,
) -> ChatResponse:
"""One metered call. ``temperature`` and ``logprobs`` pass through only when set, so a wrapped client that
does not take them (a scripted one) is called as before."""
self.check()
extra: dict[str, Any] = {}
if temperature is not None:
extra["temperature"] = temperature
if logprobs:
extra["logprobs"] = True
try:
response = self.inner.complete(
model=model, messages=messages, tools=tools, timeout_seconds=timeout_seconds, json_object=json_object
model=model, messages=messages, tools=tools, timeout_seconds=timeout_seconds, json_object=json_object, **extra
)
except Exception as exc:
self._absorb_rows()