mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-19 21:19:28 +00:00
feat(learn): the classifier program and its response cache (decision-engine 6.4)
learn/program.py: the declared signature (code, signals, roles, family in; verdict, family, confidence, rationale with cited lines out), the baseline instruction, rendering as [system: instruction + demonstrations] -- byte-identical for every candidate, so the provider's prefix cache can hit -- then [user: retrieved examples with their labels, the candidate without one]. Signals pass validate_x first (no EXCLUDED_FIELDS, missing instruments in words). Answers are one JSON object; one that does not parse or cites no line inside the excerpt is asked once more at temperature 0 and otherwise becomes unsure/parse_failed, never not_vulnerable. Classify(k): sample 1 at 0.0, the rest at 0.7; score s = mean p_j; majority verdict, ties unsure; a majority unsure cannot BLOCK. Caller caches each response as responses/<sha>.json keyed by (model, Model parameters digest, messages, temperature, sample, json_object) and replays at $0; spend is priced from the recorded usage once per distinct request, so a replay reports the same usd. Program.decide asserts the evaluation boundary on the rendered ids before any call. MeteredClient.complete passes temperature/logprobs through (only when set); DeepSeekChatClient sends temperature beside thinking=disabled. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
f5180a14ec
commit
ea61dc1bb3
@@ -121,11 +121,20 @@ class MeteredClient:
|
||||
tools: list[dict[str, object]],
|
||||
timeout_seconds: int = 60,
|
||||
json_object: bool = False,
|
||||
temperature: float | None = None,
|
||||
logprobs: bool = False,
|
||||
) -> ChatResponse:
|
||||
"""One metered call. ``temperature`` and ``logprobs`` pass through only when set, so a wrapped client that
|
||||
does not take them (a scripted one) is called as before."""
|
||||
self.check()
|
||||
extra: dict[str, Any] = {}
|
||||
if temperature is not None:
|
||||
extra["temperature"] = temperature
|
||||
if logprobs:
|
||||
extra["logprobs"] = True
|
||||
try:
|
||||
response = self.inner.complete(
|
||||
model=model, messages=messages, tools=tools, timeout_seconds=timeout_seconds, json_object=json_object
|
||||
model=model, messages=messages, tools=tools, timeout_seconds=timeout_seconds, json_object=json_object, **extra
|
||||
)
|
||||
except Exception as exc:
|
||||
self._absorb_rows()
|
||||
|
||||
Reference in New Issue
Block a user