marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-18 12:39:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(model): the chat model is deepseek-flash; the hunter reaches DeepSeek

Browse Source 
Every model call has failed since 2026-09-10 (83ab533):
- The default detector and judge model was `deepseek-v4.1-flash`, the product name.
  The API rejects it with HTTP 400 ("The supported API model names are
  deepseek-flash, deepseek-v4-pro"). The judge and pipeline record an endpoint
  failure as a degradation, never a verdict, so nothing failed loudly.
- `tool_hunter.resolve_hunter_client` resolved OpenRouter only. In this project
  DeepSeek is the chat provider and the OpenRouter key is scoped to embeddings, so
  the hunter could not reach a chat model at all; the call came back 401.

The defaults are now `deepseek-flash` (priced). The hunter resolves DeepSeek first
when its key is set, and OpenRouter stays as the explicit or fallback route.
Verified live: plain completion and a tool call through both the endpoint resolver
and the hunter client.

Not affected: the independent evaluations (finding_dump runs with
max_model_calls=0) and the 2026-09-06 hunter measurement, which predates the id.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-28 09:01:14 +02:00
co-authored by Claude Opus 5.5
parent 5835239808
commit cf579956d5
3 changed files with 42 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/openultrasast/model/endpoint.py
+7 -2
View File
@@ -29,8 +29,12 @@ OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"
# 12:00 Beijing time on 2026-09-14, billed at Flash's price. So the judge tier is no longer a more expensive
# model -- detector and judge are the same model, and any claim in this repo that the judge is a *stronger*
# second opinion is now a claim about prompt and evidence, not about model class.
DEFAULT_DETECTOR_MODEL = "deepseek-v4.1-flash"
DEFAULT_JUDGE_MODEL = "deepseek-v4.1-flash"
# The API name is `deepseek-flash`, not the product name. From 2026-09-10 to 2026-09-28 this default was
# `deepseek-v4.1-flash`, which the platform rejects with HTTP 400 ("The supported API model names are
# deepseek-flash, deepseek-v4-pro") -- every model call in that period failed and was recorded only as a
# degradation. The id is what `GET /models` lists; `deepseek-chat` is an alias the platform maps to it.
DEFAULT_DETECTOR_MODEL = "deepseek-flash"
DEFAULT_JUDGE_MODEL = "deepseek-flash"
Provider = Literal["deepseek", "openrouter", "custom", "scripted"]
_SCRIPTED_FLAGS = frozenset({"scripted", "script", "dump", "dump-only", "unsafe", "unsafe-snippet"})
@@ -50,6 +54,7 @@ class Prices:
_PRICES: dict[str, Prices] = {
# V4.1 Flash is priced as Flash; Pro requests are routed here and billed at this rate from 2026-09-14.
"deepseek-flash": Prices(cache_hit_per_m=0.014, input_per_m=0.44, output_per_m=1.32),
"deepseek-v4.1-flash": Prices(cache_hit_per_m=0.014, input_per_m=0.44, output_per_m=1.32),
"deepseek-v4-flash": Prices(cache_hit_per_m=0.014, input_per_m=0.44, output_per_m=1.32),
"deepseek-v4-flash-vision-exp": Prices(cache_hit_per_m=0.014, input_per_m=0.44, output_per_m=1.32),