mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 03:59:28 +00:00
fix(learning-harness): say why the proposer had nothing, and stop hard-coding its vendor
The meta-agent round journalled `no_proposal`. The real cause was an authentication failure: `_model_config` hard-coded the anthropic provider while the detector was talking to DeepSeek and no ANTHROPIC_API_KEY exists here. The composition itself was fine -- MetaAgent constructed, evolve started, budget $5 -- and the trace was thrown away. `no_proposal` covered two different worlds: a proposer that considered the facts and declined, and one that could not start at all. The journal now carries the proposer's own reason, and the vendor comes from `[harnessx] provider` instead of being assumed. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
84b1ba7a72
commit
cde57e4293
@@ -373,3 +373,36 @@ def test_round_zero_refuses_fewer_than_three_runs(tmp_path: Path) -> None:
|
||||
out_dir=tmp_path / "out",
|
||||
k_runs=1,
|
||||
)
|
||||
|
||||
|
||||
def test_a_round_journals_why_the_proposer_had_nothing_to_say(world) -> None: # type: ignore[no-untyped-def]
|
||||
"""`no_proposal` covers two different worlds: a proposer with nothing to add, and one that could not start.
|
||||
|
||||
Measured: the meta-agent round journalled `no_proposal` while the real cause was an authentication failure in a
|
||||
provider seam pointing at the wrong vendor. A round that failed for a fixable configuration reason must not
|
||||
look identical to a model that considered the facts and declined."""
|
||||
|
||||
class Mute:
|
||||
reason = "meta_agent_failed: RuntimeError"
|
||||
|
||||
def propose(self, facts): # type: ignore[no-untyped-def]
|
||||
del facts
|
||||
return None
|
||||
|
||||
from openultrasast.learning.rounds import run_learning_round
|
||||
|
||||
record = run_learning_round(
|
||||
world["cases"],
|
||||
family="injection",
|
||||
taxonomy=load_families(),
|
||||
configs_dir=world["configs"],
|
||||
proposer=Mute(),
|
||||
scan_factory=_injection_improves,
|
||||
model="m",
|
||||
journal=world["journal"],
|
||||
archive=world["archive"],
|
||||
floors=world["floors"],
|
||||
out_dir=world["out"],
|
||||
)
|
||||
assert record.outcome == "rejected"
|
||||
assert record.reason == "no_proposal: meta_agent_failed: RuntimeError"
|
||||
|
||||
Reference in New Issue
Block a user