feat(harnessx): configurable provider + per-task budget caps

Add a [harnessx] config block so the agentic plane can be steered from
openultrasast.toml, and make the LLM provider selectable (the hunter and verifier
previously hardcoded AnthropicProvider).

- config.HarnessxConfig: provider ("anthropic"|"openai"|"litellm", default
  anthropic), max_cost_usd (2.0), token_threshold (120000); parsed from [harnessx].
- harness_ext.build_provider: single lazy, capability-guarded provider factory.
  The provider name is validated before any import, so an unknown choice fails with
  a clear ValueError even when the extra is absent; each provider reads its own
  standard key env var (openai -> OPENAI_API_KEY, anthropic -> ANTHROPIC_API_KEY).
- HxScanOrchestrator and verify_findings_dispatch take a provider and route through
  build_provider; cli threads config.harnessx.{provider,max_cost_usd,token_threshold}
  into both the hunter and the verifier.
- README: "Configuring the HarnessX agentic plane" section (install, models,
  provider, budgets, fallback/degradation).

Test isolation: convert the "harnessx not in sys.modules" import-guards to a
hermetic subprocess fixture (tests/conftest.py assert_cold_of_harnessx) and
simulate extra-absent in the CLI dispatch tests, so the suite is deterministic
whether or not the optional extra is installed (it now is, locally). Verified the
OpenAI path constructs end-to-end (build_provider -> OpenAIProvider; orchestrator
-> Harness) with the budget threaded through.

137 passed, 3 skipped; ruff + mypy clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-06-22 17:59:44 +02:00
co-authored by Claude Opus 4.8
parent 760df7c809
commit e255eb264f
12 changed files with 240 additions and 40 deletions
+22
View File
@@ -66,6 +66,17 @@ class RulesetConfig:
min_emit_precision: float = 0.0
@dataclass(frozen=True)
class HarnessxConfig:
# Optional agentic plane (the openultrasast[harnessx] extra). `provider`
# selects the HarnessX LLM provider used by both the hunter pool and the
# llm-judge verifier; the budget caps bound per-task cost and token spend.
# Inert unless the extra is installed and a model is configured in [models].
provider: str = "anthropic"
max_cost_usd: float = 2.0
token_threshold: int = 120_000
@dataclass(frozen=True)
class ResolvedConfig:
models: ModelConfig = ModelConfig()
@@ -76,6 +87,7 @@ class ResolvedConfig:
static_analysis: StaticAnalysisConfig = StaticAnalysisConfig()
score: ScoreConfig = ScoreConfig()
ruleset: RulesetConfig = RulesetConfig()
harnessx: HarnessxConfig = HarnessxConfig()
runs_dir: str = ".openultrasast/runs"
@@ -94,6 +106,7 @@ def load_config(config_path: Path | None = None) -> ResolvedConfig:
static_analysis=_load_static_analysis(data.get("static_analysis", {})),
score=_load_score(data.get("score", {})),
ruleset=_load_ruleset_config(data.get("ruleset", {})),
harnessx=_load_harnessx(data.get("harnessx", {})),
runs_dir=os.environ.get("OPENULTRASAST_RUNS_DIR", ".openultrasast/runs"),
)
@@ -188,6 +201,15 @@ def _load_ruleset_config(value: object) -> RulesetConfig:
)
def _load_harnessx(value: object) -> HarnessxConfig:
data = _section(value)
return HarnessxConfig(
provider=_string(data.get("provider")) or "anthropic",
max_cost_usd=_float_value(data.get("max_cost_usd"), 2.0),
token_threshold=_int_value(data.get("token_threshold"), 120_000),
)
def _float_value(value: object, default: float) -> float:
if isinstance(value, bool):
return default