feat(harnessx): configurable provider + per-task budget caps

Add a [harnessx] config block so the agentic plane can be steered from
openultrasast.toml, and make the LLM provider selectable (the hunter and verifier
previously hardcoded AnthropicProvider).

- config.HarnessxConfig: provider ("anthropic"|"openai"|"litellm", default
  anthropic), max_cost_usd (2.0), token_threshold (120000); parsed from [harnessx].
- harness_ext.build_provider: single lazy, capability-guarded provider factory.
  The provider name is validated before any import, so an unknown choice fails with
  a clear ValueError even when the extra is absent; each provider reads its own
  standard key env var (openai -> OPENAI_API_KEY, anthropic -> ANTHROPIC_API_KEY).
- HxScanOrchestrator and verify_findings_dispatch take a provider and route through
  build_provider; cli threads config.harnessx.{provider,max_cost_usd,token_threshold}
  into both the hunter and the verifier.
- README: "Configuring the HarnessX agentic plane" section (install, models,
  provider, budgets, fallback/degradation).

Test isolation: convert the "harnessx not in sys.modules" import-guards to a
hermetic subprocess fixture (tests/conftest.py assert_cold_of_harnessx) and
simulate extra-absent in the CLI dispatch tests, so the suite is deterministic
whether or not the optional extra is installed (it now is, locally). Verified the
OpenAI path constructs end-to-end (build_provider -> OpenAIProvider; orchestrator
-> Harness) with the budget threaded through.

137 passed, 3 skipped; ruff + mypy clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-06-22 17:59:44 +02:00
co-authored by Claude Opus 4.8
parent 760df7c809
commit e255eb264f
12 changed files with 240 additions and 40 deletions
+31 -1
View File
@@ -13,13 +13,17 @@ from __future__ import annotations
import importlib
import importlib.util
from types import ModuleType
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from harnessx.core.harness import HarnessConfig
HARNESSX_EXTRA = "openultrasast[harnessx]"
# HarnessX providers OpenUltraSAST can drive for the hunter / llm-judge stages.
# Each reads its own standard API-key env var (ANTHROPIC_API_KEY / OPENAI_API_KEY).
SUPPORTED_PROVIDERS = ("anthropic", "openai", "litellm")
class HarnessXUnavailableError(RuntimeError):
"""Raised when a HarnessX-backed capability is requested but the extra is absent."""
@@ -37,6 +41,32 @@ def require_harnessx() -> ModuleType:
return importlib.import_module("harnessx")
def build_provider(model: str, provider: str = "anthropic") -> Any:
"""Build a HarnessX LLM provider for ``model`` (lazy, capability-guarded).
The single place that knows HarnessX provider class names. The provider name is
validated before any import so an unknown choice fails loudly even when the extra
is absent; each provider reads its own standard API-key env var. Raises
:class:`ValueError` on an unknown provider and :class:`HarnessXUnavailableError`
when the extra is not installed.
"""
key = provider.lower()
if key not in SUPPORTED_PROVIDERS:
raise ValueError(f"unknown HarnessX provider {provider!r}; choose one of {SUPPORTED_PROVIDERS}")
require_harnessx()
if key == "anthropic":
from harnessx.providers.anthropic_provider import AnthropicProvider
return AnthropicProvider(model=model)
if key == "openai":
from harnessx.providers.openai_provider import OpenAIProvider
return OpenAIProvider(model=model)
from harnessx.providers.litellm_provider import LiteLLMProvider
return LiteLLMProvider(model=model)
def build_sast(*, max_cost_usd: float = 2.0, token_threshold: int = 120_000) -> HarnessConfig:
"""Compose the SAST hunter HarnessConfig (behaviour pipeline, model-free).