feat(harnessx): configurable provider + per-task budget caps

Add a [harnessx] config block so the agentic plane can be steered from
openultrasast.toml, and make the LLM provider selectable (the hunter and verifier
previously hardcoded AnthropicProvider).

- config.HarnessxConfig: provider ("anthropic"|"openai"|"litellm", default
  anthropic), max_cost_usd (2.0), token_threshold (120000); parsed from [harnessx].
- harness_ext.build_provider: single lazy, capability-guarded provider factory.
  The provider name is validated before any import, so an unknown choice fails with
  a clear ValueError even when the extra is absent; each provider reads its own
  standard key env var (openai -> OPENAI_API_KEY, anthropic -> ANTHROPIC_API_KEY).
- HxScanOrchestrator and verify_findings_dispatch take a provider and route through
  build_provider; cli threads config.harnessx.{provider,max_cost_usd,token_threshold}
  into both the hunter and the verifier.
- README: "Configuring the HarnessX agentic plane" section (install, models,
  provider, budgets, fallback/degradation).

Test isolation: convert the "harnessx not in sys.modules" import-guards to a
hermetic subprocess fixture (tests/conftest.py assert_cold_of_harnessx) and
simulate extra-absent in the CLI dispatch tests, so the suite is deterministic
whether or not the optional extra is installed (it now is, locally). Verified the
OpenAI path constructs end-to-end (build_provider -> OpenAIProvider; orchestrator
-> Harness) with the budget threaded through.

137 passed, 3 skipped; ruff + mypy clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-06-22 17:59:44 +02:00
co-authored by Claude Opus 4.8
parent 760df7c809
commit e255eb264f
12 changed files with 240 additions and 40 deletions
+14 -5
View File
@@ -1,7 +1,5 @@
"""Verifier dispatcher: structural fallback (default) + judge verdict mapping (Phase 3 task 6.4)."""
import sys
from openultrasast.findings import StaticFinding
from openultrasast.verification import VerificationStatus, verify_findings
from openultrasast.verify_judge import _verdict_to_verification, verify_findings_dispatch
@@ -40,9 +38,20 @@ def test_dispatch_stays_structural_when_model_unset_or_extra_absent() -> None:
assert verify_findings_dispatch(findings, verifier_model="claude-x", use_harnessx=False) == verify_findings(findings)
def test_dispatch_does_not_import_harnessx_on_fallback() -> None:
verify_findings_dispatch([_finding()], verifier_model="claude-x", use_harnessx=True)
assert "harnessx" not in sys.modules
def test_dispatch_does_not_import_harnessx_on_fallback(assert_cold_of_harnessx) -> None: # type: ignore[no-untyped-def]
# Even with a model and use_harnessx=True, when the capability is absent the
# dispatch must fall back without importing the extra. Simulate absence so the
# guarantee holds whether or not the extra is installed in this environment.
assert_cold_of_harnessx(
"import openultrasast.verify_judge as vj\n"
"vj.has_harnessx = lambda: False\n"
"from openultrasast.findings import StaticFinding\n"
"f = StaticFinding(finding_id='x:app.py:1', path='app.py', title='t', severity='low',\n"
" confidence='low', evidence_level='static_corroboration', rationale='r', line=1,\n"
" function_name=None, reachability_status='unknown', reachability_evidence=[],\n"
" reachability_conditions=[], tags=[], ranking_priority=1.0)\n"
"vj.verify_findings_dispatch([f], verifier_model='claude-x', use_harnessx=True)\n"
)
def test_verdict_mapping_is_evidence_gated() -> None: