mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 20:49:28 +00:00
feat(harnessx): Phase 3 task 6 — hunter sub-harness + llm-judge verifier + fallback
Adopt HarnessX at the hunter loop and verifier, capability-gated so the default
zero-dependency scan is byte-identical and CI (no extra) exercises the fallback.
- 6.1/6.2 build_sast() (single, lazy HarnessX composition site in harness_ext.py,
mirrors DocuHarnessX make_docgen) + HxScanOrchestrator (hunter_harness.py): a
drop-in for run_hunter_pool that runs one bounded HarnessX agent task per ranked
target (ModelConfig(main=AnthropicProvider).agentic(build_sast()).run(BaseTask)),
parses findings from a JSON structured-output contract, returns the existing
HunterPoolResult, maps scan_id->session_id and emits per-target exit_reason/cost/
steps to the outer trace.
- 6.4 verify_judge.verify_findings_dispatch: a drop-in superset of
verification.verify_findings — structural verifier verbatim by default (rejects
unknown reachability), HarnessX llm-judge (build_judge_prompt/parse_judge_response)
when the extra + a verifier model are present; verdict mapped to VerificationResult
evidence-gated (plausible>=0.6 accepted; unsupported/refused rejected; infra
failure never auto-rejects).
- 6.5 cli capability gate (hx_hunter/hx_verify): selects HxScanOrchestrator /
judge only in standard mode with a configured model AND has_harnessx(); otherwise
falls back and records a {stage,reason:harnessx_extra_unavailable} degradation in
the manifest (new append-only write_manifest degradations kwarg). With no model
configured the manifest stays byte-identical.
- pyproject: mypy override ignore_missing_imports for harnessx.* (absent in the
type-check env).
Verified the HarnessX path constructs against the real extra in a throwaway venv
(build_sast -> HarnessConfig; orchestrator -> Harness; build_judge_prompt/
parse_judge_response/verdict mapping) — only the live agent.run()/provider.complete()
LLM calls are offline-untestable and stay gated. 109 tests pass (2 construction
smokes skip without the extra), ruff + mypy clean, detection gate 93.6%/0% intact.
Tasks 6.1/6.2/6.4/6.5 done; 6.3 (deterministic stages as HarnessX processors) deferred.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
5e206df166
commit
ef59c0ae3d
@@ -13,6 +13,10 @@ from __future__ import annotations
|
||||
import importlib
|
||||
import importlib.util
|
||||
from types import ModuleType
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from harnessx.core.harness import HarnessConfig
|
||||
|
||||
HARNESSX_EXTRA = "openultrasast[harnessx]"
|
||||
|
||||
@@ -31,3 +35,33 @@ def require_harnessx() -> ModuleType:
|
||||
if not has_harnessx():
|
||||
raise HarnessXUnavailableError(f"HarnessX is not installed. Install the optional agentic extra: pip install '{HARNESSX_EXTRA}'")
|
||||
return importlib.import_module("harnessx")
|
||||
|
||||
|
||||
def build_sast(*, max_cost_usd: float = 2.0, token_threshold: int = 120_000) -> HarnessConfig:
|
||||
"""Compose the SAST hunter HarnessConfig (behaviour pipeline, model-free).
|
||||
|
||||
The single, strictly-localized HarnessX composition site (mirrors DocuHarnessX
|
||||
``make_docgen``): every ``harnessx`` import is inside this function so importing
|
||||
this module never touches the optional extra. Raises :class:`HarnessXUnavailableError`
|
||||
when the extra is absent. The per-task step budget is enforced on the task at run
|
||||
time, not composed here.
|
||||
"""
|
||||
require_harnessx()
|
||||
from harnessx.bundles.context import make_context
|
||||
from harnessx.bundles.control import make_control
|
||||
from harnessx.bundles.tools import make_tools
|
||||
from harnessx.core.builder import HarnessBuilder
|
||||
from harnessx.processors.context.strategies.system_prompt.default import DefaultSystemPromptBuilder
|
||||
|
||||
builder = (
|
||||
HarnessBuilder()
|
||||
| make_context(system_builder=DefaultSystemPromptBuilder())
|
||||
| make_tools(skill_loading=False)
|
||||
| make_control(
|
||||
include_reliability=True,
|
||||
include_budget=True,
|
||||
token_threshold=token_threshold,
|
||||
max_cost_usd=max_cost_usd,
|
||||
)
|
||||
)
|
||||
return builder.build()
|
||||
|
||||
Reference in New Issue
Block a user