Commit Graph
6 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 5.5 b9afb66250 chore(harnessx-removal): delete HarnessX code and packaging (task 6)
Deletes harness_ext.py and hunter_harness.py with their HarnessX-only
tests (the judge path, verify_judge.py, went in task 5). The harnessx
extra, its mypy override and its uv.lock entries are removed (uv lock
drops only packages; no remaining version changes). The conftest
fixture becomes assert_cold_import; test_gate's two never-import checks
become one allow-list (stdlib, openultrasast, yaml) over the governance
planes' module-level imports, and the skipped live-extra tolerance test
is deleted. Audit rows for the two modules are removed, stage_processors
is reworded, and the module-audit manifest is regenerated (no orphan).

test_removed_plane_references implements Req 2.4 over src, tests and
pyproject.toml: a match must carry a `retired 2026-..` tag (the spec's
own slug is not counted). The README part is a strict xfail until task 8
rewrites the README's plane sections.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 12:18:42 +02:00
Marius CiepluchandClaude Opus 4.8 b5e40318db feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs
Release-readiness hardening.

- redaction.py (zero-dep): redact_secrets masks provider API keys, AWS/GitHub/Slack/
  Google tokens, bearer tokens, URL-embedded credentials, PEM private keys, and
  sensitive key=value assignments. Wired into the trace writer and the markdown report
  (on by default), so scan artifacts never leak a credential quoted from source.
- provider/openrouter.py: call_with_retry — exponential backoff on transient failures
  (HTTP 429/5xx, connection errors, timeouts); non-transient errors fail fast. Both the
  chat and embedding clients use it (max_attempts/retry_base_delay configurable).
- config [hardening]: redact_secrets (default true) + max_findings (bounded-CI output
  cap, 0 = unlimited; truncation is severity-ordered and disclosed as a `budget`
  degradation). Agentic spend stays bounded by [harnessx] max_cost_usd/token_threshold.
- docs/threat-model.md (trust boundaries, controls, sandbox limits) + docs/examples.md
  (six end-to-end walkthroughs); README "Security & hardening" section links both.
- CI lint/typecheck/test/detection-gate/dagger gates already existed.

tests/test_redaction.py, test_provider_retry.py, test_hardening_pipeline.py + config
tests. 206 passed, 4 skipped; ruff + mypy clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-22 20:02:34 +02:00
Marius CiepluchandClaude Opus 4.8 3fd0057e5f feat(skills): Phase 14 — Trail of Bits skill index + router
Replace the hardcoded skill-name stub with a real, routable skill index.

- skills.py (zero-dep): SkillDescriptor inventory (~22 ToB skills with routing
  metadata — languages, tags, vulnerability classes, stages — plus a short operating
  checklist each), aligned with the spec's routing examples. route_skills scores by
  vulnerability-class/tag/stage signal with a language filter+bonus, and never selects
  on a pure language match. route_skills_for_target derives vuln classes from tags;
  select_skill_context packs checklists into a char budget; skill_chunks chunks the
  guidance into the `skills` retrieval namespace.
- hunter.select_skill_snippets now delegates to the router (existing C-memory and
  crypto+auth routing expectations preserved).
- hunter_harness._describe feeds the budget-capped routed skill context into the
  HarnessX hunter prompt (the genuine consumer of skill_budget_chars).
- 8.4 zero-dep guard extended to cover skills.py.

tests/test_skills.py (9 tests: C parser → memory/fuzzing, crypto → constant-time/
vector, language-mismatch + language-only exclusion, stage filtering, budget cap,
namespace chunking). openrouter-sast-harness Phase 14 checked off.

188 passed, 4 skipped; ruff + mypy clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-22 19:51:04 +02:00
Marius CiepluchandClaude Opus 4.8 867bd536a2 feat(fusion): Phase 13 — two-panel fusion adjudication
OpenUltraCode-style deepening for findings that need more reasoning than the
ranker→hunter→verifier→mapping loop provides.

- fusion.py (zero-dep core + lazy LLM path): a deterministic trigger policy
  (critical/high severity, verifier disagreement, static-vs-semantic conflict,
  gates-risky-fix, high-assurance request); two panels that steel-man the
  vulnerability and false-positive cases; a deterministic, auditable decider that
  reconciles to one of five dispositions (accepted/rejected/mitigated/deferred/
  blocked); full disclosure (votes, model IDs, degradations, warnings, decision
  source). fuse_findings_dispatch routes panels through the configured HarnessX
  provider when panel_model + extra are present, else deterministic + a degradation.
- config [fusion]: enabled (default true), panel_model, decider_model,
  high_assurance.
- cli: standard-mode fusion stage runs on triggered findings, writes fusion.json
  and a manifest `fusion` block; quick mode never fuses. reports.write_manifest
  gained the fusion summary param.
- 8.4 zero-dep guard extended to cover fusion.py (lazy harnessx import only).
- README fusion section marked implemented + documented; openrouter-sast-harness
  Phase 13 checked off.

tests/test_fusion.py (16 tests) + config tests. 169 passed, 4 skipped; ruff + mypy
clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-22 18:57:03 +02:00
Marius CiepluchandClaude Opus 4.8 b9caeeb835 feat(slots): Phase 3 task 6.3 — model-disabled, slot-contracted deterministic stages
Express the deterministic scan stages as model-disabled processors carrying work in
named slots under a read/write allow-list validated on every lifecycle hook.

- slot_contract.py (zero-dep): SlotContractError, SlotContractMixin (reads_slots/
  writes_slots/skip_model=True, validates reads pre-hook and writes post-hook),
  SlotPipeline (runs stages in order, enforces the model-disabled invariant, emits
  stage_start/stage_end lifecycle hooks). Preserves the ProcessorSpec.reads/writes
  provenance discipline without forcing deterministic stages through HarnessX's agent
  loop ("bridge, do not map").
- stage_processors.py: Preprocess/EntryPoint/Rank/QuickFindings/PolicyScoring/Score as
  skip_model slot processors; run_quick_pipeline is byte-identical to the quick path.
  host_under_harnessx is the lazy, capability-gated bridge that disables the model on
  HarnessX's BeforeModelEvent.skip_model (proven against the real API when the extra
  is present).
- 8.4 zero-dep guard hardened: now hermetic (subprocess) and covers the two new
  modules; the module-level import scan flags only column-0 imports so the intended
  lazy import inside host_under_harnessx is allowed.

The sync HarnessRuntime stays the top-level driver (design "stays through Phase 4");
these slot processors are the model-disabled representation, ready for Phase 5.

tests/test_slot_contract.py (8 tests). 153 passed, 3 skipped; ruff + mypy clean;
detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-22 18:39:09 +02:00
Marius CiepluchandClaude Opus 4.8 649bd42fc3 feat(gate): Phase 4 task 8 — lock the detection gate into CI
Make the >=90% recall / <10% FP detection gate a first-class, score-independent
CI step with non-regression guards.

- openultrasast.gate: canonical corpus map + DetectionMetrics/DetectionGateVerdict,
  evaluate_corpus/detection_gate (no score input — cannot be unblocked by score),
  merge_gate (ANDs the hard detection gate with the advisory-first score gate),
  within_tolerance, and a `python -m openultrasast.gate` entrypoint that exits
  non-zero on a breach. Wired as an explicit step in both the GitHub and Dagger
  pipelines.
- 8.1: pipeline tests prove a high/passing score cannot unblock a recall-breaching
  change; the gate fails below 90% recall or at/above 10% FP.
- 8.2: tests/golden/detection_baseline.json + a byte-identical baseline test for
  the default ruleset / empty ledger (44/47 = 93.62% recall, 0% FP).
- 8.3: within_tolerance band logic + a capability-gated HarnessX tolerance test.
- 8.4: zero-dependency guard — governance/scoring/benchmark planes never import
  HarnessX (sys.modules check + module-scope import scan).
- test_detection_benchmarks now sources the corpus/aggregation from the gate module.

128 passed, 3 skipped; ruff + mypy clean; detection gate PASS.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-22 17:13:02 +02:00