- variant_search.search_tree skips a malformed store row with a
variants_store_row_invalid degradation instead of aborting the scan;
a hit on a demoted or flowless overlay record is never merged (test).
- seed.pair_lessons is the one shape derivation shared by the exporter
and leave-one-out; labels pass line and sink together; pairs without a
labeled function are reported as no_labeled_function.
- config parses [variants].max_mechanisms with the shared int helper.
- regress.candidate.hotspot_from_finding is the one zero-score hotspot
helper for forced inventory and variant findings.
- design synced (SearchResult, hits_to_findings summaries, candidate.py).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.3 (Req 3.4, 3.5, 3.6, 6.2).
- [variants] enabled / max_mechanisms config; MAP runs variant search
after the overlay in standard and deep scans only, appends the variant
findings, rewrites overlay.json with merged ids, and the manifest
records mechanisms_searched, files_searched, findings and
merged_into_overlay.
- REGRESS: variant findings join the candidate list after the promotions
as zero-score hotspots, under the existing safety check and cap.
- Spec notes for group 2 and the design synced (variant_search.py,
seed.py, derive_shape signature).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.1 (Req 2.1-2.4). Spec approved by the maintainer in session.
- semantic/variants.py: Shape (language, sink trailing name, arity,
source positions and kinds, closed guard kind, mechanism id) with a
stable text-free key; derive_shape finds the labeled sink call site in
the vulnerable FileIR, traces argument identifiers through binds to a
fact source, a parameter or a container read, and classifies the guard
the fixed side added from the labeled function's added statements
against a closed set (none when unknown). No path, line or literal
enters a shape; a constant-only call teaches nothing.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
- tasks.md: round-12 verdict and the decision to assemble the redaction
test's example key at runtime and rewrite the unpushed series so no
commit carries a contiguous key literal; two dated notes marked as
superseded by tasks 8.1 and 8.4.
- roadmap: the folded inventory-vs-overlay Youden item is checked.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
- tasks.md: all tasks checked; the two escalations are closed (5.1 by the
Req 10 pointer rows, 5.3 by the Req 9 review tiers); round 11 recorded
with the carry-forwards for corpus-seeded-mechanisms and
overlay-ir-completeness.
- roadmap: measured state after tiers and pointer slices (vibe-py 115,
agent-vfc 59, gates byte-identical to pre-spec); pair-corpus-honesty
marked completed.
- spec.json: completed 2026-09-05 after eleven review rounds.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 8.4 (Req 10.6, 5.1).
- harvest.merge_recipes appends recipes by name and never rewrites
existing rows.
- vibe-py/build_recipes.py --pointers: the 40 LLM-generated repositories
without a license become vendored = false rows (80 pairs, agent,
seeded) appended to recipes.toml; unlicensed human repos stay skipped.
- agent-vfc/build_recipes.py --from ... --pointers [--cache]: unlicensed
agent-authored fixes become title-tier pointer rows whose function
label is derived from the cached excerpt (30 added from a fresh
662-candidate search); TOML booleans rendered correctly.
- catalogs regenerated: vibe-py 115 rows (35 vendored + 80 pointer),
agent-vfc 59 (29 + 30); no excerpt is vendored for any pointer row.
- `pairs --slice vibe-py --pointers` scores both provenances (human 35,
agent 80); the default run skips 80 pairs with pointer_pair_skipped;
suite and gates unchanged. README/docs name the nightly command.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 8.3 (Req 10.1-10.5).
- pairs.py: catalog rows with vendored = false carry their harvest recipe
and no excerpts; evaluate_catalog harvests them into
OPENULTRASAST_PAIR_CACHE (default ~/.cache/openultrasast/pairs) only
when `pairs --pointers` or OPENULTRASAST_PAIRS_NETWORK=1 allows the
network, otherwise records pointer_pair_skipped (a failed fetch is
pointer_pair_fetch_failed with detail); a warm cache never scores
without the switch. select_vendored for tests and gates.
- harvest.py: materialize_pointer harvests into the cache without the
license requirement (nothing is redistributed); redaction applies.
- catalog_gen.py: emits pointer rows with recipe fields; training
manifests skip them.
- pair_gate and improve/evolve.evaluate_profiles score vendored pairs
only; cli `pairs --pointers`; .gitignore covers the cache path.
- tests: loader, skip/degradation, cache hit, fetch failure, warm cache
with network off, profile gate exclusion, harvest and generator paths;
conftest keeps every test offline whatever the shell exports.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Two engine specs drafted after the pair-corpus baseline, awaiting owner
approval before implementation:
- corpus-seeded-mechanisms: structural shapes derived from trusted pairs,
export, variant search merged with overlay flows, leave-one-out
measurement, and a closed `mechanisms` lever for the improve loop.
- reachability-flow-model: call graph over FunctionIR, reachability from
entry points to sinks, hunter client preferring HarnessX (task 4.1b).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Harvest 176 isolated vuln-vs-fix functions (109 CVEs, 352 training rows)
into benchmarks/pairs/vfc with provenance and a JSONL training manifest.
Score the slice as an honesty dashboard (ousast pairs --slice vfc);
keep the pair gate on the local fixtures only.
Add openultrasast[semantic] with official grammar wheels and one CST walker
into FileIR. quick stays extra-free. Host CLI is not overlay-ready. Default CI
stays extra-free; a semantic job runs pytest -m semantic.
Regex inventory stays the proposer. MAP attaches promote/demote/unadjudicated/coverage
from intra-file taint (tree-sitter when a grammar parses, Python ast as last resort).
Deep sandboxes promotions only, ordered by the existing heuristic ranker plus optional
OpenRouter mechanism recall. The SAST pair slice scores overlay outcomes; the stage-1
detection gate and local pair gate stay inventory-only.
Replace the hardcoded skill-name stub with a real, routable skill index.
- skills.py (zero-dep): SkillDescriptor inventory (~22 ToB skills with routing
metadata — languages, tags, vulnerability classes, stages — plus a short operating
checklist each), aligned with the spec's routing examples. route_skills scores by
vulnerability-class/tag/stage signal with a language filter+bonus, and never selects
on a pure language match. route_skills_for_target derives vuln classes from tags;
select_skill_context packs checklists into a char budget; skill_chunks chunks the
guidance into the `skills` retrieval namespace.
- hunter.select_skill_snippets now delegates to the router (existing C-memory and
crypto+auth routing expectations preserved).
- hunter_harness._describe feeds the budget-capped routed skill context into the
HarnessX hunter prompt (the genuine consumer of skill_budget_chars).
- 8.4 zero-dep guard extended to cover skills.py.
tests/test_skills.py (9 tests: C parser → memory/fuzzing, crypto → constant-time/
vector, language-mismatch + language-only exclusion, stage filtering, budget cap,
namespace chunking). openrouter-sast-harness Phase 14 checked off.
188 passed, 4 skipped; ruff + mypy clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A zero-dependency MCP server over stdio (newline-delimited JSON-RPC 2.0) for
OpenCode/IDE integration, exposing only stable project-level operations.
- mcp.py: McpServer (pure handle() dispatch) + serve() stdio loop, stdlib-only.
Ten tools: openultrasast.scan/status/findings/get_finding/evidence/artifacts/
benchmark/explain/propose_patch/export_report. scan/benchmark run the bounded
analysis pipeline; NO tool accepts a free-form command — no shell, Docker, or
internal hunter tools are exposed. propose_patch degrades visibly (patch oracle
is a later phase). initialize/tools/list/tools/call/ping handled; notifications
get no response; errors surface as JSON-RPC errors instead of crashing the loop.
- cli: `ousast mcp` subcommand launches the stdio server (lazy import → keeps the
mcp↔cli cycle one-directional).
- tests/test_mcp.py: tool listing is exactly the narrow surface, surface-safety
(no shell/exec/command tool), scan→status→findings→explain→evidence flow,
export_report, propose_patch degradation, and the stdio loop round-trip.
- README "MCP server" section; openrouter-sast-harness Phase 12 checked off.
Verified end-to-end over real stdio (initialize → tools/list → scan). 179 passed,
4 skipped; ruff + mypy clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Express the deterministic scan stages as model-disabled processors carrying work in
named slots under a read/write allow-list validated on every lifecycle hook.
- slot_contract.py (zero-dep): SlotContractError, SlotContractMixin (reads_slots/
writes_slots/skip_model=True, validates reads pre-hook and writes post-hook),
SlotPipeline (runs stages in order, enforces the model-disabled invariant, emits
stage_start/stage_end lifecycle hooks). Preserves the ProcessorSpec.reads/writes
provenance discipline without forcing deterministic stages through HarnessX's agent
loop ("bridge, do not map").
- stage_processors.py: Preprocess/EntryPoint/Rank/QuickFindings/PolicyScoring/Score as
skip_model slot processors; run_quick_pipeline is byte-identical to the quick path.
host_under_harnessx is the lazy, capability-gated bridge that disables the model on
HarnessX's BeforeModelEvent.skip_model (proven against the real API when the extra
is present).
- 8.4 zero-dep guard hardened: now hermetic (subprocess) and covers the two new
modules; the module-level import scan flags only column-0 imports so the intended
lazy import inside host_under_harnessx is allowed.
The sync HarnessRuntime stays the top-level driver (design "stays through Phase 4");
these slot processors are the model-disabled representation, ready for Phase 5.
tests/test_slot_contract.py (8 tests). 153 passed, 3 skipped; ruff + mypy clean;
detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
9.1 — end-to-end fallback parity: a requested-but-unavailable HarnessX standard
scan (models set, extra absent) produces byte-identical findings to the plain
deterministic path, differing only by the recorded hunter_pool/verify degradations;
quick mode is unaffected by the extra (identical findings present-or-absent, never a
degradation).
9.2 — per-slot serialization round-trip: StaticFinding / RankingScore / FileTarget /
VerificationResult / ScoreArtifact round-trip through JSON without loss, plus the real
findings.json reload path preserves nested slots (reachability evidence/conditions,
tags) rather than restoring them empty. Covers the current artifact-persistence model;
the HarnessX processor slot-contract variant stays deferred with task 6.3.
tests/test_scan_pipeline_integrity.py (8 tests). 145 passed, 3 skipped; ruff + mypy
clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Make the >=90% recall / <10% FP detection gate a first-class, score-independent
CI step with non-regression guards.
- openultrasast.gate: canonical corpus map + DetectionMetrics/DetectionGateVerdict,
evaluate_corpus/detection_gate (no score input — cannot be unblocked by score),
merge_gate (ANDs the hard detection gate with the advisory-first score gate),
within_tolerance, and a `python -m openultrasast.gate` entrypoint that exits
non-zero on a breach. Wired as an explicit step in both the GitHub and Dagger
pipelines.
- 8.1: pipeline tests prove a high/passing score cannot unblock a recall-breaching
change; the gate fails below 90% recall or at/above 10% FP.
- 8.2: tests/golden/detection_baseline.json + a byte-identical baseline test for
the default ruleset / empty ledger (44/47 = 93.62% recall, 0% FP).
- 8.3: within_tolerance band logic + a capability-gated HarnessX tolerance test.
- 8.4: zero-dependency guard — governance/scoring/benchmark planes never import
HarnessX (sys.modules check + module-scope import scan).
- test_detection_benchmarks now sources the corpus/aggregation from the gate module.
128 passed, 3 skipped; ruff + mypy clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Stand up the deterministic self-improvement loop over the ruleset/policy
governance data. Zero-dependency and fully tested offline; detection gate intact.
- 7.1/7.2 bounded levers + validator (improve/validator.py): two levers (rule,
policy); typed edits make pattern-text and 0-5 severity changes structurally
impossible; EvolveValidator rejects bad status, unknown rule_ids (loop cannot add
rules), enabled->disabled jumps (shadow staging mandatory), unresolved CWEs, and
out-of-bounds K/MIN_SCORE, raising StrictValidationError.
- 7.3 novelty + replay + journal (improve/journal.py): a JSON round journal records
every edit/lever/outcome; reverted edits are blocked from re-proposal (novelty);
a replay smoke gate boots the candidate ruleset before re-benchmarking.
- 7.4 benchmark deltas -> rule_signals.json (recall side: miss; precision side: fp).
- 7.5 hard acceptance gate + byte-for-byte revert (improve/evolve.py): a round is
accepted only if recall >= 90% AND FP < 10% AND project score did not regress AND
no matched-finding regression; otherwise the ledger is never written (byte-for-byte
revert). run_round/run_improvement drive rounds to convergence.
- 7.6 auto-shadow precision-draggers (enabled rules with FPs -> shadow, never
delete), nominate coverage gaps via rule_signals, and keep the project score as the
reward separate from the recall/FP feasibility gate (the loop cannot raise the
score by shadowing rules that breach the gate — proven by the revert test).
Design note: the loop's safety machinery is deterministic and benchmark-driven
(the proposer is the per-rule signal); the LLM MetaAgent.evolve is the optional
richer proposer that plugs into this same validated/gated machinery when the
HarnessX extra is present (offline-untestable). This mirrors how DocuHarnessX
realizes self-improvement.
9 new tests (validator, proposer/signals, accept, byte-for-byte revert, convergence);
118 pass total, 2 construction smokes skip without the extra; ruff + mypy clean;
detection gate 93.6%/0% intact. Tasks 7.1-7.6 checked off.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Adopt HarnessX at the hunter loop and verifier, capability-gated so the default
zero-dependency scan is byte-identical and CI (no extra) exercises the fallback.
- 6.1/6.2 build_sast() (single, lazy HarnessX composition site in harness_ext.py,
mirrors DocuHarnessX make_docgen) + HxScanOrchestrator (hunter_harness.py): a
drop-in for run_hunter_pool that runs one bounded HarnessX agent task per ranked
target (ModelConfig(main=AnthropicProvider).agentic(build_sast()).run(BaseTask)),
parses findings from a JSON structured-output contract, returns the existing
HunterPoolResult, maps scan_id->session_id and emits per-target exit_reason/cost/
steps to the outer trace.
- 6.4 verify_judge.verify_findings_dispatch: a drop-in superset of
verification.verify_findings — structural verifier verbatim by default (rejects
unknown reachability), HarnessX llm-judge (build_judge_prompt/parse_judge_response)
when the extra + a verifier model are present; verdict mapped to VerificationResult
evidence-gated (plausible>=0.6 accepted; unsupported/refused rejected; infra
failure never auto-rejects).
- 6.5 cli capability gate (hx_hunter/hx_verify): selects HxScanOrchestrator /
judge only in standard mode with a configured model AND has_harnessx(); otherwise
falls back and records a {stage,reason:harnessx_extra_unavailable} degradation in
the manifest (new append-only write_manifest degradations kwarg). With no model
configured the manifest stays byte-identical.
- pyproject: mypy override ignore_missing_imports for harnessx.* (absent in the
type-check env).
Verified the HarnessX path constructs against the real extra in a throwaway venv
(build_sast -> HarnessConfig; orchestrator -> Harness; build_judge_prompt/
parse_judge_response/verdict mapping) — only the live agent.run()/provider.complete()
LLM calls are offline-untestable and stay gated. 109 tests pass (2 construction
smokes skip without the extra), ruff + mypy clean, detection gate 93.6%/0% intact.
Tasks 6.1/6.2/6.4/6.5 done; 6.3 (deterministic stages as HarnessX processors) deferred.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Wire HarnessX as an optional, SHA-pinned extra with a lazy capability guard,
keeping the core install zero-dependency.
- pyproject: add [project.optional-dependencies] harnessx pinned to commit
274c512 (Darwin-Agent/HarnessX). Core dependencies stays []. uv_build emits
`Requires-Dist: harnessx @ git+...@<sha> ; extra == 'harnessx'` +
`Provides-Extra: harnessx` (verified via uv build); release ships to GitHub
Releases, not PyPI, so the direct-reference is fine.
- harness_ext.py: the capability seam. has_harnessx() probes via find_spec
(never imports harnessx); require_harnessx() imports lazily on demand and
raises HarnessXUnavailableError with install guidance when absent. Every
Phase 3+ HarnessX path routes through here so heavy imports stay cold by default.
- uv.lock refreshed: the universal lock now resolves the harnessx tree (149 pkgs)
but `uv sync --group dev` installs only 57 — harnessx is NOT importable in the
default/CI venv (verified), so CI stays lean and never re-locks.
- Tests: core deps empty; the extra is a 40-hex SHA-pinned git URL; importing
openultrasast.cli does not import harnessx; has_harnessx probes without
importing; require_harnessx raises with guidance when absent.
98 tests pass; ruff + mypy clean; detection gate 93.6%/0% intact. Spec task 5
(5.1 + 5.2) checked off; next is task 6 (build_sast hunter sub-harness).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Run and record the HarnessX dependency dry-run audit and assess blueprint
alignment with the author's DocuHarnessX, to steer OpenUltraSAST's HarnessX
adoption from opencode/claude-code.
- dependency-audit.md: records the realized transitive graph (118 packages from
30 mandatory HarnessX deps). Key finding: every heavy backend (docker/e2b/
playwright/office-PDF) is lazily imported and the gateway is a separate dist, so
the tree is an install-time (not runtime) cost; ~55-60 packages are dead weight
for SAST. Decision: ACCEPT as the optional SHA-pinned extra openultrasast[harnessx]
(core stays zero-dep; lazy heavy imports; matches DocuHarnessX precedent), with a
fully-specified vendored lean subset as the triggered fallback and upstream
dependency-extras as the preferred long-term slimming path. Records the
security-tool residual risks (browser binary, docker socket, web server, CVE
surface) and their mitigations.
- blueprint-alignment.md: DocuHarnessX <-> OpenUltraSAST assessment. They share the
spine (one composition factory, model bound in CLI via ModelConfig.agentic,
content-neutral processors over typed slots, project-loaded domain/"ontology"
seam, fail-closed LLM-judge firewall, Kiro fresh-subagent steering). DocuHarnessX
realizes "self-improving multi-agent" via the firewall + claude-code/opencode
steering loop, NOT MetaAgent.evolve (Train is future there); OpenUltraSAST builds
the evolve loop on the same substrate. Three principled divergences (hybrid runtime
confining HarnessX to the hunter pool; evolve reward/feasibility split + ledger;
zero-dep core with opt-in extra) with reconciliation, plus a concrete Phase 3
blueprint mirroring DocuHarnessX.
Checks off spec task 5.1.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
