Task 8.4 (Req 10.6, 5.1).
- harvest.merge_recipes appends recipes by name and never rewrites
existing rows.
- vibe-py/build_recipes.py --pointers: the 40 LLM-generated repositories
without a license become vendored = false rows (80 pairs, agent,
seeded) appended to recipes.toml; unlicensed human repos stay skipped.
- agent-vfc/build_recipes.py --from ... --pointers [--cache]: unlicensed
agent-authored fixes become title-tier pointer rows whose function
label is derived from the cached excerpt (30 added from a fresh
662-candidate search); TOML booleans rendered correctly.
- catalogs regenerated: vibe-py 115 rows (35 vendored + 80 pointer),
agent-vfc 59 (29 + 30); no excerpt is vendored for any pointer row.
- `pairs --slice vibe-py --pointers` scores both provenances (human 35,
agent 80); the default run skips 80 pairs with pointer_pair_skipped;
suite and gates unchanged. README/docs name the nightly command.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Regex inventory stays the proposer. MAP attaches promote/demote/unadjudicated/coverage
from intra-file taint (tree-sitter when a grammar parses, Python ast as last resort).
Deep sandboxes promotions only, ordered by the existing heuristic ranker plus optional
OpenRouter mechanism recall. The SAST pair slice scores overlay outcomes; the stage-1
detection gate and local pair gate stay inventory-only.
A zero-dependency MCP server over stdio (newline-delimited JSON-RPC 2.0) for
OpenCode/IDE integration, exposing only stable project-level operations.
- mcp.py: McpServer (pure handle() dispatch) + serve() stdio loop, stdlib-only.
Ten tools: openultrasast.scan/status/findings/get_finding/evidence/artifacts/
benchmark/explain/propose_patch/export_report. scan/benchmark run the bounded
analysis pipeline; NO tool accepts a free-form command — no shell, Docker, or
internal hunter tools are exposed. propose_patch degrades visibly (patch oracle
is a later phase). initialize/tools/list/tools/call/ping handled; notifications
get no response; errors surface as JSON-RPC errors instead of crashing the loop.
- cli: `ousast mcp` subcommand launches the stdio server (lazy import → keeps the
mcp↔cli cycle one-directional).
- tests/test_mcp.py: tool listing is exactly the narrow surface, surface-safety
(no shell/exec/command tool), scan→status→findings→explain→evidence flow,
export_report, propose_patch degradation, and the stdio loop round-trip.
- README "MCP server" section; openrouter-sast-harness Phase 12 checked off.
Verified end-to-end over real stdio (initialize → tools/list → scan). 179 passed,
4 skipped; ruff + mypy clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add a [harnessx] config block so the agentic plane can be steered from
openultrasast.toml, and make the LLM provider selectable (the hunter and verifier
previously hardcoded AnthropicProvider).
- config.HarnessxConfig: provider ("anthropic"|"openai"|"litellm", default
anthropic), max_cost_usd (2.0), token_threshold (120000); parsed from [harnessx].
- harness_ext.build_provider: single lazy, capability-guarded provider factory.
The provider name is validated before any import, so an unknown choice fails with
a clear ValueError even when the extra is absent; each provider reads its own
standard key env var (openai -> OPENAI_API_KEY, anthropic -> ANTHROPIC_API_KEY).
- HxScanOrchestrator and verify_findings_dispatch take a provider and route through
build_provider; cli threads config.harnessx.{provider,max_cost_usd,token_threshold}
into both the hunter and the verifier.
- README: "Configuring the HarnessX agentic plane" section (install, models,
provider, budgets, fallback/degradation).
Test isolation: convert the "harnessx not in sys.modules" import-guards to a
hermetic subprocess fixture (tests/conftest.py assert_cold_of_harnessx) and
simulate extra-absent in the CLI dispatch tests, so the suite is deterministic
whether or not the optional extra is installed (it now is, locally). Verified the
OpenAI path constructs end-to-end (build_provider -> OpenAIProvider; orchestrator
-> Harness) with the budget threaded through.
137 passed, 3 skipped; ruff + mypy clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Expose the bounded, gated self-improvement loop as a first-class subcommand so it
can be steered from the command line / opencode.
- `ousast improve <manifest>` resolves the target from the benchmark manifest,
runs the loop to convergence, and writes accepted edits to the loop-owned ledger
at <target>/.openultrasast/calibration/rule_policy.json — exactly where `scan`
and `benchmark` read it, so a subsequent scan of that target automatically
picks up the improved ruleset (the loop closes end-to-end).
- Flags: --max-rounds, --recall-floor, --fp-ceiling, --ledger, --journal,
--ruleset-dir, and --dry-run (runs against a throwaway ledger, touches nothing).
- Per-round output: reason, recall/fp/score deltas, and the edits applied.
- tests/test_cli_improve.py: accept path writes the loop-owned ledger (noisy rule
auto-shadowed), --dry-run never touches it, missing manifest fails loudly.
- README: quickstart entry + a "Rule-level loop: ousast improve" subsection
documenting the bounded levers, hard acceptance gate, and the HarnessX MetaAgent
as the optional proposer over the same gated machinery.
131 passed, 3 skipped; ruff + mypy clean; detection gate PASS.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
CI: the dagger job repeatedly failed (and once hung 10 min) pulling the engine
image from registry.dagger.io. Pin the dagger CLI to 0.21.7 (matching dagger-io
and the SDK's CLI_VERSION so the engine tag is deterministic), pre-pull
registry.dagger.io/engine:v0.21.7 with a dependency-free retry loop (5 attempts,
90s/pull, 20s backoff) so the action's `docker image inspect` finds it cached and
skips its own flaky pull, and add a 20-minute job timeout so a stalled pull can
no longer hang the run. The canonical `checks` job is unchanged.
Docs: add a "Project scores and central CWE policy" README section documenting
the implemented Phase 1 — central verycode CWE policy, policy-governed severity
(one CWE = one severity), the fail-loud invariant, the 0-100 project score and
two-condition advisory-first gate, the score.json/manifest artifacts, and the
CWE-120 -> CWE-121 remap.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Close the HarnessX self-improving loop without an agent in the loop. Every scan
now feeds its verifier outcomes back into the next scan's ranking:
- New `calibrate` stage (after rank, before findings) loads a persistent ledger
of scoped false-positive learnings and demotes every previously-rejected scope
(`calibrate_rankings`), writing `calibration/applied_calibrations.json`.
- New `record_calibration` stage (after verify) converts this run's non-accepted
verifier outcomes into scoped `FalsePositiveLearning` records
(`learnings_from_verifications`) and merges them into
`.openultrasast/calibration/false_positive_learnings.json`, de-duplicated by
finding ID so a repeat rejection does not compound without bound.
- Lower the calibration floor (MIN_CALIBRATED_PRIORITY = 0.1) so accumulated
rejections can demote a scope below the natural ranking minimum.
Demotion is scoped and reversible: a rejection in one path never touches an
accepted finding in another, and priority is demoted rather than the finding
deleted so the audit trail explains every downgrade. First run is a no-op
(empty ledger), so existing single-run behavior is unchanged.
Adds tests/test_pipeline_calibration.py (two-run demotion, order-independent),
documents the realized loop in the README, and checks off the
standard_security_harness milestone in tasks.md.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- OpenUltraSAST is driven from the command line (ousast CLI, or OpenCode invoked
on the repo running the harness for you); show both.
- Mark the kiro-* skills and AGENTS.md as maintainer-only SDLC tooling, not part
of using OpenUltraSAST; drop them from the user-facing skill list.
- State that OpenCode is the agent steering the HarnessX self-improving cycle
today (reads verifier/benchmark signals, applies scoped calibration via the
triage skill, re-runs the gate); autonomous in-pipeline feedback is the next
milestone.
- Remove a duplicated Fusion section.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Expand the README with: how to run the per-language benchmarks and the latest
measured results (C/C++ 93.3%, Python 93.3%, JavaScript 92.3%, Java 100%;
overall 93.6% recall, 0% false positives); the OpenCode skills, the ultra
(OpenUltraCode) fix lifecycle, and fusion deepening; how a false positive is
eliminated via the evidence-ladder verifier and scoped calibration; how a false
negative becomes a benchmark calibration record routed to a next-improvement
candidate; and how the HarnessX self-improving cycle is realized. Maturity is
marked (implemented / primitive / roadmap) to stay accurate to the current gate.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
