Close the HarnessX self-improving loop without an agent in the loop. Every scan
now feeds its verifier outcomes back into the next scan's ranking:
- New `calibrate` stage (after rank, before findings) loads a persistent ledger
of scoped false-positive learnings and demotes every previously-rejected scope
(`calibrate_rankings`), writing `calibration/applied_calibrations.json`.
- New `record_calibration` stage (after verify) converts this run's non-accepted
verifier outcomes into scoped `FalsePositiveLearning` records
(`learnings_from_verifications`) and merges them into
`.openultrasast/calibration/false_positive_learnings.json`, de-duplicated by
finding ID so a repeat rejection does not compound without bound.
- Lower the calibration floor (MIN_CALIBRATED_PRIORITY = 0.1) so accumulated
rejections can demote a scope below the natural ranking minimum.
Demotion is scoped and reversible: a rejection in one path never touches an
accepted finding in another, and priority is demoted rather than the finding
deleted so the audit trail explains every downgrade. First run is a no-op
(empty ledger), so existing single-run behavior is unchanged.
Adds tests/test_pipeline_calibration.py (two-run demotion, order-independent),
documents the realized loop in the README, and checks off the
standard_security_harness milestone in tasks.md.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- OpenUltraSAST is driven from the command line (ousast CLI, or OpenCode invoked
on the repo running the harness for you); show both.
- Mark the kiro-* skills and AGENTS.md as maintainer-only SDLC tooling, not part
of using OpenUltraSAST; drop them from the user-facing skill list.
- State that OpenCode is the agent steering the HarnessX self-improving cycle
today (reads verifier/benchmark signals, applies scoped calibration via the
triage skill, re-runs the gate); autonomous in-pipeline feedback is the next
milestone.
- Remove a duplicated Fusion section.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Expand the README with: how to run the per-language benchmarks and the latest
measured results (C/C++ 93.3%, Python 93.3%, JavaScript 92.3%, Java 100%;
overall 93.6% recall, 0% false positives); the OpenCode skills, the ultra
(OpenUltraCode) fix lifecycle, and fusion deepening; how a false positive is
eliminated via the evidence-ladder verifier and scoped calibration; how a false
negative becomes a benchmark calibration record routed to a next-improvement
candidate; and how the HarnessX self-improving cycle is realized. Maturity is
marked (implemented / primitive / roadmap) to stay accurate to the current gate.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
