Externalize detection rules to versioned TOML and remove rule-local severity, so
rules detect and the central CWE policy decides severity — fixing "rules too
influential / not centrally managed". Zero-dependency; detection is byte-identical
(benchmark recall 44/47 = 93.6%, FP 0% unchanged) and the 90/10 gate holds.
- RulesetStore (ruleset/store.py + ruleset/<lang>/rules.toml): PatternRule as data
(rule_id, title, languages, cwe, tags, pattern, status, min_evidence_level,
precision_estimate, version) with NO severity; load_ruleset (sorted, with
optional rule_policy.json ledger overlay) and a stdlib TOML write_ruleset. The
32 rules were generated from the prior PATTERN_RULES with a verified exact
round-trip (pattern/cwe/languages/tags/title).
- findings.py: drop the hardcoded PatternRule class + PATTERN_RULES tuple; load the
default ruleset from TOML; quick_scan_findings(root, targets, rankings, ruleset,
policy) resolves each finding's severity from policy via resolve_severity
(SEVERITY_LABEL maps 0-5 -> string) and tags findings with their rule status.
- Three-status volume control: StaticFinding gains status (default "enabled");
disabled rules never fire; shadow findings are partitioned out of the report and
score in the CLI (written to shadow_findings.json) — no shadow rules by default.
- Emission floors (RulesetConfig.min_emit_priority / min_emit_precision, default
0.0 = emit all) replace "every match is a finding".
- CLI loads the ruleset as a stage and uses it for the fail-loud policy check,
detection, hunter pool, and the score CWE map. Wheel ships the ruleset TOML.
85 tests pass; ruff + mypy clean. Spec tasks 3.1-3.4 checked off.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Spec-driven plan to (1) adopt HarnessX (Darwin-Agent/HarnessX) directly where it
pays off — the LLM hunter loop, the LLM-judge verifier, and a meta-harness
self-improvement loop — and migrate scan flows onto it stage-by-stage; (2) make
detection rules versioned, centrally-governed data whose severity comes from a
vendored verycode CWE policy (CWE_Score.tsv) instead of the rule itself, fixing
"rules too influential / not centrally managed"; (3) adapt rules/policy inside
HarnessX's MetaAgent.evolve loop under bounded levers with replay/novelty/
validator gates; (4) make benchmarks actionable per-rule signals with a hard
accept gate and byte-for-byte revert; (5) compute a 0-100 verycode-style project
score with a two-condition CI gate.
Key decisions captured in research.md: HarnessX is an LLM-agent harness (not a
generic stage pipeline), so it is adopted only on the agentic plane behind an
optional SHA-pinned extra while the governance/scoring/benchmark planes stay
zero-dependency; the >=90% recall / <10% FP detection gate is a hard constraint
separate from the score reward and must hold at every phase; CWE-120 is unmapped
in verycode so C/C++ buffer rules remap to CWE-121 (severity 5).
Artifacts: spec.json (phase tasks-generated, approvals pending), requirements.md
(7 requirements, EARS), design.md (three-plane architecture), tasks.md (4
phases), research.md (D1-D9 decisions). Awaiting human approval per the kiro
3-phase gate; not yet implemented.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
