feat(ruleset): Phase 2 (task 3) — rules as governed data, severity from policy

Externalize detection rules to versioned TOML and remove rule-local severity, so
rules detect and the central CWE policy decides severity — fixing "rules too
influential / not centrally managed". Zero-dependency; detection is byte-identical
(benchmark recall 44/47 = 93.6%, FP 0% unchanged) and the 90/10 gate holds.

- RulesetStore (ruleset/store.py + ruleset/<lang>/rules.toml): PatternRule as data
  (rule_id, title, languages, cwe, tags, pattern, status, min_evidence_level,
  precision_estimate, version) with NO severity; load_ruleset (sorted, with
  optional rule_policy.json ledger overlay) and a stdlib TOML write_ruleset. The
  32 rules were generated from the prior PATTERN_RULES with a verified exact
  round-trip (pattern/cwe/languages/tags/title).
- findings.py: drop the hardcoded PatternRule class + PATTERN_RULES tuple; load the
  default ruleset from TOML; quick_scan_findings(root, targets, rankings, ruleset,
  policy) resolves each finding's severity from policy via resolve_severity
  (SEVERITY_LABEL maps 0-5 -> string) and tags findings with their rule status.
- Three-status volume control: StaticFinding gains status (default "enabled");
  disabled rules never fire; shadow findings are partitioned out of the report and
  score in the CLI (written to shadow_findings.json) — no shadow rules by default.
- Emission floors (RulesetConfig.min_emit_priority / min_emit_precision, default
  0.0 = emit all) replace "every match is a finding".
- CLI loads the ruleset as a stage and uses it for the fail-loud policy check,
  detection, hunter pool, and the score CWE map. Wheel ships the ruleset TOML.

85 tests pass; ruff + mypy clean. Spec tasks 3.1-3.4 checked off.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-06-22 15:27:16 +02:00
co-authored by Claude Opus 4.8
parent fb7594a3ed
commit 444d8c9490
15 changed files with 645 additions and 347 deletions
+20 -4
View File
@@ -7,8 +7,10 @@ from pathlib import Path
from typing import Literal
from .findings import StaticFinding, quick_scan_findings
from .policy import CwePolicy
from .preprocess import FileTarget
from .rank import RankingScore
from .ruleset import PatternRule
HunterTier = Literal["A", "B", "C"]
@@ -98,13 +100,21 @@ def schedule_hunter_tasks(
return scheduled
def run_hunter_pool(root: Path, targets: list[FileTarget], rankings: list[RankingScore], *, scan_id: str) -> HunterPoolResult:
def run_hunter_pool(
root: Path,
targets: list[FileTarget],
rankings: list[RankingScore],
*,
scan_id: str,
ruleset: tuple[PatternRule, ...] | None = None,
policy: dict[str, CwePolicy] | None = None,
) -> HunterPoolResult:
tasks = schedule_hunter_tasks(targets, rankings, retrieval_context_by_path=build_retrieval_context_by_path(root, targets))
findings: list[StaticFinding] = []
trajectories: list[HunterTrajectory] = []
max_workers = min(8, max(1, len(tasks)))
with ThreadPoolExecutor(max_workers=max_workers) as executor:
for task_findings, trajectory in executor.map(lambda task: _run_hunter_task(root, scan_id, task), tasks):
for task_findings, trajectory in executor.map(lambda task: _run_hunter_task(root, scan_id, task, ruleset, policy), tasks):
findings.extend(task_findings)
trajectories.append(trajectory)
return HunterPoolResult(findings=sorted(findings, key=lambda item: item.finding_id), trajectories=trajectories)
@@ -140,8 +150,14 @@ def select_skill_snippets(target: FileTarget, *, max_snippets: int = 4) -> list[
return sorted(dict.fromkeys(snippets))[:max_snippets]
def _run_hunter_task(root: Path, scan_id: str, task: HunterTask) -> tuple[list[StaticFinding], HunterTrajectory]:
findings = quick_scan_findings(root, [task.target], [task.ranking])[: task.budget.max_findings_per_target]
def _run_hunter_task(
root: Path,
scan_id: str,
task: HunterTask,
ruleset: tuple[PatternRule, ...] | None = None,
policy: dict[str, CwePolicy] | None = None,
) -> tuple[list[StaticFinding], HunterTrajectory]:
findings = quick_scan_findings(root, [task.target], [task.ranking], ruleset, policy)[: task.budget.max_findings_per_target]
return findings, HunterTrajectory(
scan_id=scan_id,
target_path=task.target.path,