mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 4.8
2d7ef48e52
Complete Phase 2: make benchmark output drive per-rule improvement, wire the
loop-owned ledger, and turn a confirmed false positive into a score knob. Zero
new dependencies; detection stays byte-identical (recall 44/47 = 93.6%, FP 0%)
and the 90/10 gate holds.
- Per-rule attribution (4.1): BenchmarkMiss/BenchmarkFalsePositive carry rule_id;
BenchmarkMetrics.per_rule aggregates matched/missed/false_positives per rule.
- Recommendation delta (4.2): evaluate_benchmark emits RuleRecommendation
(keep/shadow/loosen) written to rule_policy_delta.json; the inert
next_improvement_candidate stub is now rule-aware ("rule:<id>" or
"language_hunter").
- Loop ledger (4.3): cli loads the ruleset with the rule_policy.json overlay
(empty -> byte-identical; an entry overlays a rule's status/floors at load);
read_rule_ledger/write_rule_ledger added for the evolve loop. A disabled rule
in the ledger no longer fires (covered by an integration test).
- FP -> reachability knob (4.4): a confirmed false-positive learning lowers a
finding's effective reachability multiplier in the score
(fp_reachability_overrides + scoring reachability_override) instead of deleting
the rule, so recall stays high while the score stops over-penalizing.
93 tests pass; ruff + mypy clean. Spec tasks 4.1-4.4 checked off — Phase 2
(Core: rules-as-data + benchmark signals) is complete. The loop-driven decision
to apply ledger flips remains the gated evolve loop (Phase 3/4).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
23 lines
461 B
JSON
23 lines
461 B
JSON
{
|
|
"feature_name": "harnessx-self-improving-rulesets",
|
|
"created_at": "2026-06-22T10:07:18Z",
|
|
"updated_at": "2026-06-22T13:45:00Z",
|
|
"language": "en",
|
|
"phase": "implementation",
|
|
"approvals": {
|
|
"requirements": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"design": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"tasks": {
|
|
"generated": true,
|
|
"approved": true
|
|
}
|
|
},
|
|
"ready_for_implementation": true
|
|
}
|