mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-19 21:19:28 +00:00
Marius CiepluchandClaude Opus 4.8
6e87a58465
Stand up the deterministic self-improvement loop over the ruleset/policy governance data. Zero-dependency and fully tested offline; detection gate intact. - 7.1/7.2 bounded levers + validator (improve/validator.py): two levers (rule, policy); typed edits make pattern-text and 0-5 severity changes structurally impossible; EvolveValidator rejects bad status, unknown rule_ids (loop cannot add rules), enabled->disabled jumps (shadow staging mandatory), unresolved CWEs, and out-of-bounds K/MIN_SCORE, raising StrictValidationError. - 7.3 novelty + replay + journal (improve/journal.py): a JSON round journal records every edit/lever/outcome; reverted edits are blocked from re-proposal (novelty); a replay smoke gate boots the candidate ruleset before re-benchmarking. - 7.4 benchmark deltas -> rule_signals.json (recall side: miss; precision side: fp). - 7.5 hard acceptance gate + byte-for-byte revert (improve/evolve.py): a round is accepted only if recall >= 90% AND FP < 10% AND project score did not regress AND no matched-finding regression; otherwise the ledger is never written (byte-for-byte revert). run_round/run_improvement drive rounds to convergence. - 7.6 auto-shadow precision-draggers (enabled rules with FPs -> shadow, never delete), nominate coverage gaps via rule_signals, and keep the project score as the reward separate from the recall/FP feasibility gate (the loop cannot raise the score by shadowing rules that breach the gate — proven by the revert test). Design note: the loop's safety machinery is deterministic and benchmark-driven (the proposer is the per-rule signal); the LLM MetaAgent.evolve is the optional richer proposer that plugs into this same validated/gated machinery when the HarnessX extra is present (offline-untestable). This mirrors how DocuHarnessX realizes self-improvement. 9 new tests (validator, proposer/signals, accept, byte-for-byte revert, convergence); 118 pass total, 2 construction smokes skip without the extra; ruff + mypy clean; detection gate 93.6%/0% intact. Tasks 7.1-7.6 checked off. Co-Authored-By: Claude Opus 4.8 <[email protected]>
23 lines
461 B
JSON
23 lines
461 B
JSON
{
|
|
"feature_name": "harnessx-self-improving-rulesets",
|
|
"created_at": "2026-06-22T10:07:18Z",
|
|
"updated_at": "2026-06-22T15:35:00Z",
|
|
"language": "en",
|
|
"phase": "implementation",
|
|
"approvals": {
|
|
"requirements": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"design": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"tasks": {
|
|
"generated": true,
|
|
"approved": true
|
|
}
|
|
},
|
|
"ready_for_implementation": true
|
|
}
|