mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 20:19:29 +00:00
feat(governance): Phase 1 — CWE policy store + project score (zero-dep)
First implementation slice of the approved harnessx-self-improving-rulesets spec (Foundation: governance + scoring planes). Additive and zero-dependency; findings stay byte-identical and the >=90% recall / <10% FP detection gate is unchanged. - PolicyStore (policy/verycode.py): vendors verycode-policies CWE_Score.tsv (167 CWEs, CRLF + trailing-space header parsed defensively); CwePolicy(severity 0-5, static/dynamic); load_policy, resolve_severity (severity is policy-governed, keyed on CWE — one CWE = one severity), assert_rules_resolve (fail-loud if an enabled rule's CWE is unmapped). Remap the C/C++ buffer rules CWE-120 -> CWE-121 (CWE-120 is absent from verycode; 121 = Stack-Based Buffer Overflow, severity 5). - ScoreModel (scoring/project_score.py): SEV_WEIGHT, REACH_MULT, finding_penalty, project_score (0-100 exponential decay; one sev-5 reachable -> ~43), two-condition gate (sev-5+reachable always fails; score<min_score fails only when blocking), build_score_artifact. - Wire into _run_scan: policy_load + fail-loud policy_check stages; score stage writes score.json and merges a score block into the manifest (advisory-first). - ScoreConfig (k/min_score/block_severity_reachable/blocking) in config.py. - pyproject uv_build ships the vendored TSV in the wheel (verified). - Tests: test_policy, test_scoring, test_score_pipeline (artifact + fail-loud). 85 passed; ruff + mypy clean. Spec phases 1.1-2.4 checked off; spec approved. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
9b399fdc60
commit
ae365f6cb7
@@ -1,22 +1,22 @@
|
||||
{
|
||||
"feature_name": "harnessx-self-improving-rulesets",
|
||||
"created_at": "2026-06-22T10:07:18Z",
|
||||
"updated_at": "2026-06-22T10:07:18Z",
|
||||
"updated_at": "2026-06-22T10:20:00Z",
|
||||
"language": "en",
|
||||
"phase": "tasks-generated",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
"approved": false
|
||||
"approved": true
|
||||
},
|
||||
"design": {
|
||||
"generated": true,
|
||||
"approved": false
|
||||
"approved": true
|
||||
},
|
||||
"tasks": {
|
||||
"generated": true,
|
||||
"approved": false
|
||||
"approved": true
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": false
|
||||
"ready_for_implementation": true
|
||||
}
|
||||
|
||||
@@ -4,30 +4,30 @@ This plan splits the fused regex layer into three governed owners (rules detect,
|
||||
|
||||
## Phase 1 — Foundation: Governance & Scoring Planes (zero-dep)
|
||||
|
||||
- [ ] 1. Stand up the central CWE policy authority
|
||||
- [ ] 1.1 Vendor the verycode CWE severity data and build the Policy Store loader
|
||||
- [x] 1. Stand up the central CWE policy authority
|
||||
- [x] 1.1 Vendor the verycode CWE severity data and build the Policy Store loader
|
||||
- Vendor the verycode CWE-to-severity dataset into the governance plane and parse it with the standard library only, tolerating the trailing-space severity column header.
|
||||
- Expose, per CWE, an authoritative 0-5 severity, a static scope flag, and a dynamic scope flag; mark dynamic-only CWEs as report-only.
|
||||
- Observable completion: loading the policy yields the full set of CWE policy entries with correct static/dynamic flags and zero third-party imports, verified by a unit test.
|
||||
- _Requirements: 2.6, 2.9_
|
||||
- _Boundary: PolicyStore_
|
||||
|
||||
- [ ] 1.2 Expose policy-governed severity resolution
|
||||
- [x] 1.2 Expose policy-governed severity resolution
|
||||
- Resolve a finding's severity exclusively from the Policy Store keyed on the rule's CWE, discarding any severity string carried in legacy rule data.
|
||||
- Guarantee that two findings whose rules map to the same CWE receive identical severity.
|
||||
- Observable completion: a resolution call returns the policy severity (never a legacy string) and is identical for two rules sharing a CWE, proven by a unit test.
|
||||
- _Requirements: 2.2, 2.3_
|
||||
- _Boundary: PolicyStore_
|
||||
|
||||
- [ ] 1.3 Enforce the fail-loud startup CWE-resolution invariant
|
||||
- [x] 1.3 Enforce the fail-loud startup CWE-resolution invariant
|
||||
- Assert at startup that every enabled rule's CWE resolves in the Policy Store; abort the scan loudly before any work when an enabled rule's CWE is unmapped.
|
||||
- Observable completion: an enabled rule with an unmapped CWE aborts the scan with a loud error and no findings are produced, proven by an integration test.
|
||||
- _Requirements: 2.7_
|
||||
- _Boundary: PolicyStore_
|
||||
- _Depends: 1.1_
|
||||
|
||||
- [ ] 2. Build the project Score Model and CI gates (zero-dep)
|
||||
- [ ] 2.1 Compute the 0-100 policy-weighted, reachability-adjusted project score
|
||||
- [x] 2. Build the project Score Model and CI gates (zero-dep)
|
||||
- [x] 2.1 Compute the 0-100 policy-weighted, reachability-adjusted project score
|
||||
- Compute a single 0-100 project score from policy-governed severity weights multiplied by a reachability multiplier over reported findings, using the standard library only.
|
||||
- Key the reachability multipliers exactly on the three emitted reachability values (`reachable`, `inferred-file-surface`, `unknown`) without redefining the vocabulary.
|
||||
- Assign a zero score penalty to findings whose CWE is dynamic-only or unmapped, reporting them as out-of-scope or unmapped instead of scoring them.
|
||||
@@ -36,14 +36,14 @@ This plan splits the fused regex layer into three governed owners (rules detect,
|
||||
- _Boundary: ScoreModel_
|
||||
- _Depends: 1.1_
|
||||
|
||||
- [ ] 2.2 Emit the scan-level score artifact and manifest score block
|
||||
- [x] 2.2 Emit the scan-level score artifact and manifest score block
|
||||
- Emit a score artifact containing the project score, the maximum severity, the total penalty, a by-category breakdown, the out-of-scope dynamic-only count, the unmapped-CWE list, and the gate verdict; include the score block in the scan manifest.
|
||||
- Observable completion: a scan writes a score artifact with every required field and merges the score block into the manifest, verified against the documented artifact schema.
|
||||
- _Requirements: 5.5_
|
||||
- _Boundary: ScoreModel_
|
||||
- _Depends: 2.1_
|
||||
|
||||
- [ ] 2.3 Implement the two-condition CI score gate (advisory-first)
|
||||
- [x] 2.3 Implement the two-condition CI score gate (advisory-first)
|
||||
- Fail the build when any reported finding maps to policy severity 5 and has reachability `reachable`.
|
||||
- When configured as blocking, fail the build when the project score is below the minimum; until the score constants are calibrated against the corpus, compute and report the score advisory-first without blocking.
|
||||
- Observable completion: the hard gate fails on a severity-5 reachable finding while the score gate reports advisory-only by default, proven by gate tests.
|
||||
@@ -51,7 +51,7 @@ This plan splits the fused regex layer into three governed owners (rules detect,
|
||||
- _Boundary: CI Score Gate_
|
||||
- _Depends: 2.1_
|
||||
|
||||
- [ ] 2.4 Centralize ruleset/policy/score configuration
|
||||
- [x] 2.4 Centralize ruleset/policy/score configuration
|
||||
- Add configuration entries for the ruleset, policy, and score data locations plus the gate thresholds and emission floors used by later phases.
|
||||
- Observable completion: the scan reads the score constants and emission thresholds from configuration rather than hardcoded literals, confirmed by a config-driven test.
|
||||
- _Requirements: 5.8_
|
||||
|
||||
Reference in New Issue
Block a user