mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 12:39:28 +00:00
feat(verification): add evidence ladder verifier
This commit is contained in:
@@ -9,7 +9,7 @@ Checklist completion is not product completion. Each phase now carries a functio
|
||||
- `integrated`: the feature participates in the harness runtime, traces, evidence, and reports.
|
||||
- `production`: the feature is robust for real repositories, CI budgets, and analyst audit.
|
||||
|
||||
Current verified gate: `scaffold_quick_scan`.
|
||||
Current verified gate: `usable_harness_mvp` partial.
|
||||
|
||||
Do not report OpenUltraSAST as a working security harness until `usable_harness_mvp` is checked.
|
||||
|
||||
@@ -77,7 +77,7 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
|
||||
- [x] Quick scan is uv-managed and covered by unit tests.
|
||||
- [ ] Quick scan includes real LLM hunter execution. This is intentionally deferred to standard mode.
|
||||
- [ ] Quick scan includes SARIF output. This is intentionally deferred to Phase 9.
|
||||
- [ ] Quick scan includes enforced verifier decisions. This is intentionally deferred to Phase 6.
|
||||
- [x] Quick scan includes enforced verifier decisions.
|
||||
|
||||
## Gate: `usable_harness_mvp`
|
||||
|
||||
@@ -85,10 +85,10 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
|
||||
- [x] Event traces are emitted for every scan stage.
|
||||
- [x] Static mapping ingestion feeds FileTarget hints and verifier evidence candidates.
|
||||
- [x] Entry-point reachability feeds FileTarget reachability hints and ranking priority.
|
||||
- [ ] Evidence ladder state machine prevents invalid verified statuses.
|
||||
- [ ] Independent verifier runs on fixture findings without hunter reasoning.
|
||||
- [x] Evidence ladder state machine prevents invalid verified statuses.
|
||||
- [x] Independent verifier runs on fixture findings without hunter reasoning.
|
||||
- [ ] JSON, Markdown, SARIF, and manifest artifacts share finding IDs and evidence references.
|
||||
- [ ] A fixture scan demonstrates one accepted static-corroborated finding and one rejected false positive.
|
||||
- [x] A fixture scan demonstrates one accepted static-corroborated finding and one rejected false positive.
|
||||
|
||||
## Phase 5: Harness Runtime (`usable_harness_mvp` prerequisite)
|
||||
|
||||
@@ -117,12 +117,12 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
|
||||
|
||||
## Phase 6: Verification (`usable_harness_mvp` prerequisite)
|
||||
|
||||
- [ ] Define evidence ladder state machine.
|
||||
- [ ] Implement independent verifier prompt and schema.
|
||||
- [ ] Ensure verifier context excludes hunter reasoning.
|
||||
- [ ] Implement pro-case, counter-case, tie-breaker, and required-next-step fields.
|
||||
- [ ] Enforce that verified report status requires at least `static_corroboration`.
|
||||
- [ ] Add tests for invalid evidence transitions.
|
||||
- [x] Define evidence ladder state machine.
|
||||
- [x] Implement independent verifier prompt and schema.
|
||||
- [x] Ensure verifier context excludes hunter reasoning.
|
||||
- [x] Implement pro-case, counter-case, tie-breaker, and required-next-step fields.
|
||||
- [x] Enforce that verified report status requires at least `static_corroboration`.
|
||||
- [x] Add tests for invalid evidence transitions.
|
||||
|
||||
## Gate: `standard_security_harness`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user