feat(verification): add evidence ladder verifier

This commit is contained in:
Marius Ciepluch
2026-06-18 09:59:28 +02:00
parent f8c690999a
commit 5519413c82
4 changed files with 286 additions and 11 deletions
+11 -11
View File
@@ -9,7 +9,7 @@ Checklist completion is not product completion. Each phase now carries a functio
- `integrated`: the feature participates in the harness runtime, traces, evidence, and reports.
- `production`: the feature is robust for real repositories, CI budgets, and analyst audit.
Current verified gate: `scaffold_quick_scan`.
Current verified gate: `usable_harness_mvp` partial.
Do not report OpenUltraSAST as a working security harness until `usable_harness_mvp` is checked.
@@ -77,7 +77,7 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
- [x] Quick scan is uv-managed and covered by unit tests.
- [ ] Quick scan includes real LLM hunter execution. This is intentionally deferred to standard mode.
- [ ] Quick scan includes SARIF output. This is intentionally deferred to Phase 9.
- [ ] Quick scan includes enforced verifier decisions. This is intentionally deferred to Phase 6.
- [x] Quick scan includes enforced verifier decisions.
## Gate: `usable_harness_mvp`
@@ -85,10 +85,10 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
- [x] Event traces are emitted for every scan stage.
- [x] Static mapping ingestion feeds FileTarget hints and verifier evidence candidates.
- [x] Entry-point reachability feeds FileTarget reachability hints and ranking priority.
- [ ] Evidence ladder state machine prevents invalid verified statuses.
- [ ] Independent verifier runs on fixture findings without hunter reasoning.
- [x] Evidence ladder state machine prevents invalid verified statuses.
- [x] Independent verifier runs on fixture findings without hunter reasoning.
- [ ] JSON, Markdown, SARIF, and manifest artifacts share finding IDs and evidence references.
- [ ] A fixture scan demonstrates one accepted static-corroborated finding and one rejected false positive.
- [x] A fixture scan demonstrates one accepted static-corroborated finding and one rejected false positive.
## Phase 5: Harness Runtime (`usable_harness_mvp` prerequisite)
@@ -117,12 +117,12 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
## Phase 6: Verification (`usable_harness_mvp` prerequisite)
- [ ] Define evidence ladder state machine.
- [ ] Implement independent verifier prompt and schema.
- [ ] Ensure verifier context excludes hunter reasoning.
- [ ] Implement pro-case, counter-case, tie-breaker, and required-next-step fields.
- [ ] Enforce that verified report status requires at least `static_corroboration`.
- [ ] Add tests for invalid evidence transitions.
- [x] Define evidence ladder state machine.
- [x] Implement independent verifier prompt and schema.
- [x] Ensure verifier context excludes hunter reasoning.
- [x] Implement pro-case, counter-case, tie-breaker, and required-next-step fields.
- [x] Enforce that verified report status requires at least `static_corroboration`.
- [x] Add tests for invalid evidence transitions.
## Gate: `standard_security_harness`