mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Fable 5.1
16d9ef7110
openultrasast-scan: evidence ladder as verification.py defines it, with the levels nothing produces today; --mode prerequisites, --fail-on, recorded degradations, run artifacts, ousast mcp; retired [evidence] threshold language removed. openultrasast-triage: reasons verified against calibration.py; deterministic fusion; adjudications as labels, plane verdicts as features, rule changes through improve or a registered experiment; fix-audit merged as a fix-and-re-scan section. kiro: spec-quick and spec-batch removed; validate-gap and validate-design merged into spec-design (--gap, --review; the review gate stays); steering-custom merged into steering; core steering names overview.md, roadmap.md, safety-net.md; instrument rule in debug and verify-completion; repo gates named in validate-impl. AGENTS.md, README skills paragraph and tests/test_agent_skills.py (.agents/skills, two product skills, drift tests) updated. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
134 lines
4.4 KiB
Markdown
134 lines
4.4 KiB
Markdown
---
|
|
name: kiro-verify-completion
|
|
description: Verify completion and success claims with fresh evidence. Use before claiming a task is complete, a fix works, tests pass, or a feature is ready for GO.
|
|
---
|
|
|
|
# kiro-verify-completion
|
|
|
|
<background_information>
|
|
This skill prevents false completion claims. A task, fix, or feature is only complete when supported by fresh evidence that matches the scope of the claim.
|
|
</background_information>
|
|
|
|
<instructions>
|
|
## When to Use
|
|
|
|
- Before saying a task is complete
|
|
- Before saying a bug is fixed
|
|
- Before saying tests pass
|
|
- Before moving to the next task in autonomous execution
|
|
- Before reporting `GO` from feature-level validation
|
|
- Before trusting another subagent's success report
|
|
|
|
Do not use this skill for early planning or speculative status updates.
|
|
|
|
## Inputs
|
|
|
|
Provide:
|
|
- The exact claim to verify
|
|
- Claim type:
|
|
- `TASK`
|
|
- `FIX`
|
|
- `TEST_OR_BUILD`
|
|
- `FEATURE_GO`
|
|
- Validation commands discovered by the controller
|
|
- Fresh command output and exit codes
|
|
- Relevant task IDs, requirement IDs, and design refs where applicable
|
|
- For feature-level claims:
|
|
- requirements coverage status
|
|
- design alignment status
|
|
- integration status
|
|
- blocked task status
|
|
|
|
## Outputs
|
|
|
|
Return one of:
|
|
- `VERIFIED`
|
|
- `NOT_VERIFIED`
|
|
- `MANUAL_VERIFY_REQUIRED`
|
|
|
|
Also return:
|
|
- Claim reviewed
|
|
- Evidence used
|
|
- Scope/evidence mismatch, if any
|
|
|
|
Use the language specified in `spec.json`.
|
|
|
|
## Gate Function
|
|
|
|
1. Identify the exact claim.
|
|
2. Identify the exact command or checklist that proves that claim.
|
|
3. Require fresh evidence from the current code state.
|
|
4. Check exit code, failure count, skipped scope, and missing coverage.
|
|
5. Reject claims that are broader than the evidence.
|
|
6. If mandatory validation cannot be completed, return `MANUAL_VERIFY_REQUIRED`.
|
|
7. Verify the instrument before you believe the measurement: prove the tool read its input (it opened one file and printed the size; the stage list names what ran). A plausible zero -- "0 findings, 0 errors, exit 0" from a tree the tool never read, or a Joern script that "finished" before a JVM could start -- is a failed instrument, not evidence. For pytest, judge only by the printed exit code (`pytest -q -p no:cacheprovider < /dev/null; echo "exit=$?"`), never by a summary line.
|
|
8. Only then allow the claim.
|
|
|
|
## Claim-Specific Rules
|
|
|
|
### TASK
|
|
Require:
|
|
- task-local verification evidence
|
|
- no unresolved blocking findings from review
|
|
- evidence aligned with the task boundary
|
|
|
|
### FIX
|
|
Require:
|
|
- evidence that the original symptom is resolved
|
|
- no broader regressions in the relevant verification scope
|
|
|
|
### TEST_OR_BUILD
|
|
Require:
|
|
- actual command output
|
|
- exit code
|
|
- no inference from unrelated checks
|
|
|
|
### FEATURE_GO
|
|
Require:
|
|
- full test suite result
|
|
- runtime smoke boot result showing the built artifact reaches its first usable state
|
|
- requirements coverage assessment
|
|
- cross-task integration assessment
|
|
- design end-to-end alignment assessment
|
|
- blocked tasks assessment
|
|
|
|
A passing test suite alone is not enough for `FEATURE_GO`.
|
|
|
|
## Stop / Escalate
|
|
|
|
Return `MANUAL_VERIFY_REQUIRED` when:
|
|
- No canonical validation command is known
|
|
- The required environment is unavailable
|
|
- A mandatory manual verification step cannot be executed
|
|
|
|
Return `NOT_VERIFIED` when:
|
|
- The command failed
|
|
- Evidence is stale
|
|
- Evidence is partial
|
|
- The claim exceeds the evidence
|
|
- The feature still has unresolved blocked tasks or uncovered requirements
|
|
|
|
## Common Rationalizations
|
|
|
|
| Rationalization | Reality |
|
|
|---|---|
|
|
| “The subagent said it succeeded” | Reported success is not verification evidence. |
|
|
| “Tests passed earlier” | Fresh evidence only. |
|
|
| “Build should be fine because lint passed” | Lint does not prove build success. |
|
|
| “Tests passed and build succeeded, so it must run” | Type erasure, module loading, native ABI, and boot-time config issues can still fail at runtime. |
|
|
| “The feature is done because all tasks are checked off” | `FEATURE_GO` also requires coverage, integration, and design alignment. |
|
|
| “The scan found nothing, so the code is clean” | An unread input and clean code both report zero. Prove the tool read its input first. |
|
|
|
|
## Output Format
|
|
|
|
```md
|
|
## Verification Result
|
|
- STATUS: VERIFIED | NOT_VERIFIED | MANUAL_VERIFY_REQUIRED
|
|
- CLAIM_TYPE: TASK | FIX | TEST_OR_BUILD | FEATURE_GO
|
|
- CLAIM: <exact claim>
|
|
- EVIDENCE: <command/checklist and result>
|
|
- GAPS: <scope/evidence mismatch or missing validation>
|
|
- NOTES: <next action if not verified>
|
|
```
|
|
</instructions>
|