mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
openultrasast-scan: evidence ladder as verification.py defines it, with the levels nothing produces today; --mode prerequisites, --fail-on, recorded degradations, run artifacts, ousast mcp; retired [evidence] threshold language removed. openultrasast-triage: reasons verified against calibration.py; deterministic fusion; adjudications as labels, plane verdicts as features, rule changes through improve or a registered experiment; fix-audit merged as a fix-and-re-scan section. kiro: spec-quick and spec-batch removed; validate-gap and validate-design merged into spec-design (--gap, --review; the review gate stays); steering-custom merged into steering; core steering names overview.md, roadmap.md, safety-net.md; instrument rule in debug and verify-completion; repo gates named in validate-impl. AGENTS.md, README skills paragraph and tests/test_agent_skills.py (.agents/skills, two product skills, drift tests) updated. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
4.4 KiB
4.4 KiB
name, description
| name | description |
|---|---|
| kiro-verify-completion | Verify completion and success claims with fresh evidence. Use before claiming a task is complete, a fix works, tests pass, or a feature is ready for GO. |
kiro-verify-completion
<background_information> This skill prevents false completion claims. A task, fix, or feature is only complete when supported by fresh evidence that matches the scope of the claim. </background_information>
## When to Use- Before saying a task is complete
- Before saying a bug is fixed
- Before saying tests pass
- Before moving to the next task in autonomous execution
- Before reporting
GOfrom feature-level validation - Before trusting another subagent's success report
Do not use this skill for early planning or speculative status updates.
Inputs
Provide:
- The exact claim to verify
- Claim type:
TASKFIXTEST_OR_BUILDFEATURE_GO
- Validation commands discovered by the controller
- Fresh command output and exit codes
- Relevant task IDs, requirement IDs, and design refs where applicable
- For feature-level claims:
- requirements coverage status
- design alignment status
- integration status
- blocked task status
Outputs
Return one of:
VERIFIEDNOT_VERIFIEDMANUAL_VERIFY_REQUIRED
Also return:
- Claim reviewed
- Evidence used
- Scope/evidence mismatch, if any
Use the language specified in spec.json.
Gate Function
- Identify the exact claim.
- Identify the exact command or checklist that proves that claim.
- Require fresh evidence from the current code state.
- Check exit code, failure count, skipped scope, and missing coverage.
- Reject claims that are broader than the evidence.
- If mandatory validation cannot be completed, return
MANUAL_VERIFY_REQUIRED. - Verify the instrument before you believe the measurement: prove the tool read its input (it opened one file and printed the size; the stage list names what ran). A plausible zero -- "0 findings, 0 errors, exit 0" from a tree the tool never read, or a Joern script that "finished" before a JVM could start -- is a failed instrument, not evidence. For pytest, judge only by the printed exit code (
pytest -q -p no:cacheprovider < /dev/null; echo "exit=$?"), never by a summary line. - Only then allow the claim.
Claim-Specific Rules
TASK
Require:
- task-local verification evidence
- no unresolved blocking findings from review
- evidence aligned with the task boundary
FIX
Require:
- evidence that the original symptom is resolved
- no broader regressions in the relevant verification scope
TEST_OR_BUILD
Require:
- actual command output
- exit code
- no inference from unrelated checks
FEATURE_GO
Require:
- full test suite result
- runtime smoke boot result showing the built artifact reaches its first usable state
- requirements coverage assessment
- cross-task integration assessment
- design end-to-end alignment assessment
- blocked tasks assessment
A passing test suite alone is not enough for FEATURE_GO.
Stop / Escalate
Return MANUAL_VERIFY_REQUIRED when:
- No canonical validation command is known
- The required environment is unavailable
- A mandatory manual verification step cannot be executed
Return NOT_VERIFIED when:
- The command failed
- Evidence is stale
- Evidence is partial
- The claim exceeds the evidence
- The feature still has unresolved blocked tasks or uncovered requirements
Common Rationalizations
| Rationalization | Reality |
|---|---|
| “The subagent said it succeeded” | Reported success is not verification evidence. |
| “Tests passed earlier” | Fresh evidence only. |
| “Build should be fine because lint passed” | Lint does not prove build success. |
| “Tests passed and build succeeded, so it must run” | Type erasure, module loading, native ABI, and boot-time config issues can still fail at runtime. |
| “The feature is done because all tasks are checked off” | FEATURE_GO also requires coverage, integration, and design alignment. |
| “The scan found nothing, so the code is clean” | An unread input and clean code both report zero. Prove the tool read its input first. |
Output Format
## Verification Result
- STATUS: VERIFIED | NOT_VERIFIED | MANUAL_VERIFY_REQUIRED
- CLAIM_TYPE: TASK | FIX | TEST_OR_BUILD | FEATURE_GO
- CLAIM: <exact claim>
- EVIDENCE: <command/checklist and result>
- GAPS: <scope/evidence mismatch or missing validation>
- NOTES: <next action if not verified>