mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
chore(skills): apply the 2026-10-02 review -- current CLI and evidence, merges, removals, steering names
openultrasast-scan: evidence ladder as verification.py defines it, with the levels nothing produces today; --mode prerequisites, --fail-on, recorded degradations, run artifacts, ousast mcp; retired [evidence] threshold language removed. openultrasast-triage: reasons verified against calibration.py; deterministic fusion; adjudications as labels, plane verdicts as features, rule changes through improve or a registered experiment; fix-audit merged as a fix-and-re-scan section. kiro: spec-quick and spec-batch removed; validate-gap and validate-design merged into spec-design (--gap, --review; the review gate stays); steering-custom merged into steering; core steering names overview.md, roadmap.md, safety-net.md; instrument rule in debug and verify-completion; repo gates named in validate-impl. AGENTS.md, README skills paragraph and tests/test_agent_skills.py (.agents/skills, two product skills, drift tests) updated. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
co-authored by
Claude Fable 5.1
Claude Fable 5.1
parent
5010b15ffc
commit
16d9ef7110
@@ -55,6 +55,14 @@ Extract:
|
||||
- The command that produced the failure
|
||||
- Whether the failure is deterministic or intermittent
|
||||
|
||||
### 1b. Verify the Instrument Before You Believe the Measurement
|
||||
Every wrong diagnosis recorded in this project had the same shape: the input was unreadable and the tool reported a plausible zero (`file_exists()` false, the parser wrote nothing, the frontend exited 0), and "no errors, 0 findings" read exactly like a clean result. Before trusting any measurement in the failure report:
|
||||
- **Prove the tool read its input.** Make it open one file and print the size. A zero from an unread tree and a zero from clean code are indistinguishable afterwards.
|
||||
- **Distrust a number that is too good.** A Joern script "finishing" in 0.5 s when a JVM takes 15 s to start is a failed launch. A harness must fail loudly with the exit code.
|
||||
- **Prefer the shipped environment** (the Docker image has a native `php`); local wrappers that proxy stdio are not valid instruments for `php2cpg`.
|
||||
- **When a result contradicts a working baseline, suspect the instrument first.**
|
||||
A plausible zero is a failed instrument until proven otherwise; classify it as `CONFIG_GAP` or `RUNTIME_MISMATCH`, not as `LOGIC_ERROR` in the code under test.
|
||||
|
||||
### 2. Inspect Local Runtime and Repository State
|
||||
Inspect the repository for local evidence:
|
||||
- `package.json`, `pyproject.toml`, `go.mod`, `Makefile`, `README*`
|
||||
@@ -134,6 +142,7 @@ If the issue is fixable by repo changes inside the current task plan, do not esc
|
||||
| “Let’s try a few fixes” | Multi-fix guessing hides root cause. |
|
||||
| “The spec is probably wrong, I’ll adapt it” | Spec conflicts must be surfaced explicitly. |
|
||||
| “The docs search is optional” | For runtime/dependency issues, docs and version issues often contain the shortest path to root cause. |
|
||||
| “It ran clean: no errors, zero findings” | A tool that never read its input reports the same. Prove the read (one file, its size) before believing the zero. |
|
||||
|
||||
## Output Format
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ description: Entry point for new work. Determines the best action path or work d
|
||||
Gather **only metadata** to determine the action path. Do NOT read full file contents yet.
|
||||
|
||||
- **Specs inventory**: Scan `.kiro/specs/*/spec.json` for `name`, `phase` fields and `approvals` status. Note feature names and their current status.
|
||||
- **Steering existence**: Check which files exist in `.kiro/steering/` (product.md, tech.md, structure.md, roadmap.md). Do NOT read their contents yet.
|
||||
- **Steering existence**: Check which files exist in `.kiro/steering/` (core: overview.md, roadmap.md, safety-net.md; custom files may exist too). Do NOT read their contents yet.
|
||||
- **Roadmap check**: If `.kiro/steering/roadmap.md` exists, read it. This contains project-level context (approach, scope, constraints, spec list) from a previous discovery session. Use it to restore project context.
|
||||
- **Top-level structure**: List the project root directory to note key directories and files. Do NOT recurse into subdirectories.
|
||||
|
||||
@@ -60,7 +60,7 @@ For Path A/B, recommend the next action and stop.
|
||||
**Only for Path C, D, and E.** Now load the context needed for discovery.
|
||||
|
||||
**In main context** (essential for dialogue with user):
|
||||
- **Steering documents**: Read product.md and tech.md (if they exist) for project goals, constraints, and tech stack
|
||||
- **Steering documents**: Read overview.md and safety-net.md (if they exist) for project goals, constraints, and the current state of the stack
|
||||
- **Relevant specs**: If the request is adjacent to an existing spec, read that spec's requirements.md to understand boundaries and avoid overlap
|
||||
|
||||
**Delegate to sub-agent** (keeps exploration out of main context):
|
||||
@@ -198,7 +198,7 @@ Use this roadmap structure:
|
||||
- [ ] feature-c -- [one-line description]. Dependencies: feature-a, feature-b
|
||||
```
|
||||
|
||||
Then write `.kiro/specs/<feature>/brief.md` for **every** feature listed under `## Specs (dependency order)` using the Path C brief format. This enables parallel spec creation via `/kiro-spec-batch`.
|
||||
Then write `.kiro/specs/<feature>/brief.md` for **every** feature listed under `## Specs (dependency order)` using the Path C brief format. Each spec is then created one at a time with `/kiro-spec-init`, in dependency order.
|
||||
|
||||
**For Path E (mixed decomposition)**:
|
||||
|
||||
@@ -219,7 +219,7 @@ Use the same roadmap structure as Path D, plus these additional sections:
|
||||
```
|
||||
|
||||
Path E rules:
|
||||
- Keep `## Specs (dependency order)` reserved for **new specs only** so `/kiro-spec-batch` can still parse it unchanged
|
||||
- Keep `## Specs (dependency order)` reserved for **new specs only** so the list stays a plain, parseable creation order
|
||||
- Record existing-spec extensions under `## Existing Spec Updates`
|
||||
- Record true no-spec work under `## Direct Implementation Candidates`
|
||||
- Write `brief.md` only for the **new specs** listed under `## Specs (dependency order)`
|
||||
@@ -235,15 +235,12 @@ Suggest the next command and stop. Do NOT automatically run downstream spec gene
|
||||
|
||||
- Path A: `/kiro-spec-requirements {feature}` to update the existing spec
|
||||
- Path B: Recommend direct implementation without creating a spec
|
||||
- Path C: Default to `/kiro-spec-init <feature-name>`
|
||||
- Optional fast path: `/kiro-spec-quick <feature-name>` when the user explicitly wants to continue immediately
|
||||
- Path D: Default to `/kiro-spec-batch` (creates all specs in parallel based on roadmap.md dependency order)
|
||||
- Optional cautious path: `/kiro-spec-init <first-feature-name>` when the user wants to validate the first slice before batching the rest
|
||||
- Path C: `/kiro-spec-init <feature-name>`, then the step-by-step phases (requirements, design, tasks), each human-reviewed
|
||||
- Path D: `/kiro-spec-init <first-feature-name>` for the first spec in roadmap.md dependency order; validate that slice before the next one
|
||||
- Path E: Choose the next command based on the new-spec portion of the decomposition
|
||||
- If there is exactly one new spec: `/kiro-spec-init <new-feature-name>`
|
||||
- If there are multiple new specs: `/kiro-spec-batch`
|
||||
- `/kiro-spec-init <new-feature-name>` for the first (or only) new spec, the rest in dependency order afterwards
|
||||
- Also note which existing specs should be revisited with `/kiro-spec-requirements <feature>`
|
||||
- Re-entry: `/kiro-spec-init <next-feature-name>` or `/kiro-spec-batch` if multiple specs remain
|
||||
- Re-entry: `/kiro-spec-init <next-feature-name>` for the next spec that remains
|
||||
|
||||
If the decomposition contains only existing-spec updates plus direct implementation candidates, do NOT use Path E. Prefer Path A when one existing spec is the clear home, or recommend the existing-spec update plus direct implementation work without creating roadmap entries.
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ You operate in two modes:
|
||||
If steering/spec context is already available from conversation, skip redundant file reads.
|
||||
Otherwise, load all necessary context:
|
||||
- `.kiro/specs/{feature}/spec.json`, `requirements.md`, `design.md`, `tasks.md`
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
|
||||
- Additional steering files only when directly relevant to the selected task's boundary, runtime prerequisites, integrations, domain rules, security/performance constraints, or team conventions that affect implementation or validation
|
||||
- Relevant local agent skills or playbooks only when they clearly match the task's host environment or use case; read the specific artifact(s) you need, not entire directories
|
||||
|
||||
|
||||
@@ -1,165 +0,0 @@
|
||||
---
|
||||
name: kiro-spec-batch
|
||||
description: Create complete specs (requirements, design, tasks) for all features in roadmap.md using parallel sub-agent dispatch by dependency wave.
|
||||
---
|
||||
|
||||
|
||||
# Spec Batch
|
||||
|
||||
<background_information>
|
||||
- **Success Criteria**:
|
||||
- All features have complete spec files (spec.json, requirements.md, design.md, tasks.md)
|
||||
- Dependency ordering respected (upstream specs complete before downstream)
|
||||
- Independent features processed in parallel via sub-agent dispatch
|
||||
- Cross-spec consistency verified (data models, interfaces, naming)
|
||||
- Mixed roadmap context understood without breaking `## Specs (dependency order)` parsing
|
||||
- Controller context stays lightweight (sub-agents do the heavy work)
|
||||
</background_information>
|
||||
|
||||
<instructions>
|
||||
|
||||
## Step 1: Read Roadmap and Validate
|
||||
|
||||
1. Read `.kiro/steering/roadmap.md`
|
||||
2. Parse the `## Specs (dependency order)` section to extract:
|
||||
- Feature names
|
||||
- One-line descriptions
|
||||
- Dependencies for each feature
|
||||
- Completion status (`[x]` = done, `[ ]` = pending)
|
||||
3. If present, also read for context:
|
||||
- `## Existing Spec Updates`
|
||||
- `## Direct Implementation Candidates`
|
||||
Do not include these in dependency-wave execution; they are awareness-only inputs for sequencing and consistency review.
|
||||
4. For each pending feature in `## Specs (dependency order)`, verify `.kiro/specs/<feature>/brief.md` exists
|
||||
5. If any brief.md is missing, stop and report: "Missing brief.md for: [list]. Run `/kiro-discovery` to generate briefs first."
|
||||
|
||||
## Step 2: Build Dependency Waves
|
||||
|
||||
Group pending features into waves based on dependencies:
|
||||
|
||||
- **Wave 1**: Features with no dependencies (or all dependencies already completed `[x]`)
|
||||
- **Wave 2**: Features whose dependencies are all in Wave 1 or already completed
|
||||
- **Wave N**: Features whose dependencies are all in earlier waves or already completed
|
||||
|
||||
Display the execution plan:
|
||||
```
|
||||
Spec Batch Plan:
|
||||
Wave 1 (parallel): app-foundation
|
||||
Wave 2 (parallel): block-editor, page-management
|
||||
Wave 3 (parallel): sidebar-navigation, database-views
|
||||
Wave 4 (parallel): cli-integration
|
||||
Total: 6 specs across 4 waves
|
||||
```
|
||||
|
||||
If roadmap contains `## Existing Spec Updates` or `## Direct Implementation Candidates`, mention them separately as non-batch items so the user can see the whole decomposition.
|
||||
|
||||
## Step 3: Execute Waves
|
||||
|
||||
For each wave, dispatch all features in the wave as **parallel sub-agents**.
|
||||
|
||||
**For each feature in the wave**, spawn a sub-agent with this task:
|
||||
|
||||
```
|
||||
Create a complete specification for feature "{feature-name}".
|
||||
|
||||
1. Read the brief at .kiro/specs/{feature-name}/brief.md for feature context
|
||||
2. Read the roadmap at .kiro/steering/roadmap.md for project context
|
||||
3. Execute the full spec pipeline. For each phase, read the corresponding skill's SKILL.md for complete instructions (templates, rules, review gates):
|
||||
a. Initialize: Read .agent/skills/kiro-spec-init/SKILL.md, then create spec.json and requirements.md
|
||||
b. Generate requirements: Read .agent/skills/kiro-spec-requirements/SKILL.md, then follow its steps
|
||||
c. Generate design: Read .agent/skills/kiro-spec-design/SKILL.md, then follow its steps
|
||||
d. Generate tasks: Read .agent/skills/kiro-spec-tasks/SKILL.md, then follow its steps
|
||||
4. Set all approvals to true in spec.json (auto-approve mode, equivalent of -y flag)
|
||||
5. Report completion with file list and task count
|
||||
```
|
||||
|
||||
If multi-agent is not available, execute features in the wave sequentially.
|
||||
|
||||
**After all sub-agents in the wave complete**:
|
||||
1. Verify each feature has: spec.json, requirements.md, design.md, tasks.md
|
||||
2. If any feature failed, report the error and continue with features that succeeded
|
||||
3. Display wave completion: "Wave N complete: [features]. Files verified."
|
||||
4. Proceed to next wave
|
||||
|
||||
## Step 4: Cross-Spec Review
|
||||
|
||||
After all waves complete, spawn a **single sub-agent** for cross-spec consistency review. This is the highest-value quality gate -- it catches issues that per-spec review gates cannot.
|
||||
|
||||
**Sub-agent task**:
|
||||
|
||||
Read ALL generated specs and check for consistency across the entire project:
|
||||
- `.kiro/specs/*/design.md` (primary: contains interfaces, data models, architecture)
|
||||
- `.kiro/specs/*/requirements.md` (for scope and acceptance criteria)
|
||||
- `.kiro/specs/*/tasks.md` (for boundary annotations only -- read _Boundary:_ lines, skip task descriptions)
|
||||
- `.kiro/steering/roadmap.md`
|
||||
|
||||
Reading priority: Focus on design.md files (they contain interfaces, data models, architecture). For requirements.md, focus on section headings and acceptance criteria. For tasks.md, focus on _Boundary:_ annotations.
|
||||
|
||||
Check:
|
||||
1. **Data model consistency**: Same entities defined consistently across specs (field names, types, relationships)
|
||||
2. **Interface alignment**: Where spec A outputs what spec B consumes, do contracts match exactly?
|
||||
3. **No duplicate functionality**: Any capability specified in more than one spec?
|
||||
4. **Dependency completeness**: Every design.md references correct upstream specs? Implicit dependencies not in roadmap?
|
||||
5. **Naming conventions**: Component names, file paths, API routes, table names consistent across specs?
|
||||
6. **Shared infrastructure**: Shared concerns (auth, error handling, logging) handled in one spec and correctly referenced?
|
||||
7. **Task boundary alignment**: Task _Boundary:_ annotations partition codebase cleanly? No files claimed by multiple specs?
|
||||
8. **Roadmap boundary continuity**: If roadmap includes `Existing Spec Updates` or `Direct Implementation Candidates`, do the generated new specs avoid absorbing that work by accident?
|
||||
9. **Architecture boundary integrity**: Do the specs preserve clean responsibility seams, avoid shared ownership, keep dependency direction coherent, and include enough revalidation triggers to catch downstream impact?
|
||||
10. **Change-friendly decomposition**: Has any spec absorbed multiple independent seams that should probably be split instead of kept together?
|
||||
|
||||
Output: CONSISTENT areas + ISSUES with (which specs, what's inconsistent, suggested fix).
|
||||
|
||||
**After the review sub-agent returns**:
|
||||
- **Critical/important issues found**: Dispatch fix sub-agents for each affected spec to apply the suggested fixes. If the issue is really a decomposition problem (for example boundary overlap or one spec carrying multiple independent seams), stop and return to roadmap/discovery instead of papering over it locally. Re-run cross-spec review after fixes (max 3 remediation rounds).
|
||||
- **Minor issues only**: Report them for user awareness, proceed to Step 5.
|
||||
- **No issues**: Proceed to Step 5.
|
||||
|
||||
## Step 5: Finalize
|
||||
|
||||
1. Scan `.kiro/specs/*/tasks.md` to verify all specs exist
|
||||
2. For each completed spec, read spec.json to confirm phase and approvals
|
||||
3. Update roadmap.md: mark completed specs as `[x]`
|
||||
4. If roadmap.md includes `Existing Spec Updates` or `Direct Implementation Candidates`, leave them untouched and mention them as remaining follow-up items unless already explicitly completed elsewhere
|
||||
|
||||
Display final summary:
|
||||
```
|
||||
Spec Batch Complete:
|
||||
✓ app-foundation: X requirements, Y design components, Z tasks
|
||||
✓ block-editor: ...
|
||||
✓ page-management: ...
|
||||
...
|
||||
Total: N specs created, M tasks generated
|
||||
Cross-spec review: PASSED / N issues found (M fixed)
|
||||
Existing spec updates pending: <count or none>
|
||||
Direct implementation candidates pending: <count or none>
|
||||
|
||||
Next: Review generated specs, then start implementation with /kiro-impl <feature>
|
||||
```
|
||||
|
||||
</instructions>
|
||||
|
||||
## Critical Constraints
|
||||
- **Controller stays lightweight**: Only read roadmap.md and brief.md existence checks in main context. All spec generation happens in sub-agents.
|
||||
- **Wave ordering is strict**: Never start a wave until all features in previous waves are complete.
|
||||
- **Parallel within waves**: All features in the same wave should be dispatched in parallel if multi-agent is available.
|
||||
- **No partial waves**: If a feature in a wave fails, still complete the other features in that wave before reporting.
|
||||
- **Skip completed specs**: Features with `[x]` in roadmap.md or existing tasks.md are skipped.
|
||||
- **`## Specs (dependency order)` remains authoritative for batch execution**: Other roadmap sections are context, not wave inputs.
|
||||
|
||||
## Safety & Fallback
|
||||
|
||||
**Sub-agent failure**:
|
||||
- Log the error, skip the failed feature
|
||||
- Continue with remaining features in the wave
|
||||
- Report failed features in the summary
|
||||
- Suggest: "Run `/kiro-spec-quick <feature> --auto` manually for failed features."
|
||||
|
||||
**Circular dependencies**:
|
||||
- If dependency graph has cycles, report the cycle and stop
|
||||
- Suggest: "Fix dependency ordering in roadmap.md"
|
||||
|
||||
**Roadmap not found**:
|
||||
- Stop and report: "No roadmap.md found. Run `/kiro-discovery` first."
|
||||
|
||||
**All specs already complete**:
|
||||
- Report: "All specs in roadmap.md are already complete. Nothing to do."
|
||||
@@ -1,13 +1,18 @@
|
||||
---
|
||||
name: kiro-spec-design
|
||||
description: Create comprehensive technical design for a specification
|
||||
description: Create comprehensive technical design for a specification; optionally run the implementation gap analysis first (--gap) or an interactive design review of an existing design (--review)
|
||||
metadata:
|
||||
shared-rules: "design-principles.md, design-discovery-full.md, design-discovery-light.md, design-synthesis.md, design-review-gate.md"
|
||||
shared-rules: "design-principles.md, design-discovery-full.md, design-discovery-light.md, design-synthesis.md, design-review-gate.md, gap-analysis.md, design-review.md"
|
||||
---
|
||||
|
||||
|
||||
# Technical Design Generator
|
||||
|
||||
**Arguments**: `$1` is the feature. `$2` may be `-y` (auto-approve requirements), `--gap` (run the
|
||||
optional gap analysis of Step 1b before designing) or `--review` (run only the optional interactive
|
||||
design review of Step 7 on an existing `design.md`). The design review gate of Step 5 always runs
|
||||
when a design is generated; the optional steps never replace it.
|
||||
|
||||
<background_information>
|
||||
- **Success Criteria**:
|
||||
- All requirements mapped to technical components with clear interfaces
|
||||
@@ -24,8 +29,8 @@ metadata:
|
||||
|
||||
**Read all necessary context**:
|
||||
- `.kiro/specs/$1/spec.json`, `requirements.md`, `design.md` (if exists)
|
||||
- `.kiro/specs/$1/research.md` (if exists, contains gap analysis from `/kiro-validate-gap`)
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- `.kiro/specs/$1/research.md` (if exists, contains the gap analysis from Step 1b or an earlier run)
|
||||
- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (this repository's core steering; custom steering files may also exist in `.kiro/steering/`)
|
||||
- Additional steering files only when directly relevant to requirement coverage, architecture boundaries, integrations, runtime prerequisites, security/performance constraints, or team conventions that affect implementation readiness
|
||||
- `.kiro/settings/templates/specs/design.md` for document structure
|
||||
- Read `rules/design-principles.md` from this skill's directory for design principles
|
||||
@@ -35,6 +40,16 @@ metadata:
|
||||
- If `-y` flag provided ($2 == "-y"): Auto-approve requirements in spec.json
|
||||
- Otherwise: Verify approval status (stop if unapproved, see Safety & Fallback)
|
||||
|
||||
### Step 1b (optional, `--gap`): Implementation Gap Analysis
|
||||
|
||||
Recommended for brownfield features; skip for greenfield. Analyze the gap between the approved requirements and the existing codebase to inform the design, following `rules/gap-analysis.md` from this skill's directory.
|
||||
|
||||
1. Read `rules/gap-analysis.md` for the analysis framework.
|
||||
2. Research in parallel where possible (sub-agents if available, otherwise sequentially): existing implementations, architecture patterns, integration points and extension possibilities; external dependency compatibility and version constraints when needed; the requirements, core steering, task-relevant extra steering and relevant local agent skills or playbooks.
|
||||
3. Execute the gap analysis: Grep and Read the codebase, use WebSearch/WebFetch for external dependencies if needed, evaluate multiple implementation approaches (extend / new / hybrid) with trade-offs, flag areas needing further research. Provide information and options, not final implementation choices.
|
||||
4. Write the analysis to `.kiro/specs/$1/research.md` (append after a `---` rule if the file exists; never overwrite earlier research) and read it back to verify the write. Use the language specified in spec.json.
|
||||
5. Unapproved requirements are a warning here, not a stop: gap analysis can inform requirement revisions. Then continue with Step 2, or stop and report if the user asked for the gap analysis alone.
|
||||
|
||||
### Step 2: Discovery & Analysis
|
||||
|
||||
**Critical: This phase ensures design is based on complete, accurate information.**
|
||||
@@ -137,6 +152,15 @@ After all findings return, synthesize in main context before proceeding.
|
||||
- Set `approvals.requirements.approved: true`
|
||||
- Update `updated_at` timestamp
|
||||
|
||||
### Step 7 (optional, `--review`): Interactive Design Review
|
||||
|
||||
A human-paced quality review of an existing `.kiro/specs/$1/design.md`, in addition to the automatic gate of Step 5. Follow `rules/design-review.md` from this skill's directory.
|
||||
|
||||
1. Gather context: spec.json, requirements.md, design.md, core steering, review-relevant extra steering and local playbooks; survey existing codebase patterns, naming conventions and component structure for reference (parallel sub-agents if available).
|
||||
2. Review: Analysis -> Critical Issues -> Strengths -> GO/NO-GO. Limit to the 3 most important concerns; recognize 1-2 strengths; engage interactively (clarifying questions, alternatives); leverage conversation history for requirements context and user intent. Quality assurance, not perfection seeking; accept acceptable risk; every suggestion must be implementable.
|
||||
3. Decide: a clear GO/NO-GO with rationale. GO: `/kiro-spec-tasks $1` (or `-y`). NO-GO: address the critical issues, re-run `/kiro-spec-design $1`, then `/kiro-spec-design $1 --review` again.
|
||||
4. If design.md does not exist, stop: "Run `/kiro-spec-design $1` first to generate design document". If the design phase is not marked generated in spec.json, warn but proceed.
|
||||
|
||||
## Critical Constraints
|
||||
- **Type Safety**:
|
||||
- Enforce strong typing aligned with the project's technology stack.
|
||||
@@ -199,7 +223,7 @@ Provide brief summary in the language specified in spec.json:
|
||||
|
||||
**If Design Approved**:
|
||||
- Review generated design at `.kiro/specs/$1/design.md`
|
||||
- **Optional**: Run `/kiro-validate-design $1` for interactive quality review
|
||||
- **Optional**: Run `/kiro-spec-design $1 --review` for the interactive quality review (Step 7)
|
||||
- Then `/kiro-spec-tasks $1 -y` to generate implementation tasks
|
||||
|
||||
**If Modifications Needed**:
|
||||
|
||||
@@ -1,256 +0,0 @@
|
||||
---
|
||||
name: kiro-spec-quick
|
||||
description: Quick spec generation with interactive or automatic mode
|
||||
---
|
||||
|
||||
# Quick Spec Generator
|
||||
|
||||
<instructions>
|
||||
## CRITICAL: Automatic Mode Execution Rules
|
||||
|
||||
**If `--auto` flag is present in `$ARGUMENTS`, you are in AUTOMATIC MODE.**
|
||||
|
||||
In Automatic Mode:
|
||||
- Execute ALL 4 phases in a continuous loop without stopping
|
||||
- Display progress after each phase (e.g., "Phase 1/4 complete: spec initialized")
|
||||
- IGNORE any "Next Step" messages from Phase 2-4 (they are for standalone usage)
|
||||
- After Phase 4, run the final sanity review before exiting
|
||||
- Stop ONLY after the sanity review completes or if error occurs
|
||||
|
||||
---
|
||||
|
||||
## Core Task
|
||||
Execute 4 spec phases sequentially. In automatic mode, execute all phases without stopping. In interactive mode, prompt user for approval between phases.
|
||||
|
||||
Before claiming quick generation is complete, run one lightweight sanity review over the generated requirements, design, and tasks. If the host supports fresh sub-agents, use one. Otherwise run the sanity review inline.
|
||||
|
||||
## Execution Steps
|
||||
|
||||
### Step 1: Parse Arguments and Initialize
|
||||
|
||||
Parse `$ARGUMENTS`:
|
||||
- If contains `--auto`: **Automatic Mode** (execute all 4 phases)
|
||||
- Otherwise: **Interactive Mode** (prompt at each phase)
|
||||
- Extract description (remove `--auto` flag if present)
|
||||
|
||||
Example:
|
||||
```
|
||||
"User profile with avatar upload --auto" → mode=automatic, description="User profile with avatar upload"
|
||||
"User profile feature" → mode=interactive, description="User profile feature"
|
||||
```
|
||||
|
||||
Display mode banner and proceed to Step 2.
|
||||
|
||||
### Step 2: Execute Phase Loop
|
||||
|
||||
Execute these 4 phases in order:
|
||||
|
||||
---
|
||||
|
||||
#### Phase 1: Initialize Spec (Direct Implementation)
|
||||
|
||||
**Core Logic**:
|
||||
|
||||
1. **Check for Brief**:
|
||||
- If `.kiro/specs/{feature-name}/brief.md` exists (created by `/kiro-discovery`), read it for discovery context (problem, approach, scope, constraints)
|
||||
- Use brief content as the project description instead of `$ARGUMENTS`
|
||||
|
||||
2. **Generate Feature Name**:
|
||||
- Convert description to kebab-case
|
||||
- Example: "User profile with avatar upload" → "user-profile-avatar-upload"
|
||||
- Keep name concise (2-4 words ideally)
|
||||
|
||||
3. **Check Uniqueness**:
|
||||
- Use Glob to check `.kiro/specs/*/`
|
||||
- If directory exists with only `brief.md` (no `spec.json`), use that directory (discovery created it)
|
||||
- Otherwise if feature name exists, append `-2`, `-3`, etc.
|
||||
|
||||
4. **Create Directory**:
|
||||
- Use Bash: `mkdir -p .kiro/specs/{feature-name}` (skip if already exists from discovery)
|
||||
|
||||
5. **Initialize Files from Templates**:
|
||||
|
||||
a. Read templates:
|
||||
```
|
||||
- .kiro/settings/templates/specs/init.json
|
||||
- .kiro/settings/templates/specs/requirements-init.md
|
||||
```
|
||||
|
||||
b. Replace placeholders:
|
||||
```
|
||||
{{FEATURE_NAME}} → feature-name
|
||||
{{TIMESTAMP}} → current ISO 8601 timestamp (use `date -u +"%Y-%m-%dT%H:%M:%SZ"`)
|
||||
{{PROJECT_DESCRIPTION}} → description
|
||||
en → language code (detect from user's input language, default to `en`)
|
||||
```
|
||||
|
||||
c. Write files using Write tool:
|
||||
```
|
||||
- .kiro/specs/{feature-name}/spec.json
|
||||
- .kiro/specs/{feature-name}/requirements.md
|
||||
```
|
||||
|
||||
6. **Output Progress**: "Phase 1/4 complete: Spec initialized at .kiro/specs/{feature-name}/"
|
||||
|
||||
**Automatic Mode**: IMMEDIATELY continue to Phase 2.
|
||||
|
||||
**Interactive Mode**: Prompt "Continue to requirements generation? (yes/no)"
|
||||
- If "no": Stop, show current state
|
||||
- If "yes": Continue to Phase 2
|
||||
|
||||
---
|
||||
|
||||
#### Phase 2: Generate Requirements
|
||||
|
||||
Invoke `/kiro-spec-requirements {feature-name}`.
|
||||
|
||||
Wait for completion. IGNORE any "Next Step" message (it is for standalone usage).
|
||||
|
||||
**Output Progress**: "Phase 2/4 complete: Requirements generated"
|
||||
|
||||
**Automatic Mode**: IMMEDIATELY continue to Phase 3.
|
||||
|
||||
**Interactive Mode**: Prompt "Continue to design generation? (yes/no)"
|
||||
- If "no": Stop, show current state
|
||||
- If "yes": Continue to Phase 3
|
||||
|
||||
---
|
||||
|
||||
#### Phase 3: Generate Design
|
||||
|
||||
Invoke `/kiro-spec-design {feature-name} -y`. The `-y` flag auto-approves requirements.
|
||||
|
||||
Wait for completion. IGNORE any "Next Step" message.
|
||||
|
||||
**Output Progress**: "Phase 3/4 complete: Design generated"
|
||||
|
||||
**Automatic Mode**: IMMEDIATELY continue to Phase 4.
|
||||
|
||||
**Interactive Mode**: Prompt "Continue to tasks generation? (yes/no)"
|
||||
- If "no": Stop, show current state
|
||||
- If "yes": Continue to Phase 4
|
||||
|
||||
---
|
||||
|
||||
#### Phase 4: Generate Tasks
|
||||
|
||||
Invoke `/kiro-spec-tasks {feature-name} -y`. The `-y` flag auto-approves requirements, design, and tasks.
|
||||
|
||||
Wait for completion.
|
||||
|
||||
**Output Progress**: "Phase 4/4 complete: Tasks generated"
|
||||
|
||||
#### Final Sanity Review
|
||||
|
||||
After Phase 4, run a lightweight sanity review before claiming completion.
|
||||
|
||||
- Review `requirements.md`, `design.md`, and `tasks.md` directly from disk. If `brief.md` exists, use it only as supporting context.
|
||||
- Prefer a fresh review sub-agent when the host supports it. Pass only file paths and the review objective; the reviewer should read the generated files itself.
|
||||
- Review focus:
|
||||
- Do requirements, design, and tasks tell a coherent story?
|
||||
- Are there obvious contradictions, missing prerequisites, or missing task coverage for required design work?
|
||||
- Are `_Depends:_`, `_Boundary:_`, and `(P)` markers plausible for implementation?
|
||||
- If the review finds only task-plan-local issues, repair or update the generated `tasks.md` once, then re-run the sanity review.
|
||||
- If the review finds a real requirements/design gap or contradiction, stop and report follow-up instead of claiming the quick spec is implementation-ready.
|
||||
|
||||
**All 4 phases plus sanity review complete.**
|
||||
|
||||
Output final completion summary (see Output Description section) and exit.
|
||||
|
||||
---
|
||||
|
||||
## Important Constraints
|
||||
|
||||
### Error Handling
|
||||
- Any phase failure stops the workflow
|
||||
- Display error and current state
|
||||
- Suggest manual recovery command
|
||||
|
||||
</instructions>
|
||||
|
||||
## Output Description
|
||||
|
||||
### Mode Banners
|
||||
|
||||
**Interactive Mode**:
|
||||
```
|
||||
Quick Spec Generation (Interactive Mode)
|
||||
|
||||
You will be prompted at each phase.
|
||||
Note: Skips gap analysis and design validation.
|
||||
```
|
||||
|
||||
**Automatic Mode**:
|
||||
```
|
||||
Quick Spec Generation (Automatic Mode)
|
||||
|
||||
All phases execute automatically without prompts.
|
||||
Note: Skips optional validations (gap analysis, design review) and user approval prompts. Internal review gates still run.
|
||||
Final sanity review still runs.
|
||||
```
|
||||
|
||||
### Intermediate Output
|
||||
|
||||
After each phase, show brief progress:
|
||||
```
|
||||
Spec initialized at .kiro/specs/{feature}/
|
||||
Requirements generated → Continuing to design...
|
||||
Design generated → Continuing to tasks...
|
||||
```
|
||||
|
||||
### Final Completion Summary
|
||||
|
||||
Provide output in the language specified in `spec.json`:
|
||||
|
||||
```
|
||||
Quick Spec Generation Complete!
|
||||
|
||||
## Generated Files:
|
||||
- .kiro/specs/{feature}/spec.json
|
||||
- .kiro/specs/{feature}/requirements.md ({X} requirements)
|
||||
- .kiro/specs/{feature}/design.md ({Y} components, {Z} endpoints)
|
||||
- .kiro/specs/{feature}/tasks.md ({N} tasks)
|
||||
|
||||
Quick generation skipped:
|
||||
- `/kiro-validate-gap` - Gap analysis (integration check)
|
||||
- `/kiro-validate-design` - Design review (architecture validation)
|
||||
|
||||
Sanity review: PASSED | FOLLOW-UP REQUIRED
|
||||
|
||||
## Next Steps:
|
||||
1. Review generated specs (especially design.md)
|
||||
2. Optional validation:
|
||||
- `/kiro-validate-gap {feature}` - Check integration with existing codebase
|
||||
- `/kiro-validate-design {feature}` - Verify architecture quality
|
||||
3. Start implementation: `/kiro-impl {feature}`
|
||||
|
||||
```
|
||||
|
||||
## Safety & Fallback
|
||||
|
||||
### Error Scenarios
|
||||
|
||||
**Template Missing**:
|
||||
- Check `.kiro/settings/templates/specs/` exists
|
||||
- Report specific missing file
|
||||
- Exit with error
|
||||
|
||||
**Directory Creation Failed**:
|
||||
- Check permissions
|
||||
- Report error with path
|
||||
- Exit with error
|
||||
|
||||
**Phase Execution Failed** (Phase 2-4):
|
||||
- Stop workflow
|
||||
- Show current state and completed phases
|
||||
- Suggest: "Continue manually from `/kiro-spec-{next-phase} {feature}`"
|
||||
|
||||
**Sanity Review Failed**:
|
||||
- Stop workflow
|
||||
- Report the exact contradiction, missing prerequisite, or task-plan issue
|
||||
- Suggest targeted follow-up with `/kiro-spec-design {feature}`, `/kiro-spec-tasks {feature}`, or manual edits depending on the finding
|
||||
|
||||
**User Cancellation** (Interactive Mode):
|
||||
- Stop gracefully
|
||||
- Show completed phases
|
||||
- Suggest manual continuation
|
||||
@@ -24,7 +24,7 @@ metadata:
|
||||
- Read `.kiro/specs/$1/spec.json` for language and metadata
|
||||
- Read `.kiro/specs/$1/brief.md` if it exists (discovery context: problem, approach, scope decisions, boundary candidates)
|
||||
- Read `.kiro/specs/$1/requirements.md` for project description
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
|
||||
- Additional steering files only when directly relevant to feature scope, user personas, business/domain rules, compliance/security constraints, operational constraints, or existing product boundaries
|
||||
- Relevant local agent skills or playbooks only when they clearly match the feature's host environment or use case and contain domain terminology or workflow rules that shape user-observable requirements
|
||||
|
||||
@@ -40,7 +40,7 @@ The following research areas are independent. Decide the optimal decomposition b
|
||||
**In main context** (essential for requirements generation):
|
||||
- Spec files: spec.json, brief.md, requirements.md (project description)
|
||||
- EARS format rules, requirements review gate, requirements template
|
||||
- Core steering: product.md, tech.md (directly inform scope and constraints)
|
||||
- Core steering: overview.md, safety-net.md (directly inform scope and constraints)
|
||||
|
||||
**Delegate to sub-agent** (keeps exploration out of main context):
|
||||
- **Codebase hints** (brownfield projects): Spawn a sub-agent to explore existing implementations that inform requirement scope. Ask it to summarize: (1) what already exists, (2) relevant interfaces/APIs, (3) patterns that new requirements should align with. Return a summary under 150 lines.
|
||||
@@ -133,7 +133,7 @@ Provide output in the language specified in spec.json with:
|
||||
**If Requirements Approved**:
|
||||
- Review generated requirements at `.kiro/specs/$1/requirements.md`
|
||||
- **Optional Gap Analysis** (for existing codebases):
|
||||
- Run `/kiro-validate-gap $1` to analyze implementation gap with current code
|
||||
- Run `/kiro-spec-design $1 --gap` to analyze the implementation gap with the current code before designing
|
||||
- Identifies existing components, integration points, and implementation strategy
|
||||
- Recommended for brownfield projects; skip for greenfield
|
||||
- Then `/kiro-spec-design $1 -y` to proceed to design phase
|
||||
|
||||
@@ -25,7 +25,7 @@ metadata:
|
||||
**Read all necessary context**:
|
||||
- `.kiro/specs/$1/spec.json`, `requirements.md`, `design.md`
|
||||
- `.kiro/specs/$1/tasks.md` (if exists, for merge mode)
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
|
||||
- Additional steering files only when directly relevant to requirements coverage, design boundaries, runtime prerequisites, or team conventions that affect task executability
|
||||
|
||||
**Validate approvals**:
|
||||
|
||||
@@ -1,145 +0,0 @@
|
||||
---
|
||||
name: kiro-steering-custom
|
||||
description: Create custom steering documents for specialized project contexts
|
||||
metadata:
|
||||
shared-rules: "steering-principles.md"
|
||||
---
|
||||
|
||||
|
||||
# Kiro Custom Steering Creation
|
||||
|
||||
<background_information>
|
||||
**Role**: Create specialized steering documents beyond core files (product, tech, structure).
|
||||
|
||||
**Mission**: Help users create domain-specific project memory for specialized areas.
|
||||
|
||||
**Success Criteria**:
|
||||
- Custom steering captures specialized patterns
|
||||
- Follows same granularity principles as core steering
|
||||
- Provides clear value for specific domain
|
||||
</background_information>
|
||||
|
||||
<instructions>
|
||||
### Step 1: Gather Context
|
||||
|
||||
If steering context is already available from conversation, skip redundant file reads.
|
||||
Otherwise:
|
||||
- Check `.kiro/settings/templates/steering-custom/` for available templates
|
||||
- Read `rules/steering-principles.md` from this skill's directory for steering principles
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Ask user** for custom steering needs:
|
||||
- Domain/topic (e.g., "API standards", "testing approach")
|
||||
- Specific requirements or patterns to document
|
||||
|
||||
2. **Check if template exists**:
|
||||
- Load from `.kiro/settings/templates/steering-custom/{name}.md` if available
|
||||
- Use as starting point, customize based on project
|
||||
|
||||
3. **Analyze codebase** (JIT) for relevant patterns:
|
||||
|
||||
#### Parallel Research
|
||||
|
||||
The following research areas are independent and can be executed in parallel:
|
||||
1. **Template & principles**: Load matching template and steering-principles.md
|
||||
2. **Domain patterns**: Analyze codebase for domain-specific patterns using Glob/Grep/Read
|
||||
|
||||
If multi-agent is enabled, spawn sub-agents for each area above. Otherwise execute sequentially.
|
||||
|
||||
After all parallel research completes, synthesize findings for steering document.
|
||||
|
||||
4. **Generate custom steering**:
|
||||
- Follow template structure if available
|
||||
- Apply principles from `rules/steering-principles.md` from this skill's directory
|
||||
- Focus on patterns, not exhaustive lists
|
||||
- Keep to 100-200 lines (2-3 minute read)
|
||||
|
||||
5. **Create file** in `.kiro/steering/{name}.md`
|
||||
|
||||
## Available Templates
|
||||
|
||||
Templates available in `.kiro/settings/templates/steering-custom/`:
|
||||
|
||||
1. **api-standards.md** - REST/GraphQL conventions, error handling
|
||||
2. **testing.md** - Test organization, mocking, coverage
|
||||
3. **security.md** - Auth patterns, input validation, secrets
|
||||
4. **database.md** - Schema design, migrations, query patterns
|
||||
5. **error-handling.md** - Error types, logging, retry strategies
|
||||
6. **authentication.md** - Auth flows, permissions, session management
|
||||
7. **deployment.md** - CI/CD, environments, rollback procedures
|
||||
|
||||
Load template when needed, customize for project.
|
||||
|
||||
## Steering Principles
|
||||
|
||||
From `rules/steering-principles.md` (in this skill's directory):
|
||||
|
||||
- **Patterns over lists**: Document patterns, not every file/component
|
||||
- **Single domain**: One topic per file
|
||||
- **Concrete examples**: Show patterns with code
|
||||
- **Maintainable size**: 100-200 lines typical
|
||||
- **Security first**: Never include secrets or sensitive data
|
||||
|
||||
</instructions>
|
||||
|
||||
## Tool guidance
|
||||
|
||||
- **Read**: Load template, analyze existing code
|
||||
- **Glob**: Find related files for pattern analysis
|
||||
- **Grep**: Search for specific patterns
|
||||
- **Bash** with `ls`: Understand relevant structure
|
||||
|
||||
**JIT Strategy**: Load template only when creating that type of steering.
|
||||
|
||||
## Output description
|
||||
|
||||
Chat summary with file location (file created directly).
|
||||
|
||||
```
|
||||
✅ Custom Steering Created
|
||||
|
||||
## Created:
|
||||
- .kiro/steering/api-standards.md
|
||||
|
||||
## Based On:
|
||||
- Template: api-standards.md
|
||||
- Analyzed: src/api/ directory patterns
|
||||
- Extracted: REST conventions, error format
|
||||
|
||||
## Content:
|
||||
- Endpoint naming patterns
|
||||
- Request/response format
|
||||
- Error handling conventions
|
||||
- Authentication approach
|
||||
|
||||
Review and customize as needed.
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Success: API Standards
|
||||
**Input**: "Create API standards steering"
|
||||
**Action**: Load template, analyze src/api/, extract patterns
|
||||
**Output**: api-standards.md with project-specific REST conventions
|
||||
|
||||
### Success: Testing Strategy
|
||||
**Input**: "Document our testing approach"
|
||||
**Action**: Load template, analyze test files, extract patterns
|
||||
**Output**: testing.md with test organization and mocking strategies
|
||||
|
||||
## Safety & Fallback
|
||||
|
||||
- **No template**: Generate from scratch based on domain knowledge
|
||||
- **Security**: Never include secrets (load principles)
|
||||
- **Validation**: Ensure doesn't duplicate core steering content
|
||||
|
||||
## Notes
|
||||
|
||||
- Templates are starting points, customize for project
|
||||
- Follow same granularity principles as core steering
|
||||
- All steering files loaded as project memory
|
||||
- Custom files equally important as core files
|
||||
- Avoid documenting agent-specific tooling directories (e.g. `.cursor/`, `.gemini/`, `.claude/`)
|
||||
- Light references to `.kiro/specs/` and `.kiro/steering/` are acceptable; avoid other `.kiro/` directories
|
||||
|
||||
@@ -1,90 +0,0 @@
|
||||
# Steering Principles
|
||||
|
||||
Steering files are **project memory**, not exhaustive specifications.
|
||||
|
||||
---
|
||||
|
||||
## Content Granularity
|
||||
|
||||
### Golden Rule
|
||||
> "If new code follows existing patterns, steering shouldn't need updating."
|
||||
|
||||
### ✅ Document
|
||||
- Organizational patterns (feature-first, layered)
|
||||
- Naming conventions (PascalCase rules)
|
||||
- Import strategies (absolute vs relative)
|
||||
- Architectural decisions (state management)
|
||||
- Technology standards (key frameworks)
|
||||
|
||||
### ❌ Avoid
|
||||
- Complete file listings
|
||||
- Every component description
|
||||
- All dependencies
|
||||
- Implementation details
|
||||
- Agent-specific tooling directories (e.g. `.cursor/`, `.gemini/`, `.claude/`)
|
||||
- Detailed documentation of `.kiro/` metadata directories (settings, automation)
|
||||
|
||||
### Example Comparison
|
||||
|
||||
**Bad** (Specification-like):
|
||||
```markdown
|
||||
- /components/Button.tsx - Primary button with variants
|
||||
- /components/Input.tsx - Text input with validation
|
||||
- /components/Modal.tsx - Modal dialog
|
||||
... (50+ files)
|
||||
```
|
||||
|
||||
**Good** (Project Memory):
|
||||
```markdown
|
||||
## UI Components (`/components/ui/`)
|
||||
Reusable, design-system aligned primitives
|
||||
- Named by function (Button, Input, Modal)
|
||||
- Export component + TypeScript interface
|
||||
- No business logic
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
Never include:
|
||||
- API keys, passwords, credentials
|
||||
- Database URLs, internal IPs
|
||||
- Secrets or sensitive data
|
||||
|
||||
---
|
||||
|
||||
## Quality Standards
|
||||
|
||||
- **Single domain**: One topic per file
|
||||
- **Concrete examples**: Show patterns with code
|
||||
- **Explain rationale**: Why decisions were made
|
||||
- **Maintainable size**: 100-200 lines typical
|
||||
|
||||
---
|
||||
|
||||
## Preservation (when updating)
|
||||
|
||||
- Preserve user sections and custom examples
|
||||
- Additive by default (add, don't replace)
|
||||
- Add `updated_at` timestamp
|
||||
- Note why changes were made
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- Templates are starting points, customize as needed
|
||||
- Follow same granularity principles as core steering
|
||||
- All steering files loaded as project memory
|
||||
- Light references to `.kiro/specs/` and `.kiro/steering/` are acceptable; avoid other `.kiro/` directories
|
||||
- Custom files equally important as core files
|
||||
|
||||
---
|
||||
|
||||
## File-Specific Focus
|
||||
|
||||
- **product.md**: Purpose, value, business context (not exhaustive features)
|
||||
- **tech.md**: Key frameworks, standards, conventions (not all dependencies)
|
||||
- **structure.md**: Organization patterns, naming rules (not directory trees)
|
||||
- **Custom files**: Specialized patterns (API, testing, security, etc.)
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: kiro-steering
|
||||
description: Manage .kiro/steering/ as persistent project knowledge
|
||||
description: Manage .kiro/steering/ as persistent project knowledge, including custom steering documents for specialized contexts
|
||||
metadata:
|
||||
shared-rules: "steering-principles.md"
|
||||
---
|
||||
@@ -11,9 +11,12 @@ metadata:
|
||||
<background_information>
|
||||
**Role**: Maintain `.kiro/steering/` as persistent project memory.
|
||||
|
||||
**Core steering in this repository**: `overview.md` (what the project is and what is actually true of it, dated current state), `roadmap.md` (milestones and status), `safety-net.md` (product direction). Custom files may exist beside them (for example a domain or standards file); all `.kiro/steering/*.md` are loaded as project memory. The generic bootstrap templates under `.kiro/settings/templates/steering/` are named `product.md`, `tech.md`, `structure.md`; they are starting points for a fresh project, not this repository's file names.
|
||||
|
||||
**Mission**:
|
||||
- Bootstrap: Generate core steering from codebase (first-time)
|
||||
- Sync: Keep steering and codebase aligned (maintenance)
|
||||
- Custom: Create specialized steering documents beyond the core files (optional, see below)
|
||||
- Preserve: User customizations are sacred, updates are additive
|
||||
|
||||
**Success Criteria**:
|
||||
@@ -27,8 +30,9 @@ metadata:
|
||||
|
||||
Check `.kiro/steering/` status:
|
||||
|
||||
**Bootstrap Mode**: Empty OR missing core files (product.md, tech.md, structure.md)
|
||||
**Sync Mode**: All core files exist
|
||||
**Bootstrap Mode**: Empty OR missing core files (here `overview.md`, `roadmap.md`, `safety-net.md`; in a fresh project the template names `product.md`, `tech.md`, `structure.md`)
|
||||
**Sync Mode**: All core files exist
|
||||
**Custom Mode**: The user asks for a specialized steering document (see Custom Steering below)
|
||||
|
||||
---
|
||||
|
||||
@@ -75,6 +79,20 @@ After all parallel research completes, synthesize patterns for steering files.
|
||||
|
||||
---
|
||||
|
||||
## Custom Steering (optional)
|
||||
|
||||
Create a specialized steering document for one domain beyond the core files.
|
||||
|
||||
1. **Ask the user** for the domain/topic (e.g. "API standards", "testing approach") and the specific patterns to document
|
||||
2. **Check for a template** in `.kiro/settings/templates/steering-custom/`: `api-standards.md`, `testing.md`, `security.md`, `database.md`, `error-handling.md`, `authentication.md`, `deployment.md`. Load the matching one as a starting point; without a template, generate from domain knowledge
|
||||
3. **Analyze the codebase** (JIT) for the domain's patterns with Glob/Grep/Read (template and principles in parallel with the domain analysis when multi-agent is available)
|
||||
4. **Generate** the document: follow the template structure if any, apply `rules/steering-principles.md`, patterns not exhaustive lists, one domain per file, 100-200 lines (a 2-3 minute read), never secrets
|
||||
5. **Create** `.kiro/steering/{name}.md`; ensure it does not duplicate core steering content
|
||||
|
||||
Custom files are loaded as project memory like the core files and carry equal weight.
|
||||
|
||||
---
|
||||
|
||||
## Granularity Principle
|
||||
|
||||
From `rules/steering-principles.md` (in this skill's directory):
|
||||
@@ -106,9 +124,9 @@ Chat summary only (files updated directly).
|
||||
✅ Steering Created
|
||||
|
||||
## Generated:
|
||||
- product.md: [Brief description]
|
||||
- tech.md: [Key stack]
|
||||
- structure.md: [Organization]
|
||||
- overview.md: [What it is, current state]
|
||||
- roadmap.md: [Milestones, status]
|
||||
- safety-net.md: [Product direction]
|
||||
|
||||
Review and approve as Source of Truth.
|
||||
```
|
||||
@@ -118,8 +136,8 @@ Review and approve as Source of Truth.
|
||||
✅ Steering Updated
|
||||
|
||||
## Changes:
|
||||
- tech.md: React 18 → 19
|
||||
- structure.md: Added API pattern
|
||||
- overview.md: Current state dated, retired component removed
|
||||
- roadmap.md: Milestone status updated
|
||||
|
||||
## Code Drift:
|
||||
- Components not following import conventions
|
||||
@@ -128,6 +146,20 @@ Review and approve as Source of Truth.
|
||||
- Consider api-standards.md
|
||||
```
|
||||
|
||||
### Custom:
|
||||
```
|
||||
✅ Custom Steering Created
|
||||
|
||||
## Created:
|
||||
- .kiro/steering/api-standards.md
|
||||
|
||||
## Based On:
|
||||
- Template: api-standards.md
|
||||
- Analyzed: src/api/ directory patterns
|
||||
|
||||
Review and customize as needed.
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Bootstrap
|
||||
@@ -136,7 +168,12 @@ Review and approve as Source of Truth.
|
||||
|
||||
### Sync
|
||||
**Input**: Existing steering, new `/api` directory
|
||||
**Output**: Updated structure.md, flagged non-compliant files, suggested api-standards.md
|
||||
**Output**: Updated overview.md, flagged non-compliant files, suggested api-standards.md
|
||||
|
||||
### Custom
|
||||
**Input**: "Create API standards steering"
|
||||
**Action**: Load template, analyze src/api/, extract patterns
|
||||
**Output**: api-standards.md with project-specific REST conventions
|
||||
|
||||
## Safety & Fallback
|
||||
|
||||
|
||||
@@ -84,7 +84,11 @@ Never include:
|
||||
|
||||
## File-Specific Focus
|
||||
|
||||
- **product.md**: Purpose, value, business context (not exhaustive features)
|
||||
- **tech.md**: Key frameworks, standards, conventions (not all dependencies)
|
||||
- **structure.md**: Organization patterns, naming rules (not directory trees)
|
||||
- **Custom files**: Specialized patterns (API, testing, security, etc.)
|
||||
This repository's core steering files:
|
||||
|
||||
- **overview.md**: What the project is and what is actually true of it; a dated current-state section wins over older history
|
||||
- **roadmap.md**: Milestones, their measured status, and the accepted order of work (not a task list)
|
||||
- **safety-net.md**: Product direction and acceptance criteria (not feature catalogs)
|
||||
- **Custom files**: Specialized patterns (API, testing, security, etc.); they may exist beside the core files and carry equal weight
|
||||
|
||||
(The generic bootstrap templates `product.md`, `tech.md`, `structure.md` under `.kiro/settings/templates/steering/` cover purpose, key frameworks and organization patterns for a fresh project.)
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
---
|
||||
name: kiro-validate-design
|
||||
description: Interactive technical design quality review and validation
|
||||
metadata:
|
||||
shared-rules: "design-review.md"
|
||||
---
|
||||
|
||||
|
||||
# Technical Design Validation
|
||||
|
||||
<background_information>
|
||||
- **Mission**: Conduct interactive quality review of technical design to ensure readiness for implementation
|
||||
- **Success Criteria**:
|
||||
- Critical issues identified (maximum 3 most important concerns)
|
||||
- Balanced assessment with strengths recognized
|
||||
- Clear GO/NO-GO decision with rationale
|
||||
- Actionable feedback for improvements if needed
|
||||
</background_information>
|
||||
|
||||
<instructions>
|
||||
## Core Task
|
||||
Interactive design quality review for feature **$1** based on approved requirements and design document.
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. **Gather Context**:
|
||||
- Read `.kiro/specs/$1/spec.json` for language and metadata
|
||||
- Read `.kiro/specs/$1/requirements.md` for requirements
|
||||
- Read `.kiro/specs/$1/design.md` for design document
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- Additional steering files only when directly relevant to architecture boundaries, integrations, runtime prerequisites, domain rules, security/performance constraints, or team conventions that affect implementation readiness
|
||||
- Relevant local agent skills or playbooks only when they clearly match the feature's host environment or use case and provide review-relevant context
|
||||
|
||||
#### Parallel Research
|
||||
|
||||
The following research areas are independent and can be executed in parallel:
|
||||
1. **Context & rules loading**: Spec documents, core steering, task-relevant extra steering, relevant local agent skills/playbooks, and `rules/design-review.md` from this skill's directory for review criteria
|
||||
2. **Codebase pattern survey**: Gather existing architecture patterns, naming conventions, and component structure from the codebase to use as reference during review
|
||||
|
||||
If multi-agent is enabled, spawn sub-agents for each area above. Otherwise execute sequentially.
|
||||
|
||||
After all parallel research completes, synthesize findings for review.
|
||||
|
||||
2. **Execute Design Review**:
|
||||
- Reference conversation history when available: leverage prior requirements discussion and user's stated design intent
|
||||
- Follow design-review.md process: Analysis → Critical Issues → Strengths → GO/NO-GO
|
||||
- Limit to 3 most important concerns
|
||||
- Engage interactively with user — ask clarifying questions, propose alternatives
|
||||
- Use language specified in spec.json for output
|
||||
|
||||
3. **Decision and Next Steps**:
|
||||
- Clear GO/NO-GO decision with rationale
|
||||
- Provide specific actionable next steps (see Next Phase below)
|
||||
|
||||
## Important Constraints
|
||||
- **Quality assurance, not perfection seeking**: Accept acceptable risk
|
||||
- **Critical focus only**: Maximum 3 issues, only those significantly impacting success
|
||||
- **Conversation-aware**: Leverage discussion history for requirements context and user intent when available
|
||||
- **Interactive approach**: Engage in dialogue, ask clarifying questions, propose alternatives
|
||||
- **Balanced assessment**: Recognize both strengths and weaknesses
|
||||
- **Actionable feedback**: All suggestions must be implementable
|
||||
- **Context Discipline**: Start with core steering and expand only with review-relevant steering or use-case-aligned local agent skills/playbooks
|
||||
</instructions>
|
||||
|
||||
## Tool Guidance
|
||||
- **Read first**: Load spec, core steering, relevant local playbooks/agent skills, and rules before review
|
||||
- **Grep if needed**: Search codebase for pattern validation or integration checks
|
||||
- **Interactive**: Engage with user throughout the review process
|
||||
|
||||
## Output Description
|
||||
Provide output in the language specified in spec.json with:
|
||||
|
||||
1. **Review Summary**: Brief overview (2-3 sentences) of design quality and readiness
|
||||
2. **Critical Issues**: Maximum 3, following design-review.md format
|
||||
3. **Design Strengths**: 1-2 positive aspects
|
||||
4. **Final Assessment**: GO/NO-GO decision with rationale and next steps
|
||||
|
||||
**Format Requirements**:
|
||||
- Use Markdown headings for clarity
|
||||
- Follow design-review.md output format
|
||||
- Keep summary concise
|
||||
|
||||
## Safety & Fallback
|
||||
|
||||
### Error Scenarios
|
||||
- **Missing Design**: If design.md doesn't exist, stop with message: "Run `/kiro-spec-design $1` first to generate design document"
|
||||
- **Design Not Generated**: If design phase not marked as generated in spec.json, warn but proceed with review
|
||||
- **Empty Steering Directory**: Warn user that project context is missing and may affect review quality
|
||||
- **Language Undefined**: Default to English (`en`) if spec.json doesn't specify language
|
||||
|
||||
### Next Phase: Task Generation
|
||||
|
||||
**If Design Passes Validation (GO Decision)**:
|
||||
- Review feedback and apply changes if needed
|
||||
- Run `/kiro-spec-tasks $1` to generate implementation tasks
|
||||
- Or `/kiro-spec-tasks $1 -y` to auto-approve and proceed directly
|
||||
|
||||
**If Design Needs Revision (NO-GO Decision)**:
|
||||
- Address critical issues identified
|
||||
- Re-run `/kiro-spec-design $1` with improvements
|
||||
- Re-validate with `/kiro-validate-design $1`
|
||||
|
||||
**Note**: Design validation is recommended but optional. Quality review helps catch issues early.
|
||||
@@ -1,109 +0,0 @@
|
||||
---
|
||||
name: kiro-validate-gap
|
||||
description: Analyze implementation gap between requirements and existing codebase
|
||||
metadata:
|
||||
shared-rules: "gap-analysis.md"
|
||||
---
|
||||
|
||||
|
||||
# Implementation Gap Validation
|
||||
|
||||
<background_information>
|
||||
- **Mission**: Analyze the gap between requirements and existing codebase to inform implementation strategy
|
||||
- **Success Criteria**:
|
||||
- Comprehensive understanding of existing codebase patterns and components
|
||||
- Clear identification of missing capabilities and integration challenges
|
||||
- Multiple viable implementation approaches evaluated
|
||||
- Technical research needs identified for design phase
|
||||
</background_information>
|
||||
|
||||
<instructions>
|
||||
## Core Task
|
||||
Analyze implementation gap for feature **$1** based on approved requirements and existing codebase.
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. **Load Context**:
|
||||
- Read `.kiro/specs/$1/spec.json` for language and metadata
|
||||
- Read `.kiro/specs/$1/requirements.md` for requirements
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- Additional steering files only when directly relevant to the feature's domain rules, integrations, runtime prerequisites, compliance/security constraints, or existing product boundaries
|
||||
- Relevant local agent skills or playbooks only when they clearly match the feature's host environment or use case and provide analysis-relevant context
|
||||
|
||||
2. **Read Analysis Guidelines**:
|
||||
- Read `rules/gap-analysis.md` from this skill's directory for comprehensive analysis framework
|
||||
|
||||
#### Parallel Research
|
||||
|
||||
The following research areas are independent and can be executed in parallel:
|
||||
1. **Codebase analysis**: Existing implementations, architecture patterns, integration points, extension possibilities
|
||||
2. **External dependency research**: Dependency compatibility, version constraints, known integration challenges (when needed)
|
||||
3. **Context loading**: Requirements, core steering, task-relevant extra steering, relevant local agent skills/playbooks, and gap-analysis rules
|
||||
|
||||
If multi-agent is enabled, spawn sub-agents for each area above. Otherwise execute sequentially.
|
||||
|
||||
After all parallel research completes, synthesize findings for gap analysis.
|
||||
|
||||
3. **Execute Gap Analysis**:
|
||||
- Follow gap-analysis.md framework for thorough investigation
|
||||
- Analyze existing codebase using Grep and Read tools
|
||||
- Use WebSearch/WebFetch for external dependency research if needed
|
||||
- Evaluate multiple implementation approaches (extend/new/hybrid)
|
||||
- Use language specified in spec.json for output
|
||||
|
||||
4. **Generate Analysis Document**:
|
||||
- Create comprehensive gap analysis following the output guidelines in gap-analysis.md
|
||||
- Present multiple viable options with trade-offs
|
||||
- Flag areas requiring further research
|
||||
|
||||
5. **Write Gap Analysis to Disk**:
|
||||
|
||||
**Write the gap analysis to disk so it survives session boundaries and can be referenced during design phase.**
|
||||
|
||||
- Save the gap analysis to `.kiro/specs/$1/research.md`
|
||||
- If the file already exists, append the new analysis (separated by a horizontal rule `---`) rather than overwriting previous research
|
||||
- Verify the file was written by reading it back
|
||||
|
||||
## Important Constraints
|
||||
- **Information over Decisions**: Provide analysis and options, not final implementation choices
|
||||
- **Multiple Options**: Present viable alternatives when applicable
|
||||
- **Thorough Investigation**: Use tools to deeply understand existing codebase
|
||||
- **Explicit Gaps**: Clearly flag areas needing research or investigation
|
||||
- **Context Discipline**: Start with core steering and expand only with analysis-relevant steering or use-case-aligned local agent skills/playbooks
|
||||
</instructions>
|
||||
|
||||
## Tool Guidance
|
||||
- **Read first**: Load spec, core steering, relevant local playbooks/agent skills, and rules before analysis
|
||||
- **Grep extensively**: Search codebase for patterns, conventions, and integration points
|
||||
- **WebSearch/WebFetch**: Research external dependencies and best practices when needed
|
||||
- **Write last**: Generate analysis only after complete investigation
|
||||
|
||||
## Output Description
|
||||
Provide output in the language specified in spec.json with:
|
||||
|
||||
1. **Analysis Summary**: Brief overview (3-5 bullets) of scope, challenges, and recommendations
|
||||
2. **Document Status**: Confirm analysis approach used
|
||||
3. **Next Steps**: Guide user on proceeding to design phase
|
||||
|
||||
**Format Requirements**:
|
||||
- Use Markdown headings for clarity
|
||||
- Keep summary concise (under 300 words)
|
||||
- Detailed analysis follows gap-analysis.md output guidelines
|
||||
|
||||
## Safety & Fallback
|
||||
|
||||
### Error Scenarios
|
||||
- **Missing Requirements**: If requirements.md doesn't exist, stop with message: "Run `/kiro-spec-requirements $1` first to generate requirements"
|
||||
- **Requirements Not Approved**: If requirements not approved, warn user but proceed (gap analysis can inform requirement revisions)
|
||||
- **Empty Steering Directory**: Warn user that project context is missing and may affect analysis quality
|
||||
- **Complex Integration Unclear**: Flag for comprehensive research in design phase rather than blocking
|
||||
- **Language Undefined**: Default to English (`en`) if spec.json doesn't specify language
|
||||
|
||||
### Next Phase: Design Generation
|
||||
|
||||
**If Gap Analysis Complete**:
|
||||
- Review gap analysis insights
|
||||
- Run `/kiro-spec-design $1` to create technical design document
|
||||
- Or `/kiro-spec-design $1 -y` to auto-approve requirements and proceed directly
|
||||
|
||||
**Note**: Gap analysis is optional but recommended for brownfield projects to inform design decisions.
|
||||
@@ -68,10 +68,16 @@ For each detected feature:
|
||||
- Read `.kiro/specs/<feature>/requirements.md` for requirements
|
||||
- Read `.kiro/specs/<feature>/design.md` for design structure
|
||||
- Read `.kiro/specs/<feature>/tasks.md` for task list and Implementation Notes
|
||||
- Core steering context: `product.md`, `tech.md`, `structure.md`
|
||||
- Core steering context: `overview.md`, `roadmap.md`, `safety-net.md` (custom steering files may also exist in `.kiro/steering/`)
|
||||
- Additional steering files only when directly relevant to the validated boundaries, runtime prerequisites, integrations, domain rules, security/performance constraints, or team conventions that affect the GO/NO-GO call
|
||||
|
||||
**Discover canonical validation commands**:
|
||||
**This repository's gates** (`.github/workflows/ci.yml`; run them as written, judge each by its own exit code):
|
||||
- `TEST_COMMANDS`: `pytest -q -p no:cacheprovider < /dev/null; echo "exit=$?"` -- judge ONLY by the printed exit code, never by a summary line; `ruff check src tests`; `ruff format --check src tests`; `mypy src/openultrasast`
|
||||
- Detection gates: `python -m openultrasast.gate` (>=90% recall, <10% FP per language; hard, score-independent), `python -m openultrasast.map_gate`, and the pair gate `python -m openultrasast.pair_gate` (local vuln vs fix). A ruleset change that fails a gate is NO-GO.
|
||||
- Docs: `mkdocs build --strict` when `docs/` or `mkdocs.yml` changed
|
||||
- `SMOKE_COMMANDS`: `ousast --help`, and for a scan change `ousast scan <fixture> --mode quick` whose `manifest.json` names the stages that ran and the files preprocess kept. A plausible zero (0 findings, 0 degradations) from a tool that never read its input is a failed instrument, not a pass: prove the tool read its input first.
|
||||
|
||||
**Discover canonical validation commands** (for other repositories, or anything the list above does not cover):
|
||||
- Inspect repository-local sources of truth in this order: project scripts/manifests (`package.json`, `pyproject.toml`, `go.mod`, `Cargo.toml`, app manifests), task runners (`Makefile`, `justfile`), CI/workflow files, existing e2e/integration configs, then `README*`
|
||||
- Derive a feature-level validation set for this repo: `TEST_COMMANDS`, `BUILD_COMMANDS`, and `SMOKE_COMMANDS`
|
||||
- Prefer commands already used by repo automation over ad hoc shell pipelines
|
||||
|
||||
@@ -61,7 +61,8 @@ Use the language specified in `spec.json`.
|
||||
4. Check exit code, failure count, skipped scope, and missing coverage.
|
||||
5. Reject claims that are broader than the evidence.
|
||||
6. If mandatory validation cannot be completed, return `MANUAL_VERIFY_REQUIRED`.
|
||||
7. Only then allow the claim.
|
||||
7. Verify the instrument before you believe the measurement: prove the tool read its input (it opened one file and printed the size; the stage list names what ran). A plausible zero -- "0 findings, 0 errors, exit 0" from a tree the tool never read, or a Joern script that "finished" before a JVM could start -- is a failed instrument, not evidence. For pytest, judge only by the printed exit code (`pytest -q -p no:cacheprovider < /dev/null; echo "exit=$?"`), never by a summary line.
|
||||
8. Only then allow the claim.
|
||||
|
||||
## Claim-Specific Rules
|
||||
|
||||
@@ -116,6 +117,7 @@ Return `NOT_VERIFIED` when:
|
||||
| “Build should be fine because lint passed” | Lint does not prove build success. |
|
||||
| “Tests passed and build succeeded, so it must run” | Type erasure, module loading, native ABI, and boot-time config issues can still fail at runtime. |
|
||||
| “The feature is done because all tasks are checked off” | `FEATURE_GO` also requires coverage, integration, and design alignment. |
|
||||
| “The scan found nothing, so the code is clean” | An unread input and clean code both report zero. Prove the tool read its input first. |
|
||||
|
||||
## Output Format
|
||||
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
---
|
||||
name: openultrasast-fix-audit
|
||||
description: OpenUltraSAST patch, fix audit, differential review, adversarial review, and patch validation workflow. Use when proposing or auditing fixes for OpenUltraSAST findings from opencode.
|
||||
---
|
||||
|
||||
# OpenUltraSAST Fix Audit
|
||||
|
||||
Use this skill to steer fixes for OpenUltraSAST findings.
|
||||
|
||||
## Fix Lifecycle
|
||||
|
||||
1. Intake: restate the finding ID, evidence level, affected path, and acceptance criteria.
|
||||
2. Plan: choose the smallest defensive change that addresses the root cause.
|
||||
3. Implement: modify only files required by the finding.
|
||||
4. Adversarial review: look for bypasses, regressions, false assumptions, and unrelated edits.
|
||||
5. Reconcile: either fix the review finding or document why it is not applicable.
|
||||
6. Verify: rerun the reproducer or affected checks.
|
||||
7. Ready: only then consider advancing to `patch_validated`.
|
||||
|
||||
## Guardrails
|
||||
|
||||
- Do not rewrite modules when a local fix is enough.
|
||||
- Do not change public APIs unless the vulnerability requires it.
|
||||
- Do not mark a patch validated from model reasoning alone.
|
||||
- Use differential review for risky changes in auth, crypto, config, parsers, and sandbox behavior.
|
||||
- Run Semgrep or CodeQL delta checks when those analyzers supplied the original evidence.
|
||||
@@ -1,35 +1,124 @@
|
||||
---
|
||||
name: openultrasast-scan
|
||||
description: OpenUltraSAST scan, index, quick, standard, deep, SARIF, and report workflows. Use when running or planning `ousast scan`, repository indexing, or evidence-gated SAST analysis from opencode.
|
||||
description: OpenUltraSAST scan workflows - `ousast scan` in quick, standard or deep mode, the evidence ladder, `--fail-on` exit codes, recorded degradations, the run directory and `ousast mcp`. Use when running or planning a scan, reading a run directory, or wiring OpenUltraSAST into an agent.
|
||||
---
|
||||
|
||||
# OpenUltraSAST Scan
|
||||
|
||||
Use this skill to steer OpenUltraSAST scan workflows from opencode.
|
||||
Use this skill to run `ousast scan PATH --mode quick|standard|deep` from an agent and to read what
|
||||
it wrote.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Confirm the target is a local repository path.
|
||||
2. Run commands through uv, never by activating a virtualenv manually.
|
||||
3. Start with quick mode unless the task explicitly requires standard or deep analysis.
|
||||
4. Treat quick findings as `static_corroboration`, not verified vulnerabilities.
|
||||
5. Preserve run artifacts under `.openultrasast/runs/<scan-id>/`.
|
||||
6. Report `scan_id`, `run_dir`, finding count, and the strongest available evidence level.
|
||||
1. Confirm the target is a local checkout (a path, not a URL).
|
||||
2. Run through the project entry point: `uv run ousast ...` inside the repository, or `ousast` from an
|
||||
environment where it is installed. Never activate a virtualenv by hand.
|
||||
3. Start with `--mode quick` unless the task needs the Joern model layer (`standard`) or sandboxed
|
||||
regression snippets (`deep`). Check the prerequisites below first; a mode whose engine is missing
|
||||
still exits 0 and records a degradation.
|
||||
4. Read the printed `scan_id=` and `run_dir=` lines; the run directory is `.openultrasast/runs/<scan-id>/`.
|
||||
5. Report the finding count, how many the verifier accepted, and **every** recorded degradation
|
||||
(`manifest.json`, key `degradations`). No findings plus a degradation is not a clean scan: the scan
|
||||
answered a smaller question than it was asked.
|
||||
6. Never call a finding verified from its title or severity. `verification.json` holds the status.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
uv run ousast scan <repo> --mode quick
|
||||
uv run ousast index <repo>
|
||||
uv run pytest
|
||||
uv run ousast scan <repo> --mode standard --fail-on verified
|
||||
uv run ousast scan <repo> --mode deep --fail-on worth-fixing
|
||||
uv run ousast scan <repo> --mode quick --config openultrasast.toml
|
||||
uv run ousast index <repo> # chunk the checkout for the embedding index
|
||||
uv run ousast mcp # the MCP server over stdio, for agent hosts
|
||||
```
|
||||
|
||||
## Evidence Rules
|
||||
## Modes and prerequisites
|
||||
|
||||
- `suspicion` means a model or weak signal raised a hypothesis.
|
||||
- `static_corroboration` means deterministic code evidence exists.
|
||||
- `crash_reproduced` requires an artifact-backed reproducer.
|
||||
- `root_cause_explained` requires independent verifier reasoning.
|
||||
- `patch_validated` requires rerunning relevant checks after the patch.
|
||||
| Mode | Adds | Needs | Without it |
|
||||
| --- | --- | --- | --- |
|
||||
| `quick` | language-scoped pattern rules, entry-point hints, ranking, verification, score; deterministic | nothing | - |
|
||||
| `standard` | the MAP stage: complexity map, semantic overlay, authorization obligations, the Joern model layer (one code property graph per repository; taint, guard-dominance and configuration questions), deterministic fusion | Joern (shipped in the Docker image, with `php-cli` for PHP); a provider key for the residual LLM question | `cpg_unavailable`; without a key the LLM question is not asked and only the graph's entailed findings are reported |
|
||||
| `deep` | REGRESS: promoted candidates run as snippets in a Docker sandbox (no network, read-only source, non-root) and get a `triggerable` / `not_triggerable` / ... verdict in `verdicts.json` | a working `docker` | `sandbox_unavailable` |
|
||||
|
||||
Do not describe a finding as verified unless its evidence level meets the configured report threshold.
|
||||
Model stages read `DEEPSEEK_API_KEY` (every LLM call); with only `OPENROUTER_API_KEY` set, chat
|
||||
calls fall back to OpenRouter. An exported shell variable wins over `.env` silently. `[models]
|
||||
hunter` in the config enables the tool hunter in `standard`; it is off by default.
|
||||
|
||||
## Exit code: `--fail-on`
|
||||
|
||||
| Policy | Exit 1 when |
|
||||
| --- | --- |
|
||||
| `never` (default) | never |
|
||||
| `findings` | any finding was reported |
|
||||
| `verified` | any verification result is `verified` (accepted at `static_corroboration` or above) |
|
||||
| `worth-fixing` | a deep-mode sandbox verdict is marked `worth_fixing` |
|
||||
|
||||
A degradation never changes the exit code. Read `manifest.json` for it.
|
||||
|
||||
## Recorded degradations
|
||||
|
||||
Each entry in `manifest.json` `degradations` is `{"stage": ..., "reason": ...}`. The ones to expect:
|
||||
|
||||
- `cpg_unavailable` (stage `model`): no Joern on the host; the model layer ran nothing.
|
||||
- `sandbox_unavailable` (stage `regress`): no working `docker`; deep mode fell back to standard.
|
||||
- `learning_endpoint_unavailable` (stage `model`): no provider key resolved. The graph's entailed
|
||||
findings are still reported; the residual question is skipped. This is recorded in `manifest.json`
|
||||
but has no line in `report.md`'s "What could not be analysed" section.
|
||||
- `hunter_model_unavailable` (stage `map`): `[models] hunter` is unset, so the tool hunter did not run.
|
||||
Recorded on every standard scan without that setting; it is expected, not an error.
|
||||
- coverage facts from the graph, rendered in `report.md` under "What could not be analysed":
|
||||
`cpg_empty`, `cpg_build_failed`, `cpg_sharded`, `files_unparsed`, `query_failed`,
|
||||
`query_budget_reserved`, `query_too_expensive`, `budget_exhausted`, `regions_truncated`,
|
||||
`region_failed`; plus `max_findings_exceeded` and `facts_unavailable` (overlay, obligations).
|
||||
|
||||
Prove the instrument read its input before believing a zero: `manifest.json` names the stages that
|
||||
ran and the files the preprocess step kept (`preprocess/file_targets.json`).
|
||||
|
||||
## Evidence ladder
|
||||
|
||||
`src/openultrasast/verification.py` defines the levels in this order. A level moves one rung at a
|
||||
time and never backward.
|
||||
|
||||
| Level | Meaning | Produced today by |
|
||||
| --- | --- | --- |
|
||||
| `suspicion` | a rule, model or weak signal raised a hypothesis | the Joern model layer (its rung, `model_entailed` or `suspicion`, travels in the finding's tags), the tool hunter, authorization-obligation checks |
|
||||
| `static_corroboration` | deterministic code evidence at the line | quick rules, the semantic overlay |
|
||||
| `crash_reproduced` | an artifact-backed reproducer | nothing: there is no crash reproducer. Deep-mode sandbox verdicts (`verdicts.json`, `worth_fixing`) do not raise the level |
|
||||
| `root_cause_explained` | independent verifier reasoning | nothing: the LLM judge was retired in v2.0.0 |
|
||||
| `exploit_demonstrated` | a demonstrated exploit | nothing |
|
||||
| `patch_validated` | relevant checks re-run after a patch | nothing: there is no patch stage |
|
||||
|
||||
The verifier's status per finding (`verification.json`): below `static_corroboration` is
|
||||
`needs_evidence`; `static_corroboration` with reachability `unknown` is `rejected` (the tie-breaker
|
||||
asks for call-graph, route, CLI, parser or dynamic evidence); `static_corroboration` with
|
||||
function-level reachability is `accepted`. `verified` means accepted at `static_corroboration` or
|
||||
above, and that is what `--fail-on verified` counts. There is no configurable report threshold: the
|
||||
`[evidence]` config section was retired in v2.0.0 and is ignored.
|
||||
|
||||
## Run artifacts
|
||||
|
||||
Under `.openultrasast/runs/<scan-id>/`:
|
||||
|
||||
- `report.md`, `report.sarif` (SARIF 2.1.0 for code hosts and IDEs), `manifest.json` (stages, degradations, model payload)
|
||||
- `findings.json`, `verification.json`, `score.json`, `shadow_findings.json` (rules shipped as `shadow`)
|
||||
- `harness.json`: the resolved harness configuration the run used
|
||||
- standard: `fusion.json`, `complexity_map.json`, `overlay.json`; deep: `verdicts.json`
|
||||
- `calibration/false_positive_learnings.json`, `rank/ranking.json`, `mapping/`, `preprocess/`, `trace/events.jsonl`
|
||||
|
||||
## Changed code only
|
||||
|
||||
`ousast pre-push --base <rev> --head <rev> --artifact <file> <repo>` replays a revision pair and
|
||||
reports only new or worsened defects (experimental; `--mode advisory` is the default and never
|
||||
blocks). The capability registry is empty, so every candidate is diagnostic.
|
||||
|
||||
## MCP
|
||||
|
||||
`ousast mcp` serves a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools:
|
||||
`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`,
|
||||
`explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a shell, Docker or a
|
||||
free-form command.
|
||||
|
||||
```jsonc
|
||||
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
|
||||
```
|
||||
|
||||
@@ -1,23 +1,29 @@
|
||||
---
|
||||
name: openultrasast-triage
|
||||
description: OpenUltraSAST false-positive elimination, ranking calibration, evidence ladder, verifier triage, and finding adjudication. Use when reviewing findings, reducing false positives, or deciding whether evidence is enough.
|
||||
description: OpenUltraSAST finding triage - false-positive adjudication with the fixed reason taxonomy, the evidence ladder, what a verdict may feed (labels, never rule edits), and fixing then re-scanning. Use when reviewing findings, reducing false positives, deciding whether evidence is enough, or auditing a fix for a finding.
|
||||
---
|
||||
|
||||
# OpenUltraSAST Triage
|
||||
|
||||
Use this skill when triaging OpenUltraSAST findings from opencode.
|
||||
Use this skill when triaging OpenUltraSAST findings or auditing a fix for one.
|
||||
|
||||
## Triage Loop
|
||||
|
||||
1. Identify the finding, path, rule, severity, confidence, and evidence level.
|
||||
2. Separate deterministic evidence from model assumptions.
|
||||
3. Check attacker control, reachability, sanitizer behavior, duplicates, impact, and contradictory evidence.
|
||||
4. If rejected or unresolved, record the false-positive reason and scope.
|
||||
5. Prefer scoped demotion over deleting a vulnerability class globally.
|
||||
6. Feed the outcome into ranking calibration and prompt constraints.
|
||||
1. Identify the finding: id, path, rule, CWE and policy severity, confidence, evidence level and
|
||||
verification status (`findings.json`, `verification.json` in the run directory).
|
||||
2. Separate deterministic evidence (rule match, graph entailment, attached reachability) from model
|
||||
assumptions.
|
||||
3. Check attacker control, reachability, sanitizer behaviour, duplicates, impact, and contradictory
|
||||
evidence.
|
||||
4. If rejected or unresolved, record the false-positive reason and its scope.
|
||||
5. Prefer scoped demotion over deleting a vulnerability class globally: a rejection in `auth/`
|
||||
demotes `auth/...`, never the class (`calibration.py`).
|
||||
6. Record the adjudication. Do not edit a rule because of one finding (see below).
|
||||
|
||||
## False-Positive Reasons
|
||||
|
||||
The taxonomy is `FalsePositiveReason` in `src/openultrasast/calibration.py`; use these values exactly:
|
||||
|
||||
- `unreachable_path`
|
||||
- `missing_attacker_control`
|
||||
- `sanitizer_disproved`
|
||||
@@ -31,4 +37,39 @@ Use this skill when triaging OpenUltraSAST findings from opencode.
|
||||
|
||||
## Decision Standard
|
||||
|
||||
If evidence is uncertain, mark the finding as requiring next evidence instead of escalating severity. Difficult or high-impact disagreements should trigger fusion/deeper reasoning when available.
|
||||
If evidence is uncertain, mark the finding `needs_evidence` and name the next evidence instead of
|
||||
escalating severity. Fusion (`fusion.json`, standard mode) is deterministic: it adjudicates between
|
||||
the quick-rule and MAP panels by fixed triggers and is not a second opinion from a model. Independent
|
||||
model agreement exists only in the plane's verify passes a and b, with pass c on disputes, and the
|
||||
`agree` step; those verdicts arrive as features of the decision engine, never as a verification
|
||||
status.
|
||||
|
||||
## What an Adjudication Feeds
|
||||
|
||||
- Recorded adjudications (a per-finding verdict with a reason) become decision-engine labels through
|
||||
the fail-closed source list `src/openultrasast/learn/sources.toml` (`[[source]] id =
|
||||
"adjudications"`). Aggregate counts label nothing; only per-finding verdicts do.
|
||||
- Plane verdicts (`agreed`, `rejected`, `disputed`) are features, never labels. "Rejected by the
|
||||
plane" is not a negative.
|
||||
- Rule changes go through the ledger: `ousast improve` (the validator and the detection gate, holdout
|
||||
pairs per round) or a registered experiment (`ousast learn experiment register|units|run|analyse`).
|
||||
Never a hand edit of `rules.toml` per miss or per false positive.
|
||||
|
||||
## Fix and Re-scan
|
||||
|
||||
1. Intake: restate the finding id, evidence level, affected path, and the acceptance criterion.
|
||||
2. Plan: the smallest defensive change that addresses the root cause.
|
||||
3. Implement: modify only the files the finding requires.
|
||||
4. Adversarial review: look for bypasses, regressions, false assumptions, and unrelated edits. Use a
|
||||
differential review for changes in auth, crypto, config, parsers, and sandbox behaviour.
|
||||
5. Reconcile: fix each review finding or document why it does not apply.
|
||||
6. Verify: a re-scan of the checkout in the same mode, or a replay of the change with
|
||||
`ousast pre-push --base <before> --head <after> --artifact <file> <repo>`, shows the finding gone
|
||||
and no new findings. That re-scan is the evidence. Nothing in the tool records `patch_validated`;
|
||||
do not claim it.
|
||||
|
||||
Guardrails:
|
||||
|
||||
- Do not rewrite modules when a local fix is enough.
|
||||
- Do not change public APIs unless the vulnerability requires it.
|
||||
- Do not call a finding fixed from model reasoning alone; the re-scan or replay decides.
|
||||
|
||||
Reference in New Issue
Block a user