marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-22 14:39:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: add kiro spec-driven tooling and steering docs

Browse Source 
Add the Kiro-style agentic SDLC tooling used to drive this project: the
.opencode/skills/kiro-* skill set (discovery, spec init/requirements/design/
tasks, validation, implementation, review, steering), the .kiro/settings
templates, and AGENTS.md describing the workflow. Update the spec docs
(assessment/design/requirements/tasks) to current state.

Scope tests/test_opencode_skills.py to the project's own openultrasast-*
skills so the third-party kiro-* tooling (which uses a different frontmatter
convention) no longer breaks the frontmatter check.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-06-21 14:28:19 +02:00
co-authored by Claude Opus 4.8
parent 8fe8911020
commit 6699f1f8f9
58 changed files with 5509 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/openrouter-sast-harness/tasks.md
+38 -1
View File
@@ -130,6 +130,42 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
- [x] Every hunter trajectory is persisted and linked to finding artifacts.
- [ ] False-positive learning updates ranking calibration from verifier outcomes.
- [ ] Fusion can be triggered for high-impact, contradictory, or difficult findings.
- [ ] Standard mode uses real language-aware LLM hunters instead of reusing quick static findings.
## Gate: `benchmark_grounded_finder`
- [ ] Benchmark manifests exist for JavaScript/Node/web, Python/web, Java, and C/C++ vulnerable projects.
- [ ] Benchmark runner executes quick, standard, and deep modes as supported and preserves benchmark artifacts.
- [ ] Benchmark reports include recall, missed expected findings, false-positive reasons, runtime, and external baseline deltas.
- [ ] Missed benchmark findings feed calibration records for rules, ranking, retrieval, hunters, SARIF ingestion, dynamic probes, or skill routing.
- [ ] At least one realistic vulnerable project and one curated CWE corpus are tracked per supported language family.
## Phase 6A: Benchmark Corpus And Scoreboard (`benchmark_grounded_finder` prerequisite)
- [x] Define benchmark manifest schema with source, language, framework, setup, expected vulnerabilities, known noise, scan modes, and optional baselines.
- [x] Implement benchmark run directory creation under `.openultrasast/benchmarks/<benchmark-run-id>/`.
- [x] Implement finding-to-ground-truth matching by CWE, rule ID, file, function, sink, and evidence text.
- [x] Implement benchmark metrics for recall, precision when possible, false-positive reasons, missed findings, runtime, model usage, and artifact links.
- [x] Add benchmark fixtures or manifests for JavaScript/Node/web, Python/web, Java, and C/C++.
- [x] Add external baseline ingestion for SARIF or normalized findings from Clearwing, Semgrep, CodeQL, clang-tidy, Bandit, npm audit, or language-appropriate tools.
- [x] Add tests proving benchmark misses are recorded as calibration inputs without automatically suppressing unrelated findings.
## Gate: `explain_and_fix_workflow`
- [ ] Explain mode can explain a finding or benchmark miss with source evidence, reachability status, missing evidence, secure pattern, and prevention guidance.
- [ ] Explain mode supports concise, learner, and reviewer audience levels without changing evidence levels.
- [ ] Interactive fixing selects relevant security skills and proposes a minimal patch plan without silently mutating the target repository.
- [ ] Fix guidance links to validation commands and required evidence transitions.
## Phase 6B: Explain Mode And Interactive Fixing (`explain_and_fix_workflow` prerequisite)
- [ ] Add explain artifact schema for finding ID, source refs, evidence refs, audience level, prevention guidance, selected skills, and validation advice.
- [ ] Add `ousast explain <finding-id> --scan <scan-id>` CLI command.
- [ ] Add explain support for benchmark misses and imported SARIF findings.
- [ ] Implement language/vulnerability skill routing for JavaScript/Node/web, Python/web, Java, and C/C++.
- [ ] Add interactive fixing plan schema with confirmation gates, selected skills, proposed patch scope, validation commands, and evidence requirements.
- [ ] Ensure interactive fixing cannot mark patches validated unless the patch oracle and sandbox validation gates pass.
- [ ] Add tests that explanation does not upgrade evidence level and patch guidance does not mutate repositories by default.
## Phase 7: Docker Sandbox (`sandboxed_dynamic_harness` prerequisite)
@@ -209,12 +245,13 @@ Do not report OpenUltraSAST as a working security harness until `usable_harness_
- [ ] Narrow MCP tools cover scan, status, findings, evidence, artifacts, patch proposal, and report export.
- [ ] OpenCode workflows can drive scan, triage, fusion, and patch audit without arbitrary shell exposure.
- [ ] Cost controls, retries, redaction, docs, examples, and CI gates are in place.
- [ ] OpenCode workflows can drive benchmark runs, finding explanations, and interactive fix proposals.
## Phase 12: MCP And OpenCode Commands (`opencode_product` prerequisite)
- [x] Add project-local opencode skills for scan, triage, fix audit, and implementation steering.
- [ ] Implement narrow MCP server.
- [ ] Add `openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `propose_patch`, and `export_report` tools.
- [ ] Add `openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`, `explain`, `propose_patch`, and `export_report` tools.
- [ ] Add opencode command docs for quick, standard, deep, verify, and patch workflows.
- [ ] Ensure MCP tools never expose arbitrary shell execution.
- [ ] Add integration test for MCP tool listing and scan status.