docs(kiro): archive the superseded roadmap; status notes on specs

A4: roadmap-2026-09-04.md moves to .kiro/archive/steering/ so it no longer
loads as steering (roadmap.md and safety-net.md replaced it); roadmap.md's
pointer follows it. overview.md's spec table is recounted from tasks.md and
spec.json (adds ai-service-plane, harnessx-removal, learned-decision-engine,
pre-push-safety-net; drops the stale "active one"), and the module line now
cites the regenerated audit: 150 modules, 145 load-bearing, 5 standalone,
0 orphaned; 1,964 tests collected.

A5: a dated status note at the top of the requirements (or brief) of 16
specs: retired, superseded, dormant or complete, and for
model-grounded-detection the removal of judge/calibrate/execution (closing
the Req 10 execution-tier option). spec.json phase corrected to
implementation-complete where every task is checked (model-grounded-
detection, propose-adjudicate-prove, tree-sitter-overlay-extra,
corpus-seeded-mechanisms, pair-corpus-honesty); openrouter-sast-harness gets
a spec.json with status retired. Nothing deleted, no spec moved.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 08:19:48 +02:00
co-authored by Claude Opus 5.5
parent 067adb9845
commit c8d2fefbb8
25 changed files with 65 additions and 19 deletions
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** dormant. Tasks were approved but none of the 24 was started; the absence-bug case it targets is handled in practice by the dominance arbiter (`model/dominance.py`) and `semantic/obligations/`. Reopen it only with a measured gap the arbiter misses.
## Introduction ## Introduction
AppSec engineers and maintainers of OpenUltraSAST scan web code, increasingly vibe-coded or agent-written, in which the dominant bugs are absences rather than flows: a handler queries a user-owned record without constraining it to the caller, a privileged route has no guard on its path, an identity is read from the request body instead of the authenticated context, a security control stays at its permissive default. Today the detector, the overlay, and the variant search model only source-to-sink flows, so these pairs score zero and cannot teach a mechanism (6 of 19 non-teaching vibe-py pairs; 15 of 29 agent-vfc rows carry absence labels). This spec adds obligations: an operation that must be discharged by a guard, an identity constraint, or a safe value before it is reached. The application's own policy, recovered from the consistency of its sibling handlers and optionally declared in a versioned file, says which operations carry which obligations. Violations are reported with the missing discharger and its evidence, measured leave-one-out on the absence pairs, and admitted into the detector through the existing improve-loop gate. AppSec engineers and maintainers of OpenUltraSAST scan web code, increasingly vibe-coded or agent-written, in which the dominant bugs are absences rather than flows: a handler queries a user-owned record without constraining it to the caller, a privileged route has no guard on its path, an identity is read from the request body instead of the authenticated context, a security control stays at its permissive default. Today the detector, the overlay, and the variant search model only source-to-sink flows, so these pairs score zero and cannot teach a mechanism (6 of 19 non-teaching vibe-py pairs; 15 of 29 agent-vfc rows carry absence labels). This spec adds obligations: an operation that must be discharged by a guard, an identity constraint, or a safe value before it is reached. The application's own policy, recovered from the consistency of its sibling handlers and optionally declared in a versioned file, says which operations carry which obligations. Violations are reported with the missing discharger and its evidence, measured leave-one-out on the absence pairs, and admitted into the detector through the existing improve-loop gate.
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** superseded by `model-grounded-detection` (spec.json `superseded_by`). 4 of 20 tasks were done and the tasks were never approved; none will be. Kept as decision history; nothing in src, tests or CI points here.
## Introduction ## Introduction
The detector's instability is structural, not statistical. Measured on 2026-09-06 over twelve injection pairs at The detector's instability is structural, not statistical. Measured on 2026-09-06 over twelve injection pairs at
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** complete, 17 of 17 tasks (spec.json phase corrected to `implementation-complete`). Its `[variants]` config keys were retired on 2026-10-02: the variant search they configured (`semantic/variants.py`) had already been deleted.
## Introduction ## Introduction
OpenUltraSAST's labeled pair corpus measures the detector but does not improve it, and its mechanism memory is empty because only sandbox-proven findings may write to it. This spec turns every trusted pair into a mechanism record, adds a structural variant search that proposes matches on any scanned tree, measures the corpus's own detection rate leave-one-out, and gives the improve loop a lever that can raise recall under the existing false-positive ceiling and per-profile holdout gate. OpenUltraSAST's labeled pair corpus measures the detector but does not improve it, and its mechanism memory is empty because only sandbox-proven findings may write to it. This spec turns every trusted pair into a mechanism record, adds a structural variant search that proposes matches on any scanned tree, measures the corpus's own detection rate leave-one-out, and gives the improve loop a lever that can raise recall under the existing false-positive ceiling and per-profile holdout gate.
@@ -1,9 +1,9 @@
{ {
"feature_name": "corpus-seeded-mechanisms", "feature_name": "corpus-seeded-mechanisms",
"created_at": "2026-09-05T06:18:31Z", "created_at": "2026-09-05T06:18:31Z",
"updated_at": "2026-09-05T16:30:00Z", "updated_at": "2026-10-02T00:00:00Z",
"language": "en", "language": "en",
"phase": "implementation", "phase": "implementation-complete",
"approvals": { "approvals": {
"requirements": { "requirements": {
"generated": true, "generated": true,
+2
View File
@@ -1,5 +1,7 @@
# Brief: flow-aware-ranking — spend the expensive query where the cheap evidence is strongest # Brief: flow-aware-ranking — spend the expensive query where the cheap evidence is strongest
> **Status 2026-10-02:** dormant. Brief only and unapproved; its 2026-09-12 amendment placed ranking inside the pre-push safety net, where the ranker now lives.
**Status:** brief only, unapproved. **Status:** brief only, unapproved.
## Current amendment — 2026-09-12 ## Current amendment — 2026-09-12
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** retired with HarnessX (see the note below). Its task 6.3 modules `stage_processors.py` and `slot_contract.py` were deleted on 2026-10-02: nothing imported them. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax > **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged. > (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
+2
View File
@@ -1,5 +1,7 @@
# Brief: ir-consolidation — stop maintaining an IR we did not need, and adopt one we do # Brief: ir-consolidation — stop maintaining an IR we did not need, and adopt one we do
> **Status 2026-10-02:** dormant. Brief only and unapproved. The subtraction it asks for still needs a measured recall comparison before the flat IR goes.
**Status:** brief only, unapproved. **Status:** brief only, unapproved.
## Why now ## Why now
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** retired, superseded by `model-grounded-detection` (spec.json `superseded_by`). 30 of 33 tasks were done; the open three will not be. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax > **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged. > (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
@@ -1,5 +1,7 @@
# Requirements Document: model-grounded-detection # Requirements Document: model-grounded-detection
> **Status 2026-10-02:** complete, 16 of 16 tasks (spec.json phase corrected to `implementation-complete`). On 2026-10-02 three of its modules were deleted because nothing imported them: `model/judge.py` (Req 8; `model/pipeline.py` does the judging), `model/calibrate.py` (Req 9; the decision engine's `learn/calibrate.py` is the live calibration) and `model/execution.py`. Deleting `execution.py` closes the Req 10 execution-tier option: a future execution tier is a new design, not this seam.
## Introduction ## Introduction
This feature supersedes `learning-harness` and `constrained-detector`. It replaces the assumption that the LLM This feature supersedes `learning-harness` and `constrained-detector`. It replaces the assumption that the LLM
@@ -1,9 +1,9 @@
{ {
"feature_name": "model-grounded-detection", "feature_name": "model-grounded-detection",
"created_at": "2026-09-08T00:00:00Z", "created_at": "2026-09-08T00:00:00Z",
"updated_at": "2026-09-08T10:00:00Z", "updated_at": "2026-10-02T00:00:00Z",
"language": "en", "language": "en",
"phase": "implementation", "phase": "implementation-complete",
"approvals": { "approvals": {
"requirements": { "requirements": {
"generated": true, "generated": true,
@@ -1,5 +1,7 @@
# Requirements: OpenRouter SAST Harness # Requirements: OpenRouter SAST Harness
> **Status 2026-10-02:** retired (see the note below). Its OpenCode skill `openultrasast-kiro-impl`, which still sent agents to this tasks.md, was deleted. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax > **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged. > (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
@@ -0,0 +1,7 @@
{
"feature_name": "openrouter-sast-harness",
"language": "en",
"status": "retired",
"updated_at": "2026-10-02T00:00:00Z",
"retired_note": "2026-09-30: HarnessX was removed by harnessx-removal; this spec had no spec.json before 2026-10-02 and is kept as decision history."
}
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** complete, 32 of 32 tasks (spec.json phase corrected to `implementation-complete`).
## Introduction ## Introduction
Engineers evolving OpenUltraSAST's overlay cannot trust the pair scoreboard: coverage detections are dropped, vfc labels mostly say `sink = "unknown"`, a quarter of the vfc corpus fails to parse because excerpts start mid-comment, two sast pairs are unlabelable as vendored, and there is no Node/TypeScript/Python web corpus and no agent-authored corpus. This spec makes the scoreboard honest per slice, per mechanism, and per provenance, adds three web and agent-authored slices, and lets the improve loop accept a change only when no provenance profile regresses. It supersedes `real-world-vfc-slice` for the corpus and scorer plane. Engineers evolving OpenUltraSAST's overlay cannot trust the pair scoreboard: coverage detections are dropped, vfc labels mostly say `sink = "unknown"`, a quarter of the vfc corpus fails to parse because excerpts start mid-comment, two sast pairs are unlabelable as vendored, and there is no Node/TypeScript/Python web corpus and no agent-authored corpus. This spec makes the scoreboard honest per slice, per mechanism, and per provenance, adds three web and agent-authored slices, and lets the improve loop accept a change only when no provenance profile regresses. It supersedes `real-world-vfc-slice` for the corpus and scorer plane.
+2 -2
View File
@@ -1,9 +1,9 @@
{ {
"feature_name": "pair-corpus-honesty", "feature_name": "pair-corpus-honesty",
"created_at": "2026-09-04T16:12:17Z", "created_at": "2026-09-04T16:12:17Z",
"updated_at": "2026-09-05T12:30:00Z", "updated_at": "2026-10-02T00:00:00Z",
"language": "en", "language": "en",
"phase": "implementation", "phase": "implementation-complete",
"supersedes": [ "supersedes": [
"real-world-vfc-slice" "real-world-vfc-slice"
], ],
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** complete, 22 of 22 tasks (spec.json phase corrected to `implementation-complete`).
## Introduction ## Introduction
OpenUltraSAST users scan **working trees they are writing**: mixed files, incomplete edits, generated code, framework wrappers, and no labels. Today the harness still *decides* with pattern inventory. The complexity map ranks; the sandbox can run; neither adjudicates “this sink is fed by a source” versus “this sink is hardcoded, sanitized, or unreadable.” OpenUltraSAST users scan **working trees they are writing**: mixed files, incomplete edits, generated code, framework wrappers, and no labels. Today the harness still *decides* with pattern inventory. The complexity map ranks; the sandbox can run; neither adjudicates “this sink is fed by a source” versus “this sink is hardcoded, sanitized, or unreadable.”
@@ -1,9 +1,9 @@
{ {
"feature_name": "propose-adjudicate-prove", "feature_name": "propose-adjudicate-prove",
"created_at": "2026-09-04T10:58:28Z", "created_at": "2026-09-04T10:58:28Z",
"updated_at": "2026-09-04T12:00:00Z", "updated_at": "2026-10-02T00:00:00Z",
"language": "en", "language": "en",
"phase": "implementation", "phase": "implementation-complete",
"approvals": { "approvals": {
"requirements": { "requirements": {
"generated": true, "generated": true,
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** dormant. 23 tasks generated, nothing approved; adopting Joern (`cpg/`, `model/taint.py`) delivered most of what it asked for. Reopen it only for a reachability gap Joern's taint does not cover.
## Introduction ## Introduction
OpenUltraSAST cannot yet steer effort where risk is reduced because it has no model of how an entry point reaches a sink. Reachability today means "inside a handler," taint is one hop inside one file, the LLM hunter cannot create a sandbox candidate, and the improve loop cannot add the facts that would raise recall. This spec adds a call graph and composed taint summaries over the existing `FunctionIR` records, attaches path-based reachability to findings and ranking, puts the hunter on paths with one checkable question per gap, turns model answers into gated facts, and ranks fix points by reachable risk removed. Concolic execution is out of scope; the sandbox remains the only place target code runs. OpenUltraSAST cannot yet steer effort where risk is reduced because it has no model of how an entry point reaches a sink. Reachability today means "inside a handler," taint is one hop inside one file, the LLM hunter cannot create a sandbox candidate, and the improve loop cannot add the facts that would raise recall. This spec adds a call graph and composed taint summaries over the existing `FunctionIR` records, attaches path-based reachability to findings and ranking, puts the hunter on paths with one checkable question per gap, turns model answers into gated facts, and ranks fix points by reachable risk removed. Concolic execution is out of scope; the sandbox remains the only place target code runs.
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** superseded by `pair-corpus-honesty` (spec.json `superseded_by`); its 12 tasks were done and the vfc pairs live on in that spec's corpus. Kept as decision history; nothing in src, tests or CI points here.
## Introduction ## Introduction
AppSec engineers cannot tell whether OpenUltraSAST improved on real product code. The labeled pair slices today are in-tree fixtures (`local`), small GitHub/SVEN functions (`github`), and textbook OWASP/Juliet files (`sast`). Operators care whether the harness fires on a real buggy snapshot of OpenSSL, Firefox, or Chromium and stays quiet after the patch. This spec adds a reviewed, isolated-function **vfc** slice drawn from public vulnerability-fixing commits in those projects, scored as an honesty dashboard (not a merge gate), so the tool can be evolved against labeled product code instead of cheat-sheet recall. AppSec engineers cannot tell whether OpenUltraSAST improved on real product code. The labeled pair slices today are in-tree fixtures (`local`), small GitHub/SVEN functions (`github`), and textbook OWASP/Juliet files (`sast`). Operators care whether the harness fires on a real buggy snapshot of OpenSSL, Firefox, or Chromium and stays quiet after the patch. This spec adds a reviewed, isolated-function **vfc** slice drawn from public vulnerability-fixing commits in those projects, scored as an honesty dashboard (not a merge gate), so the tool can be evolved against labeled product code instead of cheat-sheet recall.
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** retired (see the note below); all 31 tasks were done before HarnessX was removed. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax > **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged. > (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
@@ -1,5 +1,7 @@
# Requirements Document # Requirements Document
> **Status 2026-10-02:** complete, 13 of 13 tasks (spec.json phase corrected from `tasks-generated` to `implementation-complete`).
## Introduction ## Introduction
AppSec engineers scanning mixed working trees need overlay adjudication on Python, JavaScript, C/C++, and Java, not only on files a Python stdlib parse can read. Today a `standard` or `deep` scan leaves JS/C/Java inventory as `language_unsupported` even when a host parser CLI is present, so those proposals never promote or demote. This spec makes an **optional semantic extra** the way to turn on a real structured parse for those languages, while `quick` stays usable with no extra packages and a missing extra stays fail-closed. AppSec engineers scanning mixed working trees need overlay adjudication on Python, JavaScript, C/C++, and Java, not only on files a Python stdlib parse can read. Today a `standard` or `deep` scan leaves JS/C/Java inventory as `language_unsupported` even when a host parser CLI is present, so those proposals never promote or demote. This spec makes an **optional semantic extra** the way to turn on a real structured parse for those languages, while `quick` stays usable with no extra packages and a missing extra stays fail-closed.
@@ -1,9 +1,9 @@
{ {
"feature_name": "tree-sitter-overlay-extra", "feature_name": "tree-sitter-overlay-extra",
"created_at": "2026-09-04T14:00:00Z", "created_at": "2026-09-04T14:00:00Z",
"updated_at": "2026-09-04T15:00:00Z", "updated_at": "2026-10-02T00:00:00Z",
"language": "en", "language": "en",
"phase": "tasks-generated", "phase": "implementation-complete",
"approvals": { "approvals": {
"requirements": { "requirements": {
"generated": true, "generated": true,
+2
View File
@@ -1,5 +1,7 @@
# Brief: zero-with-a-reason — a finding count is meaningless without a denominator # Brief: zero-with-a-reason — a finding count is meaningless without a denominator
> **Status 2026-10-02:** dormant. Brief only and unapproved; not touched since 2026-09-10.
**Status:** brief only, unapproved. **Status:** brief only, unapproved.
## Why now ## Why now
+15 -8
View File
@@ -163,18 +163,25 @@ function-level pairs could not see any of them, and a repository showed all nine
## Where the specs stand ## Where the specs stand
Counted 2026-10-02 from each spec's `tasks.md` checkboxes and `spec.json`.
| spec | phase | state | | spec | phase | state |
|---|---|---| |---|---|---|
| `contributor-scan` | implementation | 14 done, 15 open — **the active one** | | `learned-decision-engine` | implementation | 11 of 17 tasks done |
| `pre-push-safety-net` | implementation | 67 of 79 tasks done |
| `contributor-scan` | implementation | 27 of 51 tasks done |
| `ai-service-plane` | implementation | 11 of 11 tasks done; the ax plane is the agentic path |
| `harnessx-removal` | complete | 8 of 8; HarnessX removed 2026-09-30 |
| `finding-feedback-loop` | design | requirements approved, design awaiting approval | | `finding-feedback-loop` | design | requirements approved, design awaiting approval |
| `model-grounded-detection` | complete | 16/16; built the arbiter this all rests on | | `model-grounded-detection` | complete | 16 of 16; built the arbiter this all rests on |
| `authorization-obligations` | tasks approved | 24 open — superseded in practice by the dominance arbiter | | `authorization-obligations` | dormant | tasks approved, 0 of 24 started; superseded in practice by the dominance arbiter |
| `constrained-detector` | tasks | 4 of 20, tasks unapproved | | `reachability-flow-model` | dormant | 23 tasks generated, nothing approved; largely delivered by adopting Joern |
| `reachability-flow-model` | tasks generated | 23 open, nothing approved — largely delivered by adopting Joern | | `flow-aware-ranking`, `zero-with-a-reason`, `ir-consolidation` | dormant | brief only, unapproved |
| `learning-harness` | implementation | 30 done, 3 open | | four others | complete | `pair-corpus-honesty`, `corpus-seeded-mechanisms`, `propose-adjudicate-prove`, `tree-sitter-overlay-extra` |
| six others | complete | `pair-corpus-honesty`, `corpus-seeded-mechanisms`, `propose-adjudicate-prove`, `three-stage-scan`, `real-world-vfc-slice`, `harnessx-self-improving-rulesets`, `tree-sitter-overlay-extra` | | six others | retired or superseded | `openrouter-sast-harness`, `three-stage-scan`, `harnessx-self-improving-rulesets`, `learning-harness` (retired with HarnessX), `constrained-detector`, `real-world-vfc-slice` (superseded) |
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax (`ai-service-plane`), > **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax (`ai-service-plane`),
> and evolutionary proposals come from the plane memory. The HarnessX specs in the table above are history. > and evolutionary proposals come from the plane memory. The HarnessX specs in the table above are history.
97 source modules, 90 load-bearing and 7 standalone capabilities; 814 tests. 150 source modules, 145 load-bearing and 5 standalone capabilities, 0 orphaned
(`benchmarks/measurements/2026-09-08-module-audit.json`, regenerated 2026-10-02); 1,964 tests collected.
+1 -1
View File
@@ -79,7 +79,7 @@ supersede this accepted sequence. Blocking deployment follows a later explicit d
The sections below preserve prior plans and measurements. Statements about the current bottleneck, The sections below preserve prior plans and measurements. Statements about the current bottleneck,
PHP coverage, rollout order or feature readiness are superseded by the current section above. PHP coverage, rollout order or feature readiness are superseded by the current section above.
**Rewritten 2026-09-09.** The previous version (kept as `roadmap-2026-09-04.md`) described a pre-Joern world — **Rewritten 2026-09-09.** The previous version (kept as `.kiro/archive/steering/roadmap-2026-09-04.md`, outside steering) described a pre-Joern world —
"adjudication is still Python-`ast` plus a tree-sitter CLI stub", "Joern-as-required deferred". Joern was "adjudication is still Python-`ast` plus a tree-sitter CLI stub", "Joern-as-required deferred". Joern was
adopted, the model layer was built, wired and measured, and two repositories have been scanned. See adopted, the model layer was built, wired and measured, and two repositories have been scanned. See
`overview.md` for what is true today; this is what happens next and in what order. `overview.md` for what is true today; this is what happens next and in what order.