marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-19 21:19:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: plain-English pass over the active specs

Browse Source 
Ran the humanizer rules over 79 prose files in the active specs (brief,
design, requirements, research and narrative docs). Removed em dashes,
significance inflation, forced rule-of-three, bold-label list items and
filler. Skipped tasks.md checklists and the retired specs (harnessx-*,
openrouter-sast-harness, unseen-repo-evaluation).

Every number, path, command, hash and identifier was preserved: checked
each file against HEAD for drift in numbers, inline code and code fences,
zero drift across all 79 files.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-12 18:16:41 +02:00
co-authored by Claude Opus 4.8
parent b08f372022
commit 39a1873e5b
79 changed files with 1103 additions and 1103 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/model-grounded-detection/requirements.md
+21 -21
View File
@@ -6,12 +6,12 @@
This feature supersedes `learning-harness` and `constrained-detector`. It replaces the assumption that the LLM
is a noisy oracle to be averaged, floored and gated with an architecture in which a **deterministic program
model arbitrates the LLM's claims**. The arbiter is a Code Property Graph (CPG) — an intermediate language
merging AST, control-flow graph, program-dependence graph and call graph — over which taint reachability and
model arbitrates the LLM's claims**. The arbiter is a Code Property Graph (CPG), an intermediate language
merging AST, control-flow graph, program-dependence graph and call graph, over which taint reachability and
guard dominance produce verdicts the LLM's assertion is no longer needed for. The measurements that force this
design are committed: the candidate ceiling (96.6% of labeled functions have an enumerable site), and the
entailment ceiling (a deterministic model *decides* only 2.2% today on our flat AST-projection IR, ~24-35%
with deeper taint — the distance to a real CPG). Context: `brief.md`,
with deeper taint, the distance to a real CPG). Context: `brief.md`,
`benchmarks/measurements/2026-09-06-trajectory-divergence.json`, `2026-09-06-candidate-ceiling-all-slices.json`,
`2026-09-08-model-entailment-ceiling.json`.
@@ -22,15 +22,15 @@ overgrown, superseded surface is how this session logged eighteen defects.
## Boundary Context
- **In scope**: removing the noise architecture and the superseded arbiter; reducing the test suite to
- In scope: removing the noise architecture and the superseded arbiter; reducing the test suite to
surviving behavior; auditing the whole tree for orphaned modules; adopting a CPG engine (Joern) as the
model-layer substrate behind an optional extra; the evidence ladder as the verdict type; taint + dominance
verdicts; porting our taxonomy and source/sink/sanitizer/guard models to the CPG; the LLM as a proposer
checked against the model; the corpus as the model's calibration set; per-slice reporting.
- **Out of scope**: building a dataflow/CFG engine by hand (the CPG is adopted, §Req 4); the execution tier
- Out of scope: building a dataflow/CFG engine by hand (the CPG is adopted, §Req 4); the execution tier
itself (deferred to Clearwing, `execution_confirmed` rung only, Req 10); prompt optimisation, reflective
evolution, any optimiser (deleted, not deferred); fine-tuning weights.
- **North star, not v1**: pure `model_entailed`-as-arbiter. The entailment ceiling says the model decides a
- North star, not v1: pure `model_entailed`-as-arbiter. The entailment ceiling says the model decides a
minority today; v1 is the *layered* arbiter (entail where it can, corroborate the taint band, honest
suspicion for the rest). This is stated so the requirements are not read as promising a soundness the
measurement denies.
@@ -45,12 +45,12 @@ so that nothing is maintained, tested, or built upon that this feature's measure
#### Acceptance Criteria
1. The system shall remove the round loop, the per-family noise floor, the regression budget, the acceptance
rule, the sign test, the evolve journal and the meta-agent proposer — concretely `learning/rounds.py`,
rule, the sign test, the evolve journal and the meta-agent proposer, concretely `learning/rounds.py`,
`learning/acceptance.py`, `learning/proposer.py`, `learning/journal.py`, and the K-run / floor / sign-test
machinery in `learning/scoring.py`.
2. The system shall remove the mechanism-optimisation loop — the mechanism store and lever in
2. The system shall remove the mechanism-optimisation loop (the mechanism store and lever in
`improve/evolve.py`, and the shape-search machinery in `semantic/mechanisms.py` and the optimisation
surface of `semantic/variants.py` — while preserving the *security models* those modules carried (the
surface of `semantic/variants.py`), while preserving the *security models* those modules carried (the
guard vocabulary and source/sink classification) for porting under Req 7.
2b. The system shall remove the per-family evolvable detector configuration and its canary/verifier surface
(`learning/detectors.py` config-evolution parts, `learning/canaries.py`, `learning/verifiers.py`,
@@ -70,7 +70,7 @@ ratchet that pins deleted mechanisms and the suite is once again a description o
1. When a mechanism is removed under Req 1, the tests that existed only to pin its behavior shall be removed
with it, in the same change, not left skipped or xfailed.
2. Every surviving test shall map to a surviving requirement — of this spec or of a still-shipping feature —
2. Every surviving test shall map to a surviving requirement (of this spec or of a still-shipping feature)
and a test that maps to nothing shall be removed.
3. The reduced suite shall be smaller than the current 832 tests, and the change shall report the count before
and after with the requirement each removed test cluster served.
@@ -78,7 +78,7 @@ ratchet that pins deleted mechanisms and the suite is once again a description o
determinism test (Req 6.4), the per-slice honesty tests, and the two ceiling measurements' regeneration
tests.
5. A test that asserts a shape "does nothing" or greps `--help`, or that passes with its capability absent,
shall be removed rather than kept — the session logged several; none survive.
shall be removed rather than kept. The session logged several; none survive.
### Requirement 3: Audit the whole tree for outgrowth
@@ -89,7 +89,7 @@ found and removed rather than carried indefinitely.
1. The system shall classify every source module as one of: load-bearing for the model-grounded architecture;
a retained standalone capability with its own users; or orphaned.
2. If a module is orphaned — no importer, no CLI entry, no gate dependency, no surviving requirement — then the
2. If a module is orphaned (no importer, no CLI entry, no gate dependency, no surviving requirement), then the
system shall remove it and record the classification.
3. The system shall name, for the audit, the older-phase subsystems whose status is not obvious
(`fusion.py`, `mcp.py`, `skills.py`, `harness_ext.py`, `hunter_harness.py`) and give each an explicit
@@ -122,7 +122,7 @@ model result is never confused with an LLM suspicion.
#### Acceptance Criteria
1. The system shall assign every finding one rung: `suspicion` (a site was enumerated / the LLM flagged it),
`model_corroborated` (the LLM's claim is consistent with the CPG — the source it names reaches the sink
`model_corroborated` (the LLM's claim is consistent with the CPG, the source it names reaches the sink
through resolved dataflow, or a guard its siblings carry is absent), `model_entailed` (the CPG alone
establishes it, independent of trusting the LLM), or `execution_confirmed` (a sandbox reproduced it).
2. The system shall establish `model_corroborated` and `model_entailed` from the CPG deterministically, with
@@ -145,21 +145,21 @@ project measured is removed on the band the model covers.
4. When the system runs the model layer twice over the same CPG, the sequence of verdicts shall be identical,
and a test shall assert it.
5. The system shall report, per family and per slice, the fraction of the corpus it can `model_entail` and
`model_corroborate` — the regeneration of the committed entailment ceiling — and shall name where it falls
`model_corroborate` (the regeneration of the committed entailment ceiling), and shall name where it falls
back to `suspicion`.
### Requirement 7: Port the taxonomy and security models onto the CPG
**Objective:** As a maintainer, I want our security knowledge — the classes we target and how their bugs look
— expressed as queries over the adopted CPG, so that the contribution is the modelling, not the engine.
**Objective:** As a maintainer, I want our security knowledge (the classes we target and how their bugs look)
expressed as queries over the adopted CPG, so that the contribution is the modelling, not the engine.
#### Acceptance Criteria
1. The system shall retain the closed family taxonomy as the routing key, unchanged in meaning.
2. The system shall express the source, sink and sanitizer models for the web/logic CWEs as CPG taint
specifications, seeded from the models the deleted modules carried.
3. The system shall express the obligation / absence model — an operation on a protected resource, and the
guards that discharge it — as CPG dominance queries, as the arbiter for the classes that never crash.
3. The system shall express the obligation / absence model (an operation on a protected resource, and the
guards that discharge it) as CPG dominance queries, as the arbiter for the classes that never crash.
4. The system shall keep the verifier boundary: the models and gold labels stay outside anything an LLM or a
future optimiser may write.
@@ -177,7 +177,7 @@ with its claim checked against the model, so that there is no oracle to average.
`model_corroborated`.
4. When the model can neither confirm nor contradict, the system shall report the claim at `suspicion`, and
this band is where a future execution tier earns its place.
5. The system shall contain no K-run averaging, no noise floor, no acceptance rule, no evolve loop — the model
5. The system shall contain no K-run averaging, no noise floor, no acceptance rule, no evolve loop. The model
check replaces all of them.
### Requirement 9: The corpus is the model's calibration set
@@ -188,7 +188,7 @@ vote, so that a gap is a gap in the model, named and fixable.
#### Acceptance Criteria
1. The system shall use the vulnerable-parent / fix-commit pair as ground truth for whether the model can
distinguish the two — the fix is what the model should be able to see.
distinguish the two. The fix is what the model should be able to see.
2. If the model cannot distinguish a pair's two sides, then the system shall record it as a named model gap
with the reason, not as a detector miss to be averaged away.
3. The system shall keep the per-slice split with the real-world slices deciding, and shall not present the
@@ -204,7 +204,7 @@ arbitrate, without building or requiring it at v1.
#### Acceptance Criteria
1. The system shall treat `execution_confirmed` as an optional rung reached only for classes where the model
layer cannot decide — memory-safety in C first.
layer cannot decide, memory-safety in C first.
2. When execution confirmation is used, the system shall adopt it from Clearwing (the existing fork), not
reimplement the sandbox lifecycle, container pooling, sanitizer images or PoC stability.
3. The system shall not require the execution tier for any web/logic class at v1.