mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-20 13:39:27 +00:00
docs(specs): approve the group-2 amendments, generate finding-feedback-loop requirements
**Amendments approved and folded into the numbered requirements** rather than left as a proposal appended to the file: - Req 5.5 — a report proportionate to the evidence behind a finding. A rule whose only evidence is a text match is summarised past a stated number of sites; a finding a proposer reasoned about keeps its detail whatever its rung; nothing leaves findings.json, which is the record a baseline is compared against. - Req 5.6 — a family that cannot decide a class states the limit, and no scan presents an absence of findings as evidence of absence. - Req 4.6 — at least one pinned checkout carries a vulnerability a family can decide in principle, so a scan producing nothing is distinguishable from an engine that cannot see. Each carries the evidence that produced it, and the amendment record keeps what group 2 confirmed rather than changed: Req 8's overfitting warning earned its place twice, Req 3.3's honesty counters made an invisible failure visible, and Req 11.2's gates held through all nine defect fixes. Req 5.5 is half implemented. Task 2.11 made pattern matches proportionate; task 2.17 now carries the model layer's own suspicion band, which is 38 of vampi's 60 findings and is an enumerated candidate the graph could NOT decide with a judge's opinion attached -- a pattern match with an opinion, not a reasoned claim, and weaker evidence than the obligations checker's structural output. **finding-feedback-loop requirements generated**, seven of them: a dismissal file the developer owns and the tool never writes, rejecting one without a reason because a suppression list without reasons is a graveyard nobody can audit; a counter-example ledger whose entries are runnable regression cases, grouped by shared cause so a recurring bug is visible as one; a model that proposes a cause into a review queue and decides nothing, validated by reverting the three known matcher fixes and measuring what it recovers; dismissals plus a recipe's known vulnerabilities as one split evaluation set with the holdout unread and asserted; evolution scoped to the judge and residual questions with the fact tables off limits to any automated means; an adoption that reports what it traded and reverts itself when net-negative; and the loop inheriting the corpus blind spot, so its evidence base is repositories and it states how few it has. Phase: requirements, generated, not approved. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
72932ed879
commit
bccc89be52
@@ -307,6 +307,21 @@
|
||||
- _Requirements: 3.1, 8.2_
|
||||
- _Depends: 2.15_
|
||||
|
||||
- [ ] 2.17 The model's own suspicion band is a pattern match with an opinion attached
|
||||
- **Req 5.5's other half.** Task 2.11 made pattern matches proportionate and left this untouched, so vampi
|
||||
still emits 61 sections for a 520-line application -- 38 of them model suspicions, roughly one per
|
||||
fourteen lines of code.
|
||||
- The epistemic case for treating them as reasoned claims does not survive inspection. A model suspicion is
|
||||
what the enumerator proposed at a site the graph could NOT decide, affirmed by a judge with no coverage
|
||||
there. That is a pattern match with an opinion attached, and it is weaker evidence than the obligations
|
||||
checker's output, which comes from a structural analysis rather than a guess.
|
||||
- So group them by family the way 2.11 groups pattern rules, and keep obligation findings and everything
|
||||
at `model_corroborated` or above in full. Expect vampi to fall from 61 sections to roughly 23.
|
||||
- Measure both repositories, and measure what it costs: if a real finding only ever appeared at model
|
||||
suspicion, this hides it, and that is the number that decides whether the rule is right.
|
||||
- _Requirements: 5.1, 5.2, 5.5_
|
||||
- _Depends: 2.11_
|
||||
|
||||
## Group 3 — Ship it
|
||||
|
||||
- [ ] 3.1 Contributor output
|
||||
|
||||
Reference in New Issue
Block a user