marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

spec(hunter-learning-loop): a pattern engine's miss is never evidence

Browse Source 
A Semgrep, quick-rule or engine match may enter the feature table as
positive evidence; its absence is never a feature, filter, demotion or
gate on a hunter suspicion, and no acceptance rule counts it. The Semgrep
arm is a comparison only and may at most replace the regex quick tier.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-13 13:01:32 +02:00
co-authored by Claude Opus 5.5
parent c95b9241d8
commit d08dea19b8
2 changed files with 8 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/breadth.toml
+1 -1
View File
@@ -257,5 +257,5 @@ closes = "U3: move under benchmarks/push or delete if only tests use them"
status = "review" status = "review"
modules = ["findings.py", "ruleset", "semantic (taint, overlay, prove_filter)"] modules = ["findings.py", "ruleset", "semantic (taint, overlay, prove_filter)"]
note = "maintainer 2026-10-13: 'semgrep oss is a good idea'; brief in benchmarks/measurements/2026-10-13-semgrep-brief/ says registry rules are not shippable, our converted rules are; 4.7 s startup floor" note = "maintainer 2026-10-13: 'semgrep oss is a good idea'; brief in benchmarks/measurements/2026-10-13-semgrep-brief/ says registry rules are not shippable, our converted rules are; 4.7 s startup floor"
closes = "run the Semgrep arm on the valid pilot (benchmarks/valid/semgrep_arm.py); if it scores at least as well, retire findings.py, ruleset/*.py and the taint overlay (~1,250 lines) and amend safety-net.md's second-engine rule with the measurement" closes = "run the Semgrep arm on the valid pilot (benchmarks/valid/semgrep_arm.py) as a comparison; it may replace the regex quick tier only (same job: pattern matches), never act as a filter on other tiers. A Semgrep miss is never evidence (maintainer 2026-10-13). If it matches the quick tier at least as well, retire findings.py and ruleset/*.py and amend safety-net.md with the measurement"
.kiro/specs/hunter-learning-loop/design.md
+7
View File
@@ -106,6 +106,13 @@ tests/test_hunt_*.py table, producers (planted-advisory test), progr
ablation masks, acceptance rule, artifact ablation masks, acceptance rule, artifact
``` ```
### Rule: a pattern engine's miss is never evidence
Maintainer, 2026-10-13: "what does not work is that we dismiss things just because semgrep cannot find the bug.
if semgrep was enough we'd use it." A Semgrep (or quick-rule, or engine) match is positive evidence and may enter
the table as a feature ("matched here, rule X"). The absence of a match is not recorded as a feature, is never a
filter, a demotion, a down-weight or a gate on a hunter suspicion, and no acceptance rule may count it. The
Semgrep arm on the pilot is a comparison arm only.
## Components and interfaces ## Components and interfaces
| Component | Intent | Requirements | Depends on | Contract | | Component | Intent | Requirements | Depends on | Contract |
|---|---|---|---|---| |---|---|---|---|---|