mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
research: Semgrep CE and Opengrep as the quick tier, measured
Engine LGPL-2.1, shelling out carries no obligation; the registry rules are under the Semgrep Rules License since 2024-12-13 and cannot ship, only our own converted rules can. Measured 4.7 s startup and 5 to 10 ms per file; converted regex and taint rules fire for Python, PHP and JavaScript; C propagation and Groovy do not; one sanitizer semantic we refuse on purpose is accepted by Semgrep. Recommendation: a Semgrep arm on the valid pilot first. Recorded in the hunter-learning-loop research note and breadth.toml. Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
108dc2e5f3
commit
e55e44d981
@@ -252,3 +252,10 @@ status = "review"
|
||||
modules = ["push_scoring.py"] # push_inputs.py moved to benchmarks/push 2026-10-13; push_scoring stays (eligibility.py:18)
|
||||
lines = 462
|
||||
closes = "U3: move under benchmarks/push or delete if only tests use them"
|
||||
|
||||
[capability.quick-rules-and-overlay]
|
||||
status = "review"
|
||||
modules = ["findings.py", "ruleset", "semantic (taint, overlay, prove_filter)"]
|
||||
note = "maintainer 2026-10-13: 'semgrep oss is a good idea'; brief in benchmarks/measurements/2026-10-13-semgrep-brief/ says registry rules are not shippable, our converted rules are; 4.7 s startup floor"
|
||||
closes = "run the Semgrep arm on the valid pilot (benchmarks/valid/semgrep_arm.py); if it scores at least as well, retire findings.py, ruleset/*.py and the taint overlay (~1,250 lines) and amend safety-net.md's second-engine rule with the measurement"
|
||||
|
||||
|
||||
@@ -75,8 +75,18 @@
|
||||
and Perl grammars that ours handles by regex today. Adopt it as the grammar packaging in `semantic/extra.py`;
|
||||
map `.c` to the c grammar first (20 of the 56 rejections clear); cache the parser. `tree-sitter-languages` is
|
||||
ABI-broken under tree-sitter 0.26; universal-ctags has no pip build.
|
||||
- **Pending**: the Semgrep CE and Opengrep brief (`~/ousast-results/fossil/semgrep-brief.md`). Producers land behind
|
||||
the same contract; the ablation decides whether their columns stay.
|
||||
- **Result, Semgrep CE and Opengrep (benchmarks/measurements/2026-10-13-semgrep-brief/BRIEF.md)**: the engine is
|
||||
LGPL-2.1 and shelling out carries no obligation, but every Semgrep-maintained registry rule has been under the
|
||||
Semgrep Rules License since 2024-12-13 (internal use only, no distribution, not in competing products), so
|
||||
nothing but our own converted rules could ship. Measured here: 4.7 s startup floor and about 5 to 10 ms per
|
||||
file after it (the regex tier runs in-process under a second); converted regex and taint rules fire with
|
||||
traces for Python, PHP and JavaScript; C propagation through `strcpy` does not (propagators are Pro); Groovy has
|
||||
no parser; a Semgrep sanitizer silenced the `ORDER BY` shape our `quoted_only` flag refuses on purpose; guards,
|
||||
dispatch facts, obligations, shadow status and emit floors have no equivalent. A swap would retire about 1,250
|
||||
lines. Recommendation taken from the brief: option C first, a Semgrep arm on the valid pilot
|
||||
(`benchmarks/valid/semgrep_arm.py`, our rules converted, scored by the same scorer) before any decision on the
|
||||
quick tier; adopting it later means amending the "no second engine" rule in safety-net.md with that measurement.
|
||||
For this spec, Semgrep findings are a candidate feature column like any other producer.
|
||||
- **Research needed**: Semgrep registry rule licence terms for an open-source tool.
|
||||
|
||||
### Training data apart from the pilot
|
||||
|
||||
Reference in New Issue
Block a user