mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
measure: Semgrep arm on the valid pilot, our rules and facts converted
0 of 12 positives; 12.6 false alarms per 100 ordinary pushes (2.1 on changed lines only). The misses come from what the rules and facts express (access control lives in obligations, which do not convert; sources cover request objects only), not from the engine. Comparison arm only: a Semgrep miss is never evidence against a hunter finding. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Marius Ciepluch
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
d08dea19b8
commit
c5bf52f47d
@@ -0,0 +1,160 @@
|
||||
# Semgrep CE as a third arm on the valid pilot (2026-10-13)
|
||||
|
||||
The question from `../2026-10-13-semgrep-brief/BRIEF.md` (option C): does Semgrep, running our own
|
||||
rules and facts, catch more on the valid pilot than our engine does? It does not. It caught none of the
|
||||
12 positives and raised false alarms on 12.6 of every 100 ordinary pushes.
|
||||
|
||||
Instrument: semgrep 1.179.0 CE (linuxbrew), `-j 1`, `--timeout 60 --max-memory 1500`, offline flags.
|
||||
Rules: `rules/` here (81 rules, from `python -m benchmarks.valid.semgrep_arm rules`). Records:
|
||||
`~/ousast-results/valid/pilot/semgrep/` (252 records, outside the repo). Same frozen manifest
|
||||
(`benchmarks/valid/pilot/pool-valid.toml`, slice 1) and same scorer as the other arms.
|
||||
|
||||
## Headline
|
||||
|
||||
| Arm | Recall (12 positives) | False alarms per 100 pushes (Wilson 95%) | Precision |
|
||||
|---|---|---|---|
|
||||
| (a) hook as shipped, 163 done | 0/12, 0.0% (0.0-24.3) | 0.7 (0.1-3.7), n=149 | 0.0% (0/1) |
|
||||
| (b) hunter, raw | 6/12, 50.0% (25.4-74.6) | 92.5 (88.4-95.2), n=239 | 2.6% |
|
||||
| (b') hunter, classifier-filtered | 5/12, 41.7% (19.3-68.0) | 54.4 | 3.6% |
|
||||
| **(c) Semgrep, our rules + facts** | **0/12, 0.0% (0.0-24.3)** | **12.6 (8.9-17.4), n=239**; cluster 3.8-23.4, ICC 0.28 | **0.0% (0/33)** |
|
||||
|
||||
`metrics_semgrep.json`: "recall 0.0% (Wilson 0.0-24.3), false alarms 12.6 per 100 pushes, precision
|
||||
0.0%; exploit-verified 0 of 5 (5 not yet proved)". Exploit-verified is not applicable: the 5 flagged
|
||||
positives are all wrong-family hits, none was a catch.
|
||||
|
||||
The (c) false-alarm rate is not like-for-like with (a). The hook keeps a quick hit only on a changed
|
||||
line or inside a changed function (`push/quick.py:122-130`); this arm counts every result anywhere in a
|
||||
changed file, as the run instructions asked. Restricting the same records afterwards to results on a
|
||||
changed line gives 5/239 = 2.1 per 100 (Wilson 0.9-4.8); within 10 lines of a change, 9/239 = 3.8
|
||||
(2.0-7.0). Scoped either way it is still a higher rate than (a), and recall stays at zero.
|
||||
|
||||
## Per positive
|
||||
|
||||
| Repo | Family | (a) hook | (b) raw | (c) Semgrep | What Semgrep reported |
|
||||
|---|---|---|---|---|---|
|
||||
| bytebase/dbhub | access_control | miss | miss | miss | `js-command-exec` at `sql-parser.ts:87`: `dollarQuoteOpenRegex.exec(...)`, a RegExp call |
|
||||
| contentful/contentful-mcp-server | untrusted_destination | miss | miss | miss | nothing |
|
||||
| fastapi-users/fastapi-users | access_control | miss | miss | miss | nothing |
|
||||
| i18next/i18next-http-middleware | output_encoding | miss | hit | miss | nothing |
|
||||
| imzbf/md-editor-v3 | output_encoding | miss | hit | miss | `js-command-exec` at `code/index.ts:37`: `re.exec(info)`, a RegExp call |
|
||||
| jupyter-lsp/jupyterlab-lsp | path | miss | hit | miss | nothing |
|
||||
| matrix-org/matrix-appservice-bridge | access_control | miss | miss | miss | nothing |
|
||||
| nvbn/thefuck | path | miss | hit | miss | nothing |
|
||||
| ondata/ckan-mcp-server | untrusted_destination | miss | hit | miss | nothing |
|
||||
| webpack/webpack-bundle-analyzer | output_encoding | miss | miss | miss | nothing |
|
||||
| zenitram/malla | output_encoding | miss | hit | miss | nothing |
|
||||
| zereight/gitlab-mcp | path | miss | miss | miss | `js-command-exec` + taint `sink:javascript:exec` at `index.ts:12429`: `/^Bearer.../i.exec(authHeader)` |
|
||||
|
||||
No rule origin caught anything. Why the positives are silent: three are access_control, which only the
|
||||
obligations express, and none of those convert. The four output_encoding positives are a value reaching
|
||||
`innerHTML` from a function argument or a markdown renderer, and the taint rules' sources are request
|
||||
objects (`req.query`, `request.args`). The intra-function CE taint never sees an argument as a source.
|
||||
The path and destination positives take their input from tool arguments or archive members, which are
|
||||
also not modelled sources. Our hook misses these 12 for the same reason. Swapping the engine does not
|
||||
change the vocabulary.
|
||||
|
||||
## False alarms (239 ordinary pushes, 30 flagged)
|
||||
|
||||
Pushes flagged, by rule origin: `js-command-exec` 16, `python-sql-injection` 8, `sink:javascript:exec`
|
||||
8, `python-unsafe-eval` 3, `sink:javascript:html.response` 2, `python-os-command` 1,
|
||||
`sink:javascript:prototype.merge` 1. The results are 95 in total: 90 injection, 3 prototype, 2
|
||||
output_encoding. Only 18 carry a taint trace; the other 77 are regex.
|
||||
|
||||
The flags come from five repos: malla 10, gitlab-mcp 10, webpack-bundle-analyzer 4, dbhub 3,
|
||||
matrix-appservice-bridge 2, thefuck 1. An ICC of 0.28 means a handful of files drive the rate. What the
|
||||
lines are:
|
||||
|
||||
- `js-command-exec` (`\b(exec|execSync)\s*\(`) and the taint sink `$OBJ.exec(...)` match `RegExp.exec`.
|
||||
In gitlab-mcp the request header is a taint source, so `/regex/.exec(authHeader)` reads as command
|
||||
injection with a dataflow trace. The rest are `childProcess.execSync` in test files.
|
||||
- `python-sql-injection` matches multi-line `cursor.execute(` calls built from f-strings or constants in
|
||||
malla's repositories. The regex cannot tell a constant from input.
|
||||
- `python-unsafe-eval` matched `eval()` inside a comment. The quick engine drops matches in comments;
|
||||
`pattern-regex` does not (see "not carried" below).
|
||||
- `sink:javascript:prototype.merge` matched `Map.set(...)`. lodash's `set` converts to `$OBJ.set(...)`.
|
||||
- `sink:javascript:html.response` matched `res.send({token, userId})`, an object Express sends as JSON.
|
||||
|
||||
## Runtime
|
||||
|
||||
251 Semgrep runs: mean 8.5 s per change, median 7.7 s, min 5.6 s, max 37.3 s (gitlab-mcp, a 453 KB
|
||||
`index.ts`). Total 35.6 min of Semgrep wall time; the whole run (with checkouts) took 36 min,
|
||||
12:31 to 13:07. 846 target files and 24.8 MB were read before the runs. Most of the per-change cost is
|
||||
Semgrep's fixed startup, about 5 s (the brief measured 4.7 s). The in-process regex tier runs in under a
|
||||
second.
|
||||
|
||||
## Instrument
|
||||
|
||||
- Every run read its targets and printed bytes per file first. No run returned faster than 1 s
|
||||
(`suspect_fast` 0). Every Semgrep exit was 0. For each change, `paths.scanned` is recorded next to
|
||||
the bytes read.
|
||||
- 1 instrument failure (0.4%, under the scorer's 5% limit): a matrix-appservice-bridge ordinary change
|
||||
whose four changed files were all deleted at head (`no_readable_targets`).
|
||||
- 30 changed files were missing at head and skipped.
|
||||
- 103 changes had targets Semgrep did not scan because no rule covers that language: `.md` 94, `.json`
|
||||
72, no extension 16, `.html` 13, `.yml` 11, `.css` 10, and others. Our hook does not read these
|
||||
either.
|
||||
- 2 PartialParsing errors (TypeScript `import("...")` types, a regex literal). Both files were still
|
||||
scanned.
|
||||
- Smoke test before the run: the converted rules fired with traces on hand-written Python, JS/TSX, Java,
|
||||
C and PHP samples (`os.system`, `innerHTML`, `dangerouslySetInnerHTML`, `executeQuery`, `strcpy`,
|
||||
`echo $x;`, `include $_GET`). So the zero recall is not a dead ruleset.
|
||||
|
||||
## Conversion report (`rules/conversion_report.json`)
|
||||
|
||||
- Regex rules: 41 of 52 converted to `pattern-regex`. 11 were skipped: 5 shadow (`php-sql-call-composition`,
|
||||
`php-code-eval`, `php-unserialize`, `php-maybe-unserialize`, `php-xss-echo-input`) and 6
|
||||
`*-language` fallbacks (inactive while their tagged rule is active). No rule targets non-code
|
||||
files, and none was dropped by an emit floor (the arm uses the hook's default floor, 0.0; `--min-precision`
|
||||
sets one). Groovy has no parser and becomes `languages: [generic]` with `paths.include: [*.groovy, *.tpl]`.
|
||||
- One rewrite: `php-sql-sprintf-unescaped` uses `(?:...){0,2000}?`, which PCRE2 rejects as "regular
|
||||
expression is too large". It became `*?`, so the window is now the whole statement instead of 2000
|
||||
characters.
|
||||
- Facts: 40 taint/search rules (JS 10, PHP 10, Python 10, Java 7, C 3), one per (language, sink id,
|
||||
CWE). Sources come from every source fact of that language; sanitizers come from cleansing sanitizer
|
||||
facts. `weak_literals` sinks (hashlib, MessageDigest) become search rules with a `metavariable-regex`.
|
||||
Format sinks focus their format argument, and parameterized calls (`execute`, `query`) focus a
|
||||
single-argument call.
|
||||
- Not convertible, skipped and named:
|
||||
- `guard` sanitizers (`php:allowlist` = `in_array`, `python:url_allowlist`): safety depends on
|
||||
dominance.
|
||||
- `parameterized` sanitizer facts (4): a call shape, expressed as the focused sink form instead.
|
||||
- `literal_format_arg` (C): expressed by focusing the format argument.
|
||||
- `dispatch:php:wordpress_hooks`: string-keyed registry routing.
|
||||
- `layout` facts (2): scope policy, not rules.
|
||||
- `v-html`: a template directive.
|
||||
- All 37 obligation tables: an absence has no pattern.
|
||||
- Lossy (converted, but with weaker semantics):
|
||||
- `quoted_only` `sql_escape` became a full sanitizer. This silences the CVE-2024-1071 shape the project
|
||||
keeps.
|
||||
- `prefix_fixes_origin` on 4 destination sinks (JS/Python outbound requests and redirects) is
|
||||
dropped. A fixed-origin prefix no longer clears the value; Django's `reverse` anchors became
|
||||
sanitizers.
|
||||
- Not carried at all: comment skipping, framework priors as a switch, `min_evidence_level`, ranking, the
|
||||
rung.
|
||||
|
||||
## What this says
|
||||
|
||||
Running the same vocabulary through Semgrep's engine adds no catches on these 12 positives. It adds
|
||||
false alarms, from rules that the quick tier keeps quiet with comment skipping and changed-line scoping.
|
||||
The misses come from the vocabulary (sources limited to request objects, no access_control without
|
||||
obligations), not from the engine. This is the measured reason the brief asked for, and it points
|
||||
against option A/B: the steering rule ("do not ... add a second engine") stands. If the arm is reused,
|
||||
fix the cheap false alarms first: exclude `RegExp.exec`, skip comments, and scope results to changed
|
||||
lines.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PYTHONPATH=$PWD
|
||||
.venv/bin/python -m benchmarks.valid.semgrep_arm rules --out ~/ousast-results/valid/pilot/semgrep-rules/
|
||||
for only in vulnerability ordinary; do
|
||||
.venv/bin/python -m benchmarks.valid.semgrep_arm run benchmarks/valid/pilot/pool-valid.toml --slice 1 \
|
||||
--repos ~/ousast-results/valid/pilot/repos_hist.json --rules ~/ousast-results/valid/pilot/semgrep-rules \
|
||||
--out ~/ousast-results/valid/pilot/semgrep --only $only
|
||||
done
|
||||
.venv/bin/python -m benchmarks.unseen.score ~/ousast-results/valid/pilot/semgrep --out ~/ousast-results/valid/pilot/scored_semgrep.json --slice-id 1
|
||||
.venv/bin/python -m benchmarks.valid.metrics ~/ousast-results/valid/pilot/semgrep --scored ~/ousast-results/valid/pilot/scored_semgrep.json --out ~/ousast-results/valid/pilot/metrics_semgrep.json
|
||||
```
|
||||
|
||||
The ordinary pass exits 2 because of the one instrument failure; the run still finishes and resumes
|
||||
past done records.
|
||||
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"exploit_verified": {
|
||||
"count": 0,
|
||||
"flagged_positive_pushes": 3,
|
||||
"flagged_positives": 5,
|
||||
"n": 0,
|
||||
"not_yet_proved": [
|
||||
"bytebase-dbhub-f2624a2ead8d31ca",
|
||||
"imzbf-md-editor-v3-14658213915dbde5",
|
||||
"imzbf-md-editor-v3-b9c833a3524a57a9",
|
||||
"zereight-gitlab-mcp-2367b459f8c3802b",
|
||||
"zereight-gitlab-mcp-768c7d5a32ccc95b"
|
||||
],
|
||||
"rate": null,
|
||||
"status": "not yet proved",
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"false_alarms_per_100_pushes": {
|
||||
"cluster95": [
|
||||
3.75,
|
||||
23.430962343096233
|
||||
],
|
||||
"count": 30,
|
||||
"icc": 0.27735846730536884,
|
||||
"n": 239,
|
||||
"rate": 12.552301255230125,
|
||||
"wilson95": [
|
||||
8.93573620529521,
|
||||
17.35366396337441
|
||||
]
|
||||
},
|
||||
"headline": "On unseen app repos: recall 0.0% (Wilson 0.0-24.3), false alarms 12.6 per 100 pushes, precision 0.0%; exploit-verified 0 of 5 (5 not yet proved).",
|
||||
"precision": {
|
||||
"count": 0,
|
||||
"n": 33,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.10427343003561192
|
||||
]
|
||||
},
|
||||
"recall": {
|
||||
"cluster95": [
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
rules:
|
||||
- id: ousast.c.regex.c-exec-family
|
||||
languages:
|
||||
- c
|
||||
- cpp
|
||||
severity: WARNING
|
||||
message: C exec command execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: c-exec-family
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(execl|execlp|execle|execv|execvp|execvpe)\s*\(
|
||||
- id: ousast.c.regex.c-shell-exec
|
||||
languages:
|
||||
- c
|
||||
- cpp
|
||||
severity: WARNING
|
||||
message: C shell execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: c-shell-exec
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(system|popen)\s*\(
|
||||
- id: ousast.c.regex.c-unsafe-copy
|
||||
languages:
|
||||
- c
|
||||
- cpp
|
||||
severity: WARNING
|
||||
message: Unsafe C string copy needs review
|
||||
metadata:
|
||||
family: memory
|
||||
cwe: CWE-121
|
||||
origin: c-unsafe-copy
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- memory_unsafe
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(gets|strcpy|strcat|sprintf)\s*\(
|
||||
- id: ousast.c.regex.c-unsafe-format
|
||||
languages:
|
||||
- c
|
||||
- cpp
|
||||
severity: WARNING
|
||||
message: Unsafe C format string needs review
|
||||
metadata:
|
||||
family: memory
|
||||
cwe: CWE-134
|
||||
origin: c-unsafe-format
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- memory_unsafe
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(printf|vprintf|vsprintf|vsnprintf|vfprintf|syslog)\s*\(\s*[A-Za-z_]
|
||||
- id: ousast.c.regex.c-unsafe-memory-copy
|
||||
languages:
|
||||
- c
|
||||
- cpp
|
||||
severity: WARNING
|
||||
message: Unsafe C memory copy needs review
|
||||
metadata:
|
||||
family: memory
|
||||
cwe: CWE-121
|
||||
origin: c-unsafe-memory-copy
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- memory_unsafe
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(memcpy|memmove|bcopy)\s*\(
|
||||
- id: ousast.c.regex.c-unsafe-scanf
|
||||
languages:
|
||||
- c
|
||||
- cpp
|
||||
severity: WARNING
|
||||
message: Unbounded C scanf needs review
|
||||
metadata:
|
||||
family: memory
|
||||
cwe: CWE-121
|
||||
origin: c-unsafe-scanf
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- memory_unsafe
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(scanf|fscanf|sscanf)\s*\([^\n]*%s
|
||||
- id: ousast.c.taint.printf
|
||||
mode: taint
|
||||
languages: &id001
|
||||
- c
|
||||
- cpp
|
||||
severity: ERROR
|
||||
message: 'printf: untrusted input reaches fprintf, printf, sprintf, syslog, vprintf, vsprintf'
|
||||
metadata:
|
||||
family: memory
|
||||
cwe: CWE-134
|
||||
origin: sink:c:printf
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- fprintf
|
||||
- printf
|
||||
- sprintf
|
||||
- syslog
|
||||
- vprintf
|
||||
- vsprintf
|
||||
pattern-sources:
|
||||
- pattern-either: &id002
|
||||
- pattern: argv
|
||||
- pattern: argv(...)
|
||||
- pattern: getenv
|
||||
- pattern: getenv(...)
|
||||
- pattern: fgets
|
||||
- pattern: fgets(...)
|
||||
- pattern: gets
|
||||
- pattern: gets(...)
|
||||
- pattern: scanf
|
||||
- pattern: scanf(...)
|
||||
- pattern: fscanf
|
||||
- pattern: fscanf(...)
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- patterns:
|
||||
- pattern: fprintf($FORMAT, ...)
|
||||
- focus-metavariable: $FORMAT
|
||||
- patterns:
|
||||
- pattern: printf($FORMAT, ...)
|
||||
- focus-metavariable: $FORMAT
|
||||
- patterns:
|
||||
- pattern: sprintf($FORMAT, ...)
|
||||
- focus-metavariable: $FORMAT
|
||||
- patterns:
|
||||
- pattern: syslog($FORMAT, ...)
|
||||
- focus-metavariable: $FORMAT
|
||||
- patterns:
|
||||
- pattern: vprintf($FORMAT, ...)
|
||||
- focus-metavariable: $FORMAT
|
||||
- patterns:
|
||||
- pattern: vsprintf($FORMAT, ...)
|
||||
- focus-metavariable: $FORMAT
|
||||
- id: ousast.c.taint.strcpy
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'strcpy: untrusted input reaches gets, sprintf, strcat, strcpy'
|
||||
metadata:
|
||||
family: memory
|
||||
cwe: CWE-121
|
||||
origin: sink:c:strcpy
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- gets
|
||||
- sprintf
|
||||
- strcat
|
||||
- strcpy
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: gets(...)
|
||||
- pattern: $OBJ.gets(...)
|
||||
- pattern: sprintf(...)
|
||||
- pattern: $OBJ.sprintf(...)
|
||||
- pattern: strcat(...)
|
||||
- pattern: $OBJ.strcat(...)
|
||||
- pattern: strcpy(...)
|
||||
- pattern: $OBJ.strcpy(...)
|
||||
- id: ousast.c.taint.system
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'system: untrusted input reaches execl, execv, execvp, popen, system'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: sink:c:system
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- execl
|
||||
- execv
|
||||
- execvp
|
||||
- popen
|
||||
- system
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: execl(...)
|
||||
- pattern: $OBJ.execl(...)
|
||||
- pattern: execv(...)
|
||||
- pattern: $OBJ.execv(...)
|
||||
- pattern: execvp(...)
|
||||
- pattern: $OBJ.execvp(...)
|
||||
- pattern: popen(...)
|
||||
- pattern: $OBJ.popen(...)
|
||||
- pattern: system(...)
|
||||
- pattern: $OBJ.system(...)
|
||||
@@ -0,0 +1,233 @@
|
||||
{
|
||||
"facts": {
|
||||
"converted": [
|
||||
"sink:c:printf",
|
||||
"sink:c:strcpy",
|
||||
"sink:c:system",
|
||||
"sink:java:MessageDigest",
|
||||
"sink:java:exec",
|
||||
"sink:java:executeQuery",
|
||||
"sink:java:file.open",
|
||||
"sink:java:outbound.request",
|
||||
"sink:java:readObject",
|
||||
"sink:java:response.write",
|
||||
"sink:javascript:eval",
|
||||
"sink:javascript:exec",
|
||||
"sink:javascript:html.response",
|
||||
"sink:javascript:html.unescaped",
|
||||
"sink:javascript:outbound.request",
|
||||
"sink:javascript:path.open",
|
||||
"sink:javascript:prototype.merge",
|
||||
"sink:javascript:query",
|
||||
"sink:javascript:redirect",
|
||||
"sink:javascript:vm.eval",
|
||||
"sink:php:code_eval",
|
||||
"sink:php:command",
|
||||
"sink:php:echo",
|
||||
"sink:php:file_inclusion",
|
||||
"sink:php:filesystem",
|
||||
"sink:php:outbound_request",
|
||||
"sink:php:redirect",
|
||||
"sink:php:sql",
|
||||
"sink:php:unserialize",
|
||||
"sink:php:wpdb",
|
||||
"sink:python:eval",
|
||||
"sink:python:execute",
|
||||
"sink:python:hashlib",
|
||||
"sink:python:html.unescaped",
|
||||
"sink:python:orm.raw",
|
||||
"sink:python:os.system",
|
||||
"sink:python:outbound.request",
|
||||
"sink:python:path.open",
|
||||
"sink:python:pickle.loads",
|
||||
"sink:python:redirect"
|
||||
],
|
||||
"lossy": [
|
||||
{
|
||||
"loss": "prefix_fixes_origin: a value appended after a fixed origin still counts as tainted",
|
||||
"origin": "sink:javascript:outbound.request"
|
||||
},
|
||||
{
|
||||
"loss": "prefix_fixes_origin: a value appended after a fixed origin still counts as tainted",
|
||||
"origin": "sink:javascript:redirect"
|
||||
},
|
||||
{
|
||||
"loss": "quoted_only escape treated as a full sanitizer (silences the CVE-2024-1071 shape)",
|
||||
"origin": "sanitizer:php:sql_escape"
|
||||
},
|
||||
{
|
||||
"loss": "prefix_fixes_origin: a value appended after a fixed origin still counts as tainted",
|
||||
"origin": "sink:python:outbound.request"
|
||||
},
|
||||
{
|
||||
"loss": "prefix_fixes_origin: a value appended after a fixed origin still counts as tainted",
|
||||
"origin": "sink:python:redirect"
|
||||
}
|
||||
],
|
||||
"skipped": [
|
||||
{
|
||||
"origin": "sanitizer:c:literal_format",
|
||||
"reason": "literal_format_arg: a shape; a literal format is never tainted at the focused sink argument"
|
||||
},
|
||||
{
|
||||
"origin": "sanitizer:java:parameterized_statement",
|
||||
"reason": "parameterized: a call shape, not a cleansing; expressed as the single-argument sink form"
|
||||
},
|
||||
{
|
||||
"origin": "sanitizer:javascript:parameterized_query",
|
||||
"reason": "parameterized: a call shape, not a cleansing; expressed as the single-argument sink form"
|
||||
},
|
||||
{
|
||||
"origin": "layout:javascript:javascript_first_party_tests",
|
||||
"reason": "layout: scope policy (vendored, tests), not a rule"
|
||||
},
|
||||
{
|
||||
"origin": "sink:javascript:html.unescaped:v-html",
|
||||
"reason": "template directive, not a call"
|
||||
},
|
||||
{
|
||||
"origin": "sanitizer:php:allowlist",
|
||||
"reason": "guard: a check whose safety depends on dominance; Semgrep has no equivalent"
|
||||
},
|
||||
{
|
||||
"origin": "sanitizer:php:prepared_statement",
|
||||
"reason": "parameterized: a call shape, not a cleansing; expressed as the single-argument sink form"
|
||||
},
|
||||
{
|
||||
"origin": "dispatch:php:wordpress_hooks",
|
||||
"reason": "dispatch: string-keyed hook registry routing; no Semgrep equivalent"
|
||||
},
|
||||
{
|
||||
"origin": "layout:php:php_layout",
|
||||
"reason": "layout: scope policy (vendored, tests), not a rule"
|
||||
},
|
||||
{
|
||||
"origin": "sanitizer:python:parameterized_execute",
|
||||
"reason": "parameterized: a call shape, not a cleansing; expressed as the single-argument sink form"
|
||||
},
|
||||
{
|
||||
"origin": "sanitizer:python:url_allowlist",
|
||||
"reason": "guard: a check whose safety depends on dominance; Semgrep has no equivalent"
|
||||
}
|
||||
]
|
||||
},
|
||||
"files": {
|
||||
"c.yaml": 9,
|
||||
"groovy.yaml": 1,
|
||||
"java.yaml": 11,
|
||||
"javascript.yaml": 21,
|
||||
"php.yaml": 19,
|
||||
"python.yaml": 20
|
||||
},
|
||||
"not_carried": [
|
||||
"comment skipping: the quick engine drops a regex match inside a comment; pattern-regex does not",
|
||||
"framework priors and fallback rules: the tagged rule is emitted, its language-level fallback is not",
|
||||
"min_evidence_level, ranking priority, the rung: post-match policy on our side"
|
||||
],
|
||||
"obligations": {
|
||||
"reason": "an obligation finds a missing guard (an absence), which no pattern expresses",
|
||||
"skipped_tables": 37
|
||||
},
|
||||
"regex": {
|
||||
"converted": [
|
||||
"c-exec-family",
|
||||
"c-shell-exec",
|
||||
"c-unsafe-copy",
|
||||
"c-unsafe-format",
|
||||
"c-unsafe-memory-copy",
|
||||
"c-unsafe-scanf",
|
||||
"groovy-xss-unescaped",
|
||||
"java-command-exec",
|
||||
"java-deserialization",
|
||||
"java-sql-injection",
|
||||
"java-weak-hash",
|
||||
"js-command-exec",
|
||||
"js-deserialize",
|
||||
"js-dom-xss",
|
||||
"js-eval",
|
||||
"js-function-constructor",
|
||||
"js-path-traversal",
|
||||
"js-spawn-shell",
|
||||
"js-sql-injection",
|
||||
"js-ssrf",
|
||||
"js-weak-hash",
|
||||
"js-xss-response",
|
||||
"php-wpdb-sql-composition",
|
||||
"php-sql-unquoted-concat",
|
||||
"php-sql-unquoted-interpolation",
|
||||
"php-sql-sprintf-unescaped",
|
||||
"php-command-injection",
|
||||
"php-file-inclusion-input",
|
||||
"php-path-input",
|
||||
"php-ssrf-input",
|
||||
"php-open-redirect",
|
||||
"python-flask-debug",
|
||||
"python-os-command",
|
||||
"python-path-traversal",
|
||||
"python-shell-true",
|
||||
"python-sql-injection",
|
||||
"python-ssrf",
|
||||
"python-ssti",
|
||||
"python-unsafe-deserialization",
|
||||
"python-unsafe-eval",
|
||||
"python-weak-hash"
|
||||
],
|
||||
"rewritten": [
|
||||
{
|
||||
"origin": "php-sql-sprintf-unescaped",
|
||||
"rewrite": "bounded lazy group repeat {0,N}? widened to *? (PCRE2 size limit)"
|
||||
}
|
||||
],
|
||||
"skipped": [
|
||||
{
|
||||
"origin": "js-ssrf-language",
|
||||
"reason": "fallback for js-ssrf: the tagged rule is active"
|
||||
},
|
||||
{
|
||||
"origin": "php-sql-call-composition",
|
||||
"reason": "shadow status (precision_estimate 0.25); not admitted by the hook"
|
||||
},
|
||||
{
|
||||
"origin": "php-sql-unquoted-concat-language",
|
||||
"reason": "fallback for php-sql-unquoted-concat: the tagged rule is active"
|
||||
},
|
||||
{
|
||||
"origin": "php-sql-unquoted-interpolation-language",
|
||||
"reason": "fallback for php-sql-unquoted-interpolation: the tagged rule is active"
|
||||
},
|
||||
{
|
||||
"origin": "php-sql-sprintf-unescaped-language",
|
||||
"reason": "fallback for php-sql-sprintf-unescaped: the tagged rule is active"
|
||||
},
|
||||
{
|
||||
"origin": "php-code-eval",
|
||||
"reason": "shadow status (precision_estimate 0.333); not admitted by the hook"
|
||||
},
|
||||
{
|
||||
"origin": "php-unserialize",
|
||||
"reason": "shadow status (precision_estimate 0.25); not admitted by the hook"
|
||||
},
|
||||
{
|
||||
"origin": "php-maybe-unserialize",
|
||||
"reason": "shadow status (precision_estimate 0.25); not admitted by the hook"
|
||||
},
|
||||
{
|
||||
"origin": "php-xss-echo-input",
|
||||
"reason": "shadow status (precision_estimate 0.333); not admitted by the hook"
|
||||
},
|
||||
{
|
||||
"origin": "php-xss-echo-input-language",
|
||||
"reason": "fallback for php-xss-echo-input: the tagged rule is active"
|
||||
},
|
||||
{
|
||||
"origin": "python-unsafe-deserialization-language",
|
||||
"reason": "fallback for python-unsafe-deserialization: the tagged rule is active"
|
||||
}
|
||||
]
|
||||
},
|
||||
"validation": {
|
||||
"exit": 0,
|
||||
"stderr": "Configuration is valid - found 0 configuration error(s), and 81 rule(s).\n",
|
||||
"stdout": ""
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
rules:
|
||||
- id: ousast.groovy.regex.groovy-xss-unescaped
|
||||
languages:
|
||||
- generic
|
||||
severity: WARNING
|
||||
message: Groovy template unescaped output needs review
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: groovy-xss-unescaped
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (yieldUnescaped)\s+[A-Za-z_]
|
||||
paths:
|
||||
include:
|
||||
- '*.groovy'
|
||||
- '*.tpl'
|
||||
@@ -0,0 +1,362 @@
|
||||
rules:
|
||||
- id: ousast.java.regex.java-command-exec
|
||||
languages:
|
||||
- java
|
||||
severity: WARNING
|
||||
message: Java runtime command execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: java-command-exec
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(exec)\s*\(
|
||||
- id: ousast.java.regex.java-deserialization
|
||||
languages:
|
||||
- java
|
||||
severity: WARNING
|
||||
message: Java unsafe deserialization needs review
|
||||
metadata:
|
||||
family: deserialization
|
||||
cwe: CWE-502
|
||||
origin: java-deserialization
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- deserialization
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \.(readObject)\s*\(
|
||||
- id: ousast.java.regex.java-sql-injection
|
||||
languages:
|
||||
- java
|
||||
severity: WARNING
|
||||
message: Java SQL string composition needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: java-sql-injection
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?i)\"\s*(select|insert|update|delete)\b[^\"]*\"\s*\+
|
||||
- id: ousast.java.regex.java-weak-hash
|
||||
languages:
|
||||
- java
|
||||
severity: WARNING
|
||||
message: Java weak hash needs review
|
||||
metadata:
|
||||
family: config_secrets
|
||||
cwe: CWE-327
|
||||
origin: java-weak-hash
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- crypto
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: MessageDigest\.getInstance\s*\(\s*\"(MD5|SHA-1)\"
|
||||
- id: ousast.java.weak.MessageDigest
|
||||
languages: &id001
|
||||
- java
|
||||
severity: WARNING
|
||||
message: 'MessageDigest: a weak literal reaches getInstance'
|
||||
metadata:
|
||||
family: config_secrets
|
||||
cwe: CWE-327
|
||||
origin: sink:java:MessageDigest
|
||||
precision_estimate: unmeasured
|
||||
kind: weak_literal
|
||||
weak_literals:
|
||||
- MD5
|
||||
- SHA-1
|
||||
- SHA1
|
||||
pattern-either:
|
||||
- patterns:
|
||||
- pattern: getInstance($ALGORITHM, ...)
|
||||
- metavariable-regex:
|
||||
metavariable: $ALGORITHM
|
||||
regex: ^["']?(MD5|SHA\-1|SHA1)["']?$
|
||||
- patterns:
|
||||
- pattern: $OBJ.getInstance($ALGORITHM, ...)
|
||||
- metavariable-regex:
|
||||
metavariable: $ALGORITHM
|
||||
regex: ^["']?(MD5|SHA\-1|SHA1)["']?$
|
||||
- id: ousast.java.taint.exec
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'exec: untrusted input reaches exec'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: sink:java:exec
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- exec
|
||||
pattern-sources:
|
||||
- pattern-either: &id002
|
||||
- pattern: getParameter
|
||||
- pattern: $OBJ.getParameter
|
||||
- pattern: getParameter(...)
|
||||
- pattern: $OBJ.getParameter(...)
|
||||
- pattern: getHeader
|
||||
- pattern: $OBJ.getHeader
|
||||
- pattern: getHeader(...)
|
||||
- pattern: $OBJ.getHeader(...)
|
||||
- pattern: getCookies
|
||||
- pattern: $OBJ.getCookies
|
||||
- pattern: getCookies(...)
|
||||
- pattern: $OBJ.getCookies(...)
|
||||
- pattern: getQueryString
|
||||
- pattern: $OBJ.getQueryString
|
||||
- pattern: getQueryString(...)
|
||||
- pattern: $OBJ.getQueryString(...)
|
||||
- pattern: readLine
|
||||
- pattern: $OBJ.readLine
|
||||
- pattern: readLine(...)
|
||||
- pattern: $OBJ.readLine(...)
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: exec(...)
|
||||
- pattern: $OBJ.exec(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- &id003
|
||||
pattern: getCanonicalPath(...)
|
||||
- &id004
|
||||
pattern: $OBJ.getCanonicalPath(...)
|
||||
- &id005
|
||||
pattern: normalize(...)
|
||||
- &id006
|
||||
pattern: $OBJ.normalize(...)
|
||||
- &id007
|
||||
pattern: URLEncoder.encode(...)
|
||||
- &id008
|
||||
pattern: ESAPI.encoder(...)
|
||||
- &id009
|
||||
pattern: encodeForHTML(...)
|
||||
- &id010
|
||||
pattern: $OBJ.encodeForHTML(...)
|
||||
- &id011
|
||||
pattern: encodeForURL(...)
|
||||
- &id012
|
||||
pattern: $OBJ.encodeForURL(...)
|
||||
- &id013
|
||||
pattern: StringEscapeUtils.escapeHtml4(...)
|
||||
- id: ousast.java.taint.executeQuery
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'executeQuery: untrusted input reaches execute, executeQuery, prepareCall'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: sink:java:executeQuery
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- execute
|
||||
- executeQuery
|
||||
- prepareCall
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: execute(...)
|
||||
- pattern: $OBJ.execute(...)
|
||||
- pattern: executeQuery(...)
|
||||
- pattern: $OBJ.executeQuery(...)
|
||||
- pattern: prepareCall(...)
|
||||
- pattern: $OBJ.prepareCall(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- id: ousast.java.taint.file-open
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'file.open: untrusted input reaches File, FileInputStream, FileOutputStream, FileReader, FileWriter, Files.newInputStream, Files.readAllBytes, Paths.get, RandomAccessFile'
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: sink:java:file.open
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- File
|
||||
- FileInputStream
|
||||
- FileOutputStream
|
||||
- FileReader
|
||||
- FileWriter
|
||||
- Files.newInputStream
|
||||
- Files.readAllBytes
|
||||
- Paths.get
|
||||
- RandomAccessFile
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: new File(...)
|
||||
- pattern: new FileInputStream(...)
|
||||
- pattern: new FileOutputStream(...)
|
||||
- pattern: new FileReader(...)
|
||||
- pattern: new FileWriter(...)
|
||||
- pattern: Files.newInputStream(...)
|
||||
- pattern: Files.readAllBytes(...)
|
||||
- pattern: Paths.get(...)
|
||||
- pattern: new RandomAccessFile(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- id: ousast.java.taint.outbound-request
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'outbound.request: untrusted input reaches exchange, getForObject, openConnection, openStream, sendRedirect'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: sink:java:outbound.request
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- exchange
|
||||
- getForObject
|
||||
- openConnection
|
||||
- openStream
|
||||
- sendRedirect
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: exchange(...)
|
||||
- pattern: $OBJ.exchange(...)
|
||||
- pattern: getForObject(...)
|
||||
- pattern: $OBJ.getForObject(...)
|
||||
- pattern: openConnection(...)
|
||||
- pattern: $OBJ.openConnection(...)
|
||||
- pattern: openStream(...)
|
||||
- pattern: $OBJ.openStream(...)
|
||||
- pattern: sendRedirect(...)
|
||||
- pattern: $OBJ.sendRedirect(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- id: ousast.java.taint.readObject
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'readObject: untrusted input reaches readObject'
|
||||
metadata:
|
||||
family: deserialization
|
||||
cwe: CWE-502
|
||||
origin: sink:java:readObject
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- readObject
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: readObject(...)
|
||||
- pattern: $OBJ.readObject(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- id: ousast.java.taint.response-write
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'response.write: untrusted input reaches addHeader, getWriter, print, println, setHeader, write'
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: sink:java:response.write
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- addHeader
|
||||
- getWriter
|
||||
- print
|
||||
- println
|
||||
- setHeader
|
||||
- write
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: addHeader(...)
|
||||
- pattern: $OBJ.addHeader(...)
|
||||
- pattern: getWriter(...)
|
||||
- pattern: $OBJ.getWriter(...)
|
||||
- pattern: print(...)
|
||||
- pattern: $OBJ.print(...)
|
||||
- pattern: println(...)
|
||||
- pattern: $OBJ.println(...)
|
||||
- pattern: setHeader(...)
|
||||
- pattern: $OBJ.setHeader(...)
|
||||
- pattern: write(...)
|
||||
- pattern: $OBJ.write(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
@@ -0,0 +1,679 @@
|
||||
rules:
|
||||
- id: ousast.javascript.regex.js-command-exec
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: Node child_process command execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: js-command-exec
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(exec|execSync)\s*\(
|
||||
- id: ousast.javascript.regex.js-deserialize
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: JavaScript unsafe deserialization needs review
|
||||
metadata:
|
||||
family: deserialization
|
||||
cwe: CWE-502
|
||||
origin: js-deserialize
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- deserialization
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(unserialize)\s*\(
|
||||
- id: ousast.javascript.regex.js-dom-xss
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: DOM sink with untrusted input needs review
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: js-dom-xss
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \.(innerHTML|outerHTML)\s*=\s*[^\n]*\b(?:req|request|location|document\.location)
|
||||
- id: ousast.javascript.regex.js-eval
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: Dynamic JavaScript evaluation needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: js-eval
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(eval)\s*\(
|
||||
- id: ousast.javascript.regex.js-function-constructor
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: JavaScript Function constructor needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: js-function-constructor
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: new\s+(Function)\s*\(
|
||||
- id: ousast.javascript.regex.js-path-traversal
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: Node filesystem access with external input needs review
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: js-path-traversal
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- filesystem_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \bfs\.(readFile|readFileSync|writeFile|writeFileSync|createReadStream|unlink|unlinkSync)\s*\(\s*[^\n]*\b(?:req|request)\.
|
||||
- id: ousast.javascript.regex.js-spawn-shell
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: Node spawn with shell needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: js-spawn-shell
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(spawn|spawnSync)\s*\([^\n]*shell\s*:\s*true
|
||||
- id: ousast.javascript.regex.js-sql-injection
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: JavaScript SQL string composition needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: js-sql-injection
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \.(query|execute)\s*\(\s*[^\n]*(?:\$\{|\+\s*[A-Za-z_$])
|
||||
- id: ousast.javascript.regex.js-ssrf
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: JavaScript outbound request with external input needs review
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: js-ssrf
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- network_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(axios|fetch|got)\s*[.(][^\n]*\b(?:req|request)\.
|
||||
- id: ousast.javascript.regex.js-weak-hash
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: JavaScript weak hash needs review
|
||||
metadata:
|
||||
family: config_secrets
|
||||
cwe: CWE-327
|
||||
origin: js-weak-hash
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- crypto
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: createHash\s*\(\s*[\"'](md5|sha1)[\"']
|
||||
- id: ousast.javascript.regex.js-xss-response
|
||||
languages:
|
||||
- javascript
|
||||
- typescript
|
||||
severity: WARNING
|
||||
message: Unsanitized HTTP response reflection needs review
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: js-xss-response
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \.(send|write|end)\s*\(\s*[^\n]*\b(?:req|request)\.(?:query|params|body)
|
||||
- id: ousast.javascript.taint.eval
|
||||
mode: taint
|
||||
languages: &id001
|
||||
- javascript
|
||||
- typescript
|
||||
severity: ERROR
|
||||
message: 'eval: untrusted input reaches eval'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: sink:javascript:eval
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- eval
|
||||
pattern-sources:
|
||||
- pattern-either: &id002
|
||||
- pattern: req.query
|
||||
- pattern: req.query(...)
|
||||
- pattern: req.body
|
||||
- pattern: req.body(...)
|
||||
- pattern: req.params
|
||||
- pattern: req.params(...)
|
||||
- pattern: req.cookies
|
||||
- pattern: req.cookies(...)
|
||||
- pattern: req.headers
|
||||
- pattern: req.headers(...)
|
||||
- pattern: request.query
|
||||
- pattern: request.query(...)
|
||||
- pattern: req.url
|
||||
- pattern: req.url(...)
|
||||
- pattern: request.url
|
||||
- pattern: request.url(...)
|
||||
- pattern: req.rawHeaders
|
||||
- pattern: req.rawHeaders(...)
|
||||
- pattern: url.parse
|
||||
- pattern: url.parse(...)
|
||||
- pattern: parsedUrl
|
||||
- pattern: $OBJ.parsedUrl
|
||||
- pattern: parsedUrl(...)
|
||||
- pattern: $OBJ.parsedUrl(...)
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: eval(...)
|
||||
- pattern: $OBJ.eval(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- &id003
|
||||
pattern: path.normalize(...)
|
||||
- &id004
|
||||
pattern: path.resolve(...)
|
||||
- &id005
|
||||
pattern: encodeURIComponent(...)
|
||||
- &id006
|
||||
pattern: $OBJ.encodeURIComponent(...)
|
||||
- &id007
|
||||
pattern: encodeURI(...)
|
||||
- &id008
|
||||
pattern: $OBJ.encodeURI(...)
|
||||
- &id009
|
||||
pattern: escape(...)
|
||||
- &id010
|
||||
pattern: $OBJ.escape(...)
|
||||
- &id011
|
||||
pattern: escapeHtml(...)
|
||||
- &id012
|
||||
pattern: $OBJ.escapeHtml(...)
|
||||
- &id013
|
||||
pattern: sanitizeHtml(...)
|
||||
- &id014
|
||||
pattern: $OBJ.sanitizeHtml(...)
|
||||
- &id015
|
||||
pattern: DOMPurify.sanitize(...)
|
||||
- id: ousast.javascript.taint.exec
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'exec: untrusted input reaches exec, execSync'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: sink:javascript:exec
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- exec
|
||||
- execSync
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: exec(...)
|
||||
- pattern: $OBJ.exec(...)
|
||||
- pattern: execSync(...)
|
||||
- pattern: $OBJ.execSync(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.html-response
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'html.response: untrusted input reaches res.end, res.send, res.write, response.end, response.send, response.write'
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: sink:javascript:html.response
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- res.end
|
||||
- res.send
|
||||
- res.write
|
||||
- response.end
|
||||
- response.send
|
||||
- response.write
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: res.end(...)
|
||||
- pattern: res.send(...)
|
||||
- pattern: res.write(...)
|
||||
- pattern: response.end(...)
|
||||
- pattern: response.send(...)
|
||||
- pattern: response.write(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.html-unescaped
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'html.unescaped: untrusted input reaches dangerouslySetInnerHTML, document.write, innerHTML, insertAdjacentHTML, outerHTML, v-html'
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: sink:javascript:html.unescaped
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- dangerouslySetInnerHTML
|
||||
- document.write
|
||||
- innerHTML
|
||||
- insertAdjacentHTML
|
||||
- outerHTML
|
||||
- v-html
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- patterns:
|
||||
- pattern: '{__html: $VALUE}'
|
||||
- focus-metavariable: $VALUE
|
||||
- pattern: document.write(...)
|
||||
- patterns:
|
||||
- pattern: $OBJ.innerHTML = $VALUE
|
||||
- focus-metavariable: $VALUE
|
||||
- pattern: insertAdjacentHTML(...)
|
||||
- pattern: $OBJ.insertAdjacentHTML(...)
|
||||
- patterns:
|
||||
- pattern: $OBJ.outerHTML = $VALUE
|
||||
- focus-metavariable: $VALUE
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.outbound-request
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'outbound.request: untrusted input reaches axios, fetch, got, http.get, http.request, https.get, https.request, needle, request, superagent, undici, unirest'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: sink:javascript:outbound.request
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- axios
|
||||
- fetch
|
||||
- got
|
||||
- http.get
|
||||
- http.request
|
||||
- https.get
|
||||
- https.request
|
||||
- needle
|
||||
- request
|
||||
- superagent
|
||||
- undici
|
||||
- unirest
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: axios(...)
|
||||
- pattern: $OBJ.axios(...)
|
||||
- pattern: fetch(...)
|
||||
- pattern: $OBJ.fetch(...)
|
||||
- pattern: got(...)
|
||||
- pattern: $OBJ.got(...)
|
||||
- pattern: http.get(...)
|
||||
- pattern: http.request(...)
|
||||
- pattern: https.get(...)
|
||||
- pattern: https.request(...)
|
||||
- pattern: needle(...)
|
||||
- pattern: $OBJ.needle(...)
|
||||
- pattern: request(...)
|
||||
- pattern: $OBJ.request(...)
|
||||
- pattern: superagent(...)
|
||||
- pattern: $OBJ.superagent(...)
|
||||
- pattern: undici(...)
|
||||
- pattern: $OBJ.undici(...)
|
||||
- pattern: unirest(...)
|
||||
- pattern: $OBJ.unirest(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.path-open
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'path.open: untrusted input reaches createReadStream, readFile, readFileSync, sendFile, unlink, unlinkSync'
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: sink:javascript:path.open
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- createReadStream
|
||||
- readFile
|
||||
- readFileSync
|
||||
- sendFile
|
||||
- unlink
|
||||
- unlinkSync
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: createReadStream(...)
|
||||
- pattern: $OBJ.createReadStream(...)
|
||||
- pattern: readFile(...)
|
||||
- pattern: $OBJ.readFile(...)
|
||||
- pattern: readFileSync(...)
|
||||
- pattern: $OBJ.readFileSync(...)
|
||||
- pattern: sendFile(...)
|
||||
- pattern: $OBJ.sendFile(...)
|
||||
- pattern: unlink(...)
|
||||
- pattern: $OBJ.unlink(...)
|
||||
- pattern: unlinkSync(...)
|
||||
- pattern: $OBJ.unlinkSync(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.prototype-merge
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'prototype.merge: untrusted input reaches Object.assign, defaultsDeep, extend, merge, mergeDeep, set, setWith'
|
||||
metadata:
|
||||
family: prototype
|
||||
cwe: CWE-1321
|
||||
origin: sink:javascript:prototype.merge
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- Object.assign
|
||||
- defaultsDeep
|
||||
- extend
|
||||
- merge
|
||||
- mergeDeep
|
||||
- set
|
||||
- setWith
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: Object.assign(...)
|
||||
- pattern: defaultsDeep(...)
|
||||
- pattern: $OBJ.defaultsDeep(...)
|
||||
- pattern: extend(...)
|
||||
- pattern: $OBJ.extend(...)
|
||||
- pattern: merge(...)
|
||||
- pattern: $OBJ.merge(...)
|
||||
- pattern: mergeDeep(...)
|
||||
- pattern: $OBJ.mergeDeep(...)
|
||||
- pattern: set(...)
|
||||
- pattern: $OBJ.set(...)
|
||||
- pattern: setWith(...)
|
||||
- pattern: $OBJ.setWith(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.query
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'query: untrusted input reaches execute, query'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: sink:javascript:query
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- execute
|
||||
- query
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- patterns:
|
||||
- pattern: execute($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
- patterns:
|
||||
- pattern: $OBJ.execute($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
- patterns:
|
||||
- pattern: query($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
- patterns:
|
||||
- pattern: $OBJ.query($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.redirect
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'redirect: untrusted input reaches location.assign, location.replace, redirect'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-601
|
||||
origin: sink:javascript:redirect
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- location.assign
|
||||
- location.replace
|
||||
- redirect
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: location.assign(...)
|
||||
- pattern: location.replace(...)
|
||||
- pattern: redirect(...)
|
||||
- pattern: $OBJ.redirect(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- id: ousast.javascript.taint.vm-eval
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'vm.eval: untrusted input reaches compileFunction, runInContext, runInNewContext, runInThisContext'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: sink:javascript:vm.eval
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- compileFunction
|
||||
- runInContext
|
||||
- runInNewContext
|
||||
- runInThisContext
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: compileFunction(...)
|
||||
- pattern: $OBJ.compileFunction(...)
|
||||
- pattern: runInContext(...)
|
||||
- pattern: $OBJ.runInContext(...)
|
||||
- pattern: runInNewContext(...)
|
||||
- pattern: $OBJ.runInNewContext(...)
|
||||
- pattern: runInThisContext(...)
|
||||
- pattern: $OBJ.runInThisContext(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
@@ -0,0 +1,647 @@
|
||||
rules:
|
||||
- id: ousast.php.regex.php-wpdb-sql-composition
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: WordPress $wpdb query composed from a variable without $wpdb->prepare
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: php-wpdb-sql-composition
|
||||
precision_estimate: '0.032'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \$wpdb->(get_var|get_results|get_row|get_col|query|replace)\s*\(\s*(?!\$wpdb->prepare)(?:"[^"\n]*\{?\$(?!wpdb->)[A-Za-z_]|[^\n;]*?["']\s*\.\s*\$(?!wpdb->)[A-Za-z_]|[^\n;]*?\$_(?:GET|POST|REQUEST|COOKIE)\b)
|
||||
- id: ousast.php.regex.php-sql-unquoted-concat
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP SQL literal concatenates a variable into an unquoted position
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: php-sql-unquoted-concat
|
||||
precision_estimate: '0.034'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?<![$\w>])(?=(VALUES|IN|ORDER|GROUP|LIMIT|OFFSET|WHERE|AND|OR|SET|HAVING)\b)(?:(?:VALUES|IN)\s*\(|(?:ORDER|GROUP)\s+BY\b[^"'\n]*|(?:LIMIT|OFFSET)\s+(?:\d+\s*,\s*)?|(?:WHERE|AND|OR|SET|HAVING)\s+[\w.`]+\s*(?:=|<>|!=|<=|>=|<|>|LIKE)\s*)["']\s*\.\s*\$(?!wpdb->)(?!\w*(?:format|placeholder))[A-Za-z_]
|
||||
- id: ousast.php.regex.php-sql-unquoted-interpolation
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP SQL string interpolates a variable into an unquoted position
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: php-sql-unquoted-interpolation
|
||||
precision_estimate: '0.129'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?<![$\w>])(?=(VALUES|IN|ORDER|GROUP|LIMIT|OFFSET|WHERE|AND|OR|SET|HAVING)\b)(?:(?:VALUES|IN)\s*\(\s*|(?:ORDER|GROUP)\s+BY\b[^"'\n;]*?|(?:LIMIT|OFFSET)\s+(?:\d+\s*,\s*)?|(?:WHERE|AND|OR|SET|HAVING)\s+[\w.`]+\s*(?:=|<>|!=|<=|>=|<|>|LIKE)\s*)\{?\$(?!wpdb->)(?!\w*(?:format|placeholder))[A-Za-z_]
|
||||
- id: ousast.php.regex.php-sql-sprintf-unescaped
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP sprintf-built SQL with an unescaped %s value
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: php-sql-sprintf-unescaped
|
||||
precision_estimate: '0.109'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (sprintf)\(\s*["']\s*(?:SELECT|INSERT|UPDATE|DELETE|REPLACE)\b(?=(?:(?!\);)[\s\S])*?'%s')(?=(?:(?![Ee]scape|quote\(|intval\(|absint\(|\(int\)|esc_sql\()[\s\S])*?\);)
|
||||
- id: ousast.php.regex.php-command-injection
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP shell command built from a variable without escapeshellarg
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: php-command-injection
|
||||
precision_estimate: '1.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?<![\w>:$\\])(?<!function )(system|exec|shell_exec|passthru|popen|proc_open|pcntl_exec)\s*\((?![^\n]*escapeshell(?:arg|cmd)\s*\()[^,\n;]*\$[A-Za-z_]
|
||||
- id: ousast.php.regex.php-file-inclusion-input
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP include/require of a path from request input
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-98
|
||||
origin: php-file-inclusion-input
|
||||
precision_estimate: '0.5'
|
||||
kind: regex
|
||||
tags:
|
||||
- filesystem_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?m)^(?![^\n]*\b(?:basename|realpath)\s*\()[^\n]*?(?<![\w'"$>:-])(include|include_once|require|require_once)\b(?!['"\]])\s*\(?[^\n;]*\$_(?:GET|POST|REQUEST|COOKIE)\b
|
||||
- id: ousast.php.regex.php-path-input
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP filesystem call on a path from request input
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: php-path-input
|
||||
precision_estimate: '1.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- filesystem_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?m)^(?![^\n]*\b(?:basename|realpath)\s*\()[^\n]*?(?<![\w>:$\\])(fopen|file_get_contents|file_put_contents|readfile|unlink|copy|rename|scandir|opendir|mkdir|rmdir|move_uploaded_file)\s*\([^\n;]*\$_(?:GET|POST|REQUEST|COOKIE)\b
|
||||
- id: ousast.php.regex.php-ssrf-input
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP outbound request to a URL from request input
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: php-ssrf-input
|
||||
precision_estimate: '1.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- network_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?<![\w>:$\\])(curl_init|curl_setopt|fsockopen|stream_socket_client|get_headers)\s*\([^\n;]*\$_(?:GET|POST|REQUEST|COOKIE)\b
|
||||
- id: ousast.php.regex.php-open-redirect
|
||||
languages:
|
||||
- php
|
||||
severity: WARNING
|
||||
message: PHP Location header from request input
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-601
|
||||
origin: php-open-redirect
|
||||
precision_estimate: '1.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (?<![\w>:$\\])(header)\s*\(\s*["']Location:[^\n;]*(?:\$_(?:GET|POST|REQUEST|COOKIE)\b|HTTP_REFERER)
|
||||
- id: ousast.php.taint.code_eval
|
||||
mode: taint
|
||||
languages: &id001
|
||||
- php
|
||||
severity: ERROR
|
||||
message: 'code_eval: untrusted input reaches assert, create_function, eval'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: sink:php:code_eval
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- assert
|
||||
- create_function
|
||||
- eval
|
||||
pattern-sources:
|
||||
- pattern-either: &id002
|
||||
- pattern: $_GET
|
||||
- pattern: $_GET[...]
|
||||
- pattern: $_POST
|
||||
- pattern: $_POST[...]
|
||||
- pattern: $_REQUEST
|
||||
- pattern: $_REQUEST[...]
|
||||
- pattern: $_COOKIE
|
||||
- pattern: $_COOKIE[...]
|
||||
- pattern: $_FILES
|
||||
- pattern: $_FILES[...]
|
||||
- pattern: $_SERVER
|
||||
- pattern: $_SERVER[...]
|
||||
- pattern: file_get_contents("php://input")
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: assert(...)
|
||||
- pattern: create_function(...)
|
||||
- pattern: eval(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- &id003
|
||||
pattern: htmlspecialchars(...)
|
||||
- &id004
|
||||
pattern: htmlentities(...)
|
||||
- &id005
|
||||
pattern: html_entity_decode_safe(...)
|
||||
- &id006
|
||||
pattern: mysqli_real_escape_string(...)
|
||||
- &id007
|
||||
pattern: mysql_real_escape_string(...)
|
||||
- &id008
|
||||
pattern: pg_escape_string(...)
|
||||
- &id009
|
||||
pattern: esc_sql(...)
|
||||
- &id010
|
||||
pattern: pg_escape_literal(...)
|
||||
- &id011
|
||||
pattern: $wpdb->prepare(...)
|
||||
- &id012
|
||||
pattern: escapeshellarg(...)
|
||||
- &id013
|
||||
pattern: escapeshellcmd(...)
|
||||
- &id014
|
||||
pattern: basename(...)
|
||||
- &id015
|
||||
pattern: realpath(...)
|
||||
- &id016
|
||||
pattern: intval(...)
|
||||
- &id017
|
||||
pattern: floatval(...)
|
||||
- &id018
|
||||
pattern: filter_var(...)
|
||||
- &id019
|
||||
pattern: settype(...)
|
||||
- &id020
|
||||
pattern: absint(...)
|
||||
- id: ousast.php.taint.command
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'command: untrusted input reaches exec, passthru, pcntl_exec, popen, proc_open, shell_exec, system'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: sink:php:command
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- exec
|
||||
- passthru
|
||||
- pcntl_exec
|
||||
- popen
|
||||
- proc_open
|
||||
- shell_exec
|
||||
- system
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: exec(...)
|
||||
- pattern: passthru(...)
|
||||
- pattern: pcntl_exec(...)
|
||||
- pattern: popen(...)
|
||||
- pattern: proc_open(...)
|
||||
- pattern: shell_exec(...)
|
||||
- pattern: system(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.echo
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'echo: untrusted input reaches echo, fwrite, print, print_r, printf, var_dump, vprintf'
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: sink:php:echo
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- echo
|
||||
- fwrite
|
||||
- print
|
||||
- print_r
|
||||
- printf
|
||||
- var_dump
|
||||
- vprintf
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: echo $VALUE;
|
||||
- pattern: fwrite(...)
|
||||
- pattern: print $VALUE;
|
||||
- pattern: print_r(...)
|
||||
- pattern: printf(...)
|
||||
- pattern: var_dump(...)
|
||||
- pattern: vprintf(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.file_inclusion
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'file_inclusion: untrusted input reaches include, include_once, require, require_once'
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-98
|
||||
origin: sink:php:file_inclusion
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- include
|
||||
- include_once
|
||||
- require
|
||||
- require_once
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: include ...
|
||||
- pattern: include_once ...
|
||||
- pattern: require ...
|
||||
- pattern: require_once ...
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.filesystem
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'filesystem: untrusted input reaches copy, file_get_contents, file_put_contents, fopen, mkdir, move_uploaded_file, opendir, readfile, rename, rmdir, scandir, unlink'
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: sink:php:filesystem
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- copy
|
||||
- file_get_contents
|
||||
- file_put_contents
|
||||
- fopen
|
||||
- mkdir
|
||||
- move_uploaded_file
|
||||
- opendir
|
||||
- readfile
|
||||
- rename
|
||||
- rmdir
|
||||
- scandir
|
||||
- unlink
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: copy(...)
|
||||
- pattern: file_get_contents(...)
|
||||
- pattern: file_put_contents(...)
|
||||
- pattern: fopen(...)
|
||||
- pattern: mkdir(...)
|
||||
- pattern: move_uploaded_file(...)
|
||||
- pattern: opendir(...)
|
||||
- pattern: readfile(...)
|
||||
- pattern: rename(...)
|
||||
- pattern: rmdir(...)
|
||||
- pattern: scandir(...)
|
||||
- pattern: unlink(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.outbound_request
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'outbound_request: untrusted input reaches curl_exec, curl_setopt, fsockopen, get_headers, stream_socket_client'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: sink:php:outbound_request
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- curl_exec
|
||||
- curl_setopt
|
||||
- fsockopen
|
||||
- get_headers
|
||||
- stream_socket_client
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: curl_exec(...)
|
||||
- pattern: curl_setopt(...)
|
||||
- pattern: fsockopen(...)
|
||||
- pattern: get_headers(...)
|
||||
- pattern: stream_socket_client(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.redirect
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'redirect: untrusted input reaches header'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-601
|
||||
origin: sink:php:redirect
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- header
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: header(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.sql
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'sql: untrusted input reaches mysql_query, mysqli_multi_query, mysqli_query, mysqli_real_query, pg_query, pg_send_query, sqlite_query, sqlite_unbuffered_query'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: sink:php:sql
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- mysql_query
|
||||
- mysqli_multi_query
|
||||
- mysqli_query
|
||||
- mysqli_real_query
|
||||
- pg_query
|
||||
- pg_send_query
|
||||
- sqlite_query
|
||||
- sqlite_unbuffered_query
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: mysql_query(...)
|
||||
- pattern: mysqli_multi_query(...)
|
||||
- pattern: mysqli_query(...)
|
||||
- pattern: mysqli_real_query(...)
|
||||
- pattern: pg_query(...)
|
||||
- pattern: pg_send_query(...)
|
||||
- pattern: sqlite_query(...)
|
||||
- pattern: sqlite_unbuffered_query(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.unserialize
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'unserialize: untrusted input reaches maybe_unserialize, unserialize'
|
||||
metadata:
|
||||
family: deserialization
|
||||
cwe: CWE-502
|
||||
origin: sink:php:unserialize
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- maybe_unserialize
|
||||
- unserialize
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: maybe_unserialize(...)
|
||||
- pattern: unserialize(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
- id: ousast.php.taint.wpdb
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'wpdb: untrusted input reaches $wpdb->get_col, $wpdb->get_results, $wpdb->get_row, $wpdb->get_var, $wpdb->query, $wpdb->replace'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: sink:php:wpdb
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- $wpdb->get_col
|
||||
- $wpdb->get_results
|
||||
- $wpdb->get_row
|
||||
- $wpdb->get_var
|
||||
- $wpdb->query
|
||||
- $wpdb->replace
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: $wpdb->get_col(...)
|
||||
- pattern: $wpdb->get_results(...)
|
||||
- pattern: $wpdb->get_row(...)
|
||||
- pattern: $wpdb->get_var(...)
|
||||
- pattern: $wpdb->query(...)
|
||||
- pattern: $wpdb->replace(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- *id007
|
||||
- *id008
|
||||
- *id009
|
||||
- *id010
|
||||
- *id011
|
||||
- *id012
|
||||
- *id013
|
||||
- *id014
|
||||
- *id015
|
||||
- *id016
|
||||
- *id017
|
||||
- *id018
|
||||
- *id019
|
||||
- *id020
|
||||
@@ -0,0 +1,558 @@
|
||||
rules:
|
||||
- id: ousast.python.regex.python-flask-debug
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Flask debug mode enabled needs review
|
||||
metadata:
|
||||
family: config_secrets
|
||||
cwe: CWE-489
|
||||
origin: python-flask-debug
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- insecure_default
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \.run\s*\([^\n]*(debug)\s*=\s*True
|
||||
- id: ousast.python.regex.python-os-command
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Python OS command execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: python-os-command
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(os\.system|os\.popen|os\.execl|os\.execlp|os\.execv|os\.execvp)\s*\(
|
||||
- id: ousast.python.regex.python-path-traversal
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Python file open with external input needs review
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: python-path-traversal
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- filesystem_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(open)\s*\(\s*[^\n]*(?:request\.|req\.|argv|input\s*\(|params\[|GET\[|POST\[)
|
||||
- id: ousast.python.regex.python-shell-true
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Subprocess shell execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: python-shell-true
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: subprocess\.[a-zA-Z_]+\s*\([^\n]*shell\s*=\s*True
|
||||
- id: ousast.python.regex.python-sql-injection
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Python SQL string composition needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: python-sql-injection
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: (\.execute)\s*\(\s*[^\n]*(?:[\"']\s*%\s*[(A-Za-z_]|\.format\s*\(|[\"']\s*\+|\bf[\"'])
|
||||
- id: ousast.python.regex.python-ssrf
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Python outbound request with external input needs review
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: python-ssrf
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- network_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(requests\.(?:get|post|put|delete|head|patch)|urllib\.request\.urlopen|urlopen)\s*\(\s*[^\n]*(?:request\.|req\.|argv|input\s*\()
|
||||
- id: ousast.python.regex.python-ssti
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Python server-side template injection needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-94
|
||||
origin: python-ssti
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- injection
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(render_template_string)\s*\(
|
||||
- id: ousast.python.regex.python-unsafe-deserialization
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Unsafe Python deserialization needs review
|
||||
metadata:
|
||||
family: deserialization
|
||||
cwe: CWE-502
|
||||
origin: python-unsafe-deserialization
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- deserialization
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(pickle\.loads|pickle\.load|cPickle\.loads|yaml\.load|marshal\.loads)\s*\(
|
||||
- id: ousast.python.regex.python-unsafe-eval
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Dynamic Python execution needs review
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: python-unsafe-eval
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- syscall_entry
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \b(eval|exec)\s*\(
|
||||
- id: ousast.python.regex.python-weak-hash
|
||||
languages:
|
||||
- python
|
||||
severity: WARNING
|
||||
message: Python weak hash needs review
|
||||
metadata:
|
||||
family: config_secrets
|
||||
cwe: CWE-327
|
||||
origin: python-weak-hash
|
||||
precision_estimate: '0.0'
|
||||
kind: regex
|
||||
tags:
|
||||
- crypto
|
||||
status: enabled
|
||||
version: '1'
|
||||
pattern-regex: \bhashlib\.(md5|sha1)\s*\(
|
||||
- id: ousast.python.taint.eval
|
||||
mode: taint
|
||||
languages: &id001
|
||||
- python
|
||||
severity: ERROR
|
||||
message: 'eval: untrusted input reaches eval, exec'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-95
|
||||
origin: sink:python:eval
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- eval
|
||||
- exec
|
||||
pattern-sources:
|
||||
- pattern-either: &id002
|
||||
- pattern: request.args
|
||||
- pattern: request.args(...)
|
||||
- pattern: request.form
|
||||
- pattern: request.form(...)
|
||||
- pattern: request.data
|
||||
- pattern: request.data(...)
|
||||
- pattern: request.json
|
||||
- pattern: request.json(...)
|
||||
- pattern: request.cookies
|
||||
- pattern: request.cookies(...)
|
||||
- pattern: request.values
|
||||
- pattern: request.values(...)
|
||||
- pattern: request.get_json
|
||||
- pattern: request.get_json(...)
|
||||
- pattern: request.headers
|
||||
- pattern: request.headers(...)
|
||||
- pattern: request.files
|
||||
- pattern: request.files(...)
|
||||
- pattern: request.get
|
||||
- pattern: request.get(...)
|
||||
- pattern: request.GET
|
||||
- pattern: request.GET(...)
|
||||
- pattern: request.POST
|
||||
- pattern: request.POST(...)
|
||||
- pattern: request.body
|
||||
- pattern: request.body(...)
|
||||
- pattern: request.COOKIES
|
||||
- pattern: request.COOKIES(...)
|
||||
- pattern: request.FILES
|
||||
- pattern: request.FILES(...)
|
||||
- pattern: request.META
|
||||
- pattern: request.META(...)
|
||||
- pattern: request.query_params
|
||||
- pattern: request.query_params(...)
|
||||
- pattern: cleaned_data
|
||||
- pattern: $OBJ.cleaned_data
|
||||
- pattern: cleaned_data(...)
|
||||
- pattern: $OBJ.cleaned_data(...)
|
||||
- pattern: sys.argv
|
||||
- pattern: sys.argv(...)
|
||||
- pattern: sys.stdin
|
||||
- pattern: sys.stdin(...)
|
||||
- pattern: stdin.read
|
||||
- pattern: stdin.read(...)
|
||||
- pattern: input(...)
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: eval(...)
|
||||
- pattern: $OBJ.eval(...)
|
||||
- pattern: exec(...)
|
||||
- pattern: $OBJ.exec(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- &id003
|
||||
pattern: ast.literal_eval(...)
|
||||
- &id004
|
||||
pattern: yaml.safe_load(...)
|
||||
- &id005
|
||||
pattern: safe_load(...)
|
||||
- &id006
|
||||
pattern: $OBJ.safe_load(...)
|
||||
- id: ousast.python.taint.execute
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'execute: untrusted input reaches execute, executemany'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: sink:python:execute
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- execute
|
||||
- executemany
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- patterns:
|
||||
- pattern: execute($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
- patterns:
|
||||
- pattern: $OBJ.execute($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
- patterns:
|
||||
- pattern: executemany($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
- patterns:
|
||||
- pattern: $OBJ.executemany($QUERY)
|
||||
- focus-metavariable: $QUERY
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.weak.hashlib
|
||||
languages: *id001
|
||||
severity: WARNING
|
||||
message: 'hashlib: a weak literal reaches hashlib.md5, hashlib.new, hashlib.sha1'
|
||||
metadata:
|
||||
family: config_secrets
|
||||
cwe: CWE-327
|
||||
origin: sink:python:hashlib
|
||||
precision_estimate: unmeasured
|
||||
kind: weak_literal
|
||||
weak_literals:
|
||||
- MD5
|
||||
- SHA1
|
||||
- md5
|
||||
- sha-1
|
||||
- sha1
|
||||
pattern-either:
|
||||
- pattern: hashlib.md5(...)
|
||||
- patterns:
|
||||
- pattern: hashlib.new($ALGORITHM, ...)
|
||||
- metavariable-regex:
|
||||
metavariable: $ALGORITHM
|
||||
regex: ^["']?(MD5|SHA1|md5|sha\-1|sha1)["']?$
|
||||
- pattern: hashlib.sha1(...)
|
||||
- id: ousast.python.taint.html-unescaped
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'html.unescaped: untrusted input reaches HttpResponse, Markup, format_html, mark_safe, render_template_string'
|
||||
metadata:
|
||||
family: output_encoding
|
||||
cwe: CWE-79
|
||||
origin: sink:python:html.unescaped
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- HttpResponse
|
||||
- Markup
|
||||
- format_html
|
||||
- mark_safe
|
||||
- render_template_string
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: HttpResponse(...)
|
||||
- pattern: $OBJ.HttpResponse(...)
|
||||
- pattern: Markup(...)
|
||||
- pattern: $OBJ.Markup(...)
|
||||
- pattern: format_html(...)
|
||||
- pattern: $OBJ.format_html(...)
|
||||
- pattern: mark_safe(...)
|
||||
- pattern: $OBJ.mark_safe(...)
|
||||
- pattern: render_template_string(...)
|
||||
- pattern: $OBJ.render_template_string(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.taint.orm-raw
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'orm.raw: untrusted input reaches RawSQL, extra, from_statement, objects.raw, raw, text'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-89
|
||||
origin: sink:python:orm.raw
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- RawSQL
|
||||
- extra
|
||||
- from_statement
|
||||
- objects.raw
|
||||
- raw
|
||||
- text
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: RawSQL(...)
|
||||
- pattern: $OBJ.RawSQL(...)
|
||||
- pattern: extra(...)
|
||||
- pattern: $OBJ.extra(...)
|
||||
- pattern: from_statement(...)
|
||||
- pattern: $OBJ.from_statement(...)
|
||||
- pattern: objects.raw(...)
|
||||
- pattern: raw(...)
|
||||
- pattern: $OBJ.raw(...)
|
||||
- pattern: text(...)
|
||||
- pattern: $OBJ.text(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.taint.os-system
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'os.system: untrusted input reaches os.popen, os.system, subprocess.Popen, subprocess.call, subprocess.check_output, subprocess.run'
|
||||
metadata:
|
||||
family: injection
|
||||
cwe: CWE-78
|
||||
origin: sink:python:os.system
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- os.popen
|
||||
- os.system
|
||||
- subprocess.Popen
|
||||
- subprocess.call
|
||||
- subprocess.check_output
|
||||
- subprocess.run
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: os.popen(...)
|
||||
- pattern: os.system(...)
|
||||
- pattern: subprocess.Popen(...)
|
||||
- pattern: subprocess.call(...)
|
||||
- pattern: subprocess.check_output(...)
|
||||
- pattern: subprocess.run(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.taint.outbound-request
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'outbound.request: untrusted input reaches httpx.get, httpx.post, requests.get, requests.post, requests.request, urlopen, urlretrieve'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-918
|
||||
origin: sink:python:outbound.request
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- httpx.get
|
||||
- httpx.post
|
||||
- requests.get
|
||||
- requests.post
|
||||
- requests.request
|
||||
- urlopen
|
||||
- urlretrieve
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: httpx.get(...)
|
||||
- pattern: httpx.post(...)
|
||||
- pattern: requests.get(...)
|
||||
- pattern: requests.post(...)
|
||||
- pattern: requests.request(...)
|
||||
- pattern: urlopen(...)
|
||||
- pattern: $OBJ.urlopen(...)
|
||||
- pattern: urlretrieve(...)
|
||||
- pattern: $OBJ.urlretrieve(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.taint.path-open
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'path.open: untrusted input reaches FileResponse, open, os.remove, os.unlink, send_file, send_from_directory, shutil.copy, shutil.move, shutil.rmtree'
|
||||
metadata:
|
||||
family: path
|
||||
cwe: CWE-22
|
||||
origin: sink:python:path.open
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- FileResponse
|
||||
- open
|
||||
- os.remove
|
||||
- os.unlink
|
||||
- send_file
|
||||
- send_from_directory
|
||||
- shutil.copy
|
||||
- shutil.move
|
||||
- shutil.rmtree
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: FileResponse(...)
|
||||
- pattern: $OBJ.FileResponse(...)
|
||||
- pattern: open(...)
|
||||
- pattern: $OBJ.open(...)
|
||||
- pattern: os.remove(...)
|
||||
- pattern: os.unlink(...)
|
||||
- pattern: send_file(...)
|
||||
- pattern: $OBJ.send_file(...)
|
||||
- pattern: send_from_directory(...)
|
||||
- pattern: $OBJ.send_from_directory(...)
|
||||
- pattern: shutil.copy(...)
|
||||
- pattern: shutil.move(...)
|
||||
- pattern: shutil.rmtree(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.taint.pickle-loads
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'pickle.loads: untrusted input reaches cPickle.loads, marshal.loads, pickle.load, pickle.loads, yaml.load'
|
||||
metadata:
|
||||
family: deserialization
|
||||
cwe: CWE-502
|
||||
origin: sink:python:pickle.loads
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- cPickle.loads
|
||||
- marshal.loads
|
||||
- pickle.load
|
||||
- pickle.loads
|
||||
- yaml.load
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: cPickle.loads(...)
|
||||
- pattern: marshal.loads(...)
|
||||
- pattern: pickle.load(...)
|
||||
- pattern: pickle.loads(...)
|
||||
- pattern: yaml.load(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- id: ousast.python.taint.redirect
|
||||
mode: taint
|
||||
languages: *id001
|
||||
severity: ERROR
|
||||
message: 'redirect: untrusted input reaches HttpResponseRedirect, redirect, url_for'
|
||||
metadata:
|
||||
family: untrusted_destination
|
||||
cwe: CWE-601
|
||||
origin: sink:python:redirect
|
||||
precision_estimate: unmeasured
|
||||
kind: taint
|
||||
calls:
|
||||
- HttpResponseRedirect
|
||||
- redirect
|
||||
- url_for
|
||||
pattern-sources:
|
||||
- pattern-either: *id002
|
||||
pattern-sinks:
|
||||
- pattern-either:
|
||||
- pattern: HttpResponseRedirect(...)
|
||||
- pattern: $OBJ.HttpResponseRedirect(...)
|
||||
- pattern: redirect(...)
|
||||
- pattern: $OBJ.redirect(...)
|
||||
- pattern: url_for(...)
|
||||
- pattern: $OBJ.url_for(...)
|
||||
pattern-sanitizers:
|
||||
- pattern-either:
|
||||
- *id003
|
||||
- *id004
|
||||
- *id005
|
||||
- *id006
|
||||
- pattern: reverse(...)
|
||||
- pattern: $OBJ.reverse(...)
|
||||
- pattern: reverse_lazy(...)
|
||||
- pattern: $OBJ.reverse_lazy(...)
|
||||
@@ -0,0 +1,500 @@
|
||||
{
|
||||
"alerts_only": {
|
||||
"count": 30,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
},
|
||||
"breakdowns": {
|
||||
"coverage": {
|
||||
"engine_covered": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
},
|
||||
"quick_only": {
|
||||
"alerts_only": {
|
||||
"count": 30,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
0.0375,
|
||||
0.23430962343096234
|
||||
],
|
||||
"count": 30,
|
||||
"icc": 0.27735846730536884,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
}
|
||||
},
|
||||
"unanalysable": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"family": {
|
||||
"access_control": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 3,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.5615060804490177
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
},
|
||||
"output_encoding": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 4,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.48990002040399916
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
},
|
||||
"path": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 3,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.5615060804490177
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
},
|
||||
"unknown": {
|
||||
"alerts_only": {
|
||||
"count": 30,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
0.0375,
|
||||
0.23430962343096234
|
||||
],
|
||||
"count": 30,
|
||||
"icc": 0.27735846730536884,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
}
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 2,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.6576280471103807
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"method": {
|
||||
"first_parent": {
|
||||
"alerts_only": {
|
||||
"count": 30,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
0.0375,
|
||||
0.23430962343096234
|
||||
],
|
||||
"count": 30,
|
||||
"icc": 0.27735846730536884,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
}
|
||||
},
|
||||
"fix_site": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"post_cutoff": {
|
||||
"unknown": {
|
||||
"alerts_only": {
|
||||
"count": 30,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
0.0375,
|
||||
0.23430962343096234
|
||||
],
|
||||
"count": 30,
|
||||
"icc": 0.27735846730536884,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"stream": {
|
||||
"advisory": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"alerts": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"quick": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"confirmed": {
|
||||
"alerts_only": {
|
||||
"count": 0,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"catch": {
|
||||
"count": 0,
|
||||
"n": 12,
|
||||
"rate": 0.0,
|
||||
"wilson95": [
|
||||
0,
|
||||
0.2425007574992425
|
||||
]
|
||||
},
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"count": 0,
|
||||
"icc": null,
|
||||
"n": 0,
|
||||
"rate": null,
|
||||
"wilson95": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
}
|
||||
},
|
||||
"coverage": {
|
||||
"engine_covered": 0,
|
||||
"instrument_failure": 1,
|
||||
"quick_only": 251,
|
||||
"unanalysable": 0
|
||||
},
|
||||
"deadlines": [
|
||||
{
|
||||
"applied_deadline_scale": null,
|
||||
"count": 252,
|
||||
"deadline_scale": null,
|
||||
"deadline_seconds": null,
|
||||
"lane": "unknown"
|
||||
}
|
||||
],
|
||||
"false_alarm": {
|
||||
"cluster95": [
|
||||
0.0375,
|
||||
0.23430962343096234
|
||||
],
|
||||
"count": 30,
|
||||
"icc": 0.27735846730536884,
|
||||
"n": 239,
|
||||
"rate": 0.12552301255230125,
|
||||
"wilson95": [
|
||||
0.08935736205295211,
|
||||
0.17353663963374408
|
||||
]
|
||||
},
|
||||
"total": 252,
|
||||
"unseen": {
|
||||
"gate": "not_run"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,634 @@
|
||||
"""Semgrep CE as a benchmark arm: our regex rules and taint facts converted, run per frozen change.
|
||||
|
||||
Two subcommands. ``rules`` converts every enabled quick rule to a ``pattern-regex`` rule and every
|
||||
source/sink/sanitizer fact triple to a ``mode: taint`` rule, one YAML file per language, and writes a
|
||||
conversion report naming what was skipped and why. ``run`` checks out each frozen change's head into a
|
||||
temporary worktree, runs Semgrep over the changed files only, proves the targets were read (bytes per
|
||||
file), and writes one scorer-shaped record per change, resumable exactly like ``hunter_arm``.
|
||||
|
||||
Nothing here ships: the rules are ours, the engine is a measurement instrument.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from openultrasast.hunter_tools import clamp_repo_path
|
||||
from openultrasast.model.taxonomy import load_families
|
||||
from openultrasast.ruleset import DEFAULT_RULESET_DIR, load_ruleset
|
||||
from openultrasast.semantic.facts import DEFAULT_FACTS_DIR, load_facts
|
||||
|
||||
if __package__ in {None, ""}:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from benchmarks.unseen.extract import SHA, InstrumentFailure
|
||||
from benchmarks.unseen.replay import canonical, load_slice
|
||||
from benchmarks.valid.catalog import local_git
|
||||
|
||||
# Our language ids to Semgrep's. Groovy has no Semgrep parser: `generic` plus a path filter.
|
||||
SEMGREP_LANGUAGES = {
|
||||
"c": "c",
|
||||
"cpp": "cpp",
|
||||
"python": "python",
|
||||
"javascript": "javascript",
|
||||
"typescript": "typescript",
|
||||
"java": "java",
|
||||
"php": "php",
|
||||
}
|
||||
GENERIC_PATHS = {"groovy": ["*.groovy", "*.tpl"]}
|
||||
# The rule file a rule lands in: one per language family.
|
||||
FILE_OF_LANGUAGE = {"cpp": "c", "typescript": "javascript"}
|
||||
# Fact languages whose Semgrep rules need a second dialect.
|
||||
FACT_DIALECTS = {"c": ["c", "cpp"], "javascript": ["javascript", "typescript"]}
|
||||
# PHP language constructs are not calls: `echo(...)` would parse as a call expression, and Semgrep's PHP
|
||||
# pattern parser takes `echo` only as a whole statement.
|
||||
PHP_CONSTRUCTS = {
|
||||
"echo": "echo $VALUE;",
|
||||
"print": "print $VALUE;",
|
||||
"include": "include ...",
|
||||
"include_once": "include_once ...",
|
||||
"require": "require ...",
|
||||
"require_once": "require_once ...",
|
||||
}
|
||||
# DOM properties assigned, not called.
|
||||
ASSIGNED_SINKS = {"innerHTML", "outerHTML"}
|
||||
SEMGREP_FLAGS = (
|
||||
"--json",
|
||||
"--metrics=off",
|
||||
"--disable-version-check",
|
||||
"--no-git-ignore",
|
||||
"--timeout",
|
||||
"60",
|
||||
"--max-memory",
|
||||
"1500",
|
||||
"-j",
|
||||
"1",
|
||||
)
|
||||
# A Semgrep launch that returns faster than its own startup never ran the engine.
|
||||
MIN_PLAUSIBLE_SECONDS = 1.0
|
||||
# PCRE2 compiles a bounded repeat of a group by copying it: `(?:...){0,2000}?` is "too large". The lazy
|
||||
# unbounded form keeps the match and widens the window from 2000 characters to the statement.
|
||||
BOUNDED_GROUP_REPEAT = re.compile(r"\{0,\d+\}\?")
|
||||
|
||||
|
||||
def _family_of(cwe):
|
||||
family = load_families().family_of_cwe(cwe or "")
|
||||
return family.id if family else "unknown"
|
||||
|
||||
|
||||
def _ident(value):
|
||||
return re.sub(r"[^A-Za-z0-9_-]+", "-", value).strip("-")
|
||||
|
||||
|
||||
# --- regex rules -------------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _raw_rules(ruleset_dir):
|
||||
rows = []
|
||||
for path in sorted(Path(ruleset_dir).rglob("*.toml")):
|
||||
if path.parent.name in {"semantic", "obligations"} or path.parent == Path(ruleset_dir):
|
||||
continue
|
||||
for item in tomllib.loads(path.read_text()).get("rule", []):
|
||||
rows.append(item)
|
||||
return rows
|
||||
|
||||
|
||||
def convert_regex_rules(ruleset_dir=DEFAULT_RULESET_DIR, *, min_precision=0.0):
|
||||
"""Every enabled quick rule as a Semgrep ``pattern-regex`` rule; the rest named with a reason."""
|
||||
active = {rule.rule_id: rule for rule in load_ruleset(Path(ruleset_dir))}
|
||||
converted, skipped, rewrites = {}, [], []
|
||||
for item in _raw_rules(ruleset_dir):
|
||||
rule_id = str(item["rule_id"])
|
||||
rule = active.get(rule_id)
|
||||
if rule is None:
|
||||
skipped.append({"origin": rule_id, "reason": f"fallback for {item.get('fallback_for')}: the tagged rule is active"})
|
||||
continue
|
||||
if rule.status == "disabled":
|
||||
skipped.append({"origin": rule_id, "reason": "disabled"})
|
||||
continue
|
||||
if rule.status == "shadow":
|
||||
skipped.append(
|
||||
{"origin": rule_id, "reason": f"shadow status (precision_estimate {rule.precision_estimate}); not admitted by the hook"}
|
||||
)
|
||||
continue
|
||||
if rule.precision_estimate < min_precision:
|
||||
skipped.append({"origin": rule_id, "reason": f"emit floor: precision_estimate {rule.precision_estimate} < {min_precision}"})
|
||||
continue
|
||||
if not rule.languages:
|
||||
skipped.append({"origin": rule_id, "reason": "non-code file targets: the rule applies to every file"})
|
||||
continue
|
||||
languages = [SEMGREP_LANGUAGES[language] for language in rule.languages if language in SEMGREP_LANGUAGES]
|
||||
generic = [language for language in rule.languages if language in GENERIC_PATHS]
|
||||
unsupported = [language for language in rule.languages if language not in SEMGREP_LANGUAGES and language not in GENERIC_PATHS]
|
||||
if unsupported:
|
||||
skipped.append({"origin": rule_id, "reason": f"no Semgrep language for {unsupported}"})
|
||||
continue
|
||||
family = _family_of(rule.cwe)
|
||||
pattern = BOUNDED_GROUP_REPEAT.sub("*?", rule.pattern)
|
||||
if pattern != rule.pattern:
|
||||
rewrites.append({"origin": rule_id, "rewrite": "bounded lazy group repeat {0,N}? widened to *? (PCRE2 size limit)"})
|
||||
entry = {
|
||||
"id": f"ousast.{FILE_OF_LANGUAGE.get(rule.languages[0], rule.languages[0])}.regex.{_ident(rule_id)}",
|
||||
"languages": languages or ["generic"],
|
||||
"severity": "WARNING",
|
||||
"message": rule.title,
|
||||
"metadata": {
|
||||
"family": family,
|
||||
"cwe": rule.cwe,
|
||||
"origin": rule_id,
|
||||
"precision_estimate": str(rule.precision_estimate),
|
||||
"kind": "regex",
|
||||
"tags": list(rule.tags),
|
||||
"status": rule.status,
|
||||
"version": rule.version,
|
||||
},
|
||||
"pattern-regex": pattern,
|
||||
}
|
||||
if not languages:
|
||||
entry["paths"] = {"include": [glob for language in generic for glob in GENERIC_PATHS[language]]}
|
||||
key = FILE_OF_LANGUAGE.get(rule.languages[0], rule.languages[0])
|
||||
converted.setdefault(key, []).append(entry)
|
||||
return converted, skipped, rewrites
|
||||
|
||||
|
||||
# --- taint facts -------------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _call_patterns(call, language):
|
||||
"""The Semgrep call shapes of one fact token (a function, a method, a construct)."""
|
||||
if language == "php":
|
||||
if call in PHP_CONSTRUCTS:
|
||||
return [PHP_CONSTRUCTS[call]]
|
||||
return [f"{call}(...)"]
|
||||
if "." in call or "->" in call:
|
||||
return [f"{call}(...)"]
|
||||
if language == "java" and call[:1].isupper():
|
||||
return [f"new {call}(...)"]
|
||||
return [f"{call}(...)", f"$OBJ.{call}(...)"]
|
||||
|
||||
|
||||
def _source_patterns(token, language):
|
||||
if language == "php":
|
||||
if token == "php://input":
|
||||
return ['file_get_contents("php://input")']
|
||||
return [token, f"{token}[...]"]
|
||||
if token.endswith("("):
|
||||
return [f"{token}...)"]
|
||||
if "." in token:
|
||||
return [token, f"{token}(...)"]
|
||||
if language == "c":
|
||||
return [token, f"{token}(...)"]
|
||||
return [token, f"$OBJ.{token}", f"{token}(...)", f"$OBJ.{token}(...)"]
|
||||
|
||||
|
||||
def _sink_entries(call, language, *, format_arg=None, single_argument=False):
|
||||
"""Sink patterns for one call; a focused argument where the fact names one."""
|
||||
if call in ASSIGNED_SINKS:
|
||||
return [{"patterns": [{"pattern": f"$OBJ.{call} = $VALUE"}, {"focus-metavariable": "$VALUE"}]}]
|
||||
if call == "dangerouslySetInnerHTML":
|
||||
return [{"patterns": [{"pattern": "{__html: $VALUE}"}, {"focus-metavariable": "$VALUE"}]}]
|
||||
if format_arg is not None:
|
||||
arguments = ", ".join([*(f"$ARG{index}" for index in range(format_arg)), "$FORMAT", "..."])
|
||||
heads = [call] if "." in call or language == "c" else [call, f"$OBJ.{call}"]
|
||||
return [{"patterns": [{"pattern": f"{head}({arguments})"}, {"focus-metavariable": "$FORMAT"}]} for head in heads]
|
||||
if single_argument:
|
||||
heads = [call] if "." in call or "->" in call else [call, f"$OBJ.{call}"]
|
||||
return [{"patterns": [{"pattern": f"{head}($QUERY)"}, {"focus-metavariable": "$QUERY"}]} for head in heads]
|
||||
return [{"pattern": pattern} for pattern in _call_patterns(call, language)]
|
||||
|
||||
|
||||
def _weak_literal_rule(language, sink_id, cwe, calls, literals, dialects):
|
||||
"""A `weak_literals` sink is a search, not a flow: the call with a named weak literal, or a call named after one."""
|
||||
alternatives = []
|
||||
lowered = {literal.lower() for literal in literals}
|
||||
for call in sorted(calls):
|
||||
for head in [call] if "." in call else [call, f"$OBJ.{call}"]:
|
||||
if call.rsplit(".", 1)[-1].lower() in lowered:
|
||||
alternatives.append({"pattern": f"{head}(...)"})
|
||||
else:
|
||||
alternatives.append(
|
||||
{
|
||||
"patterns": [
|
||||
{"pattern": f"{head}($ALGORITHM, ...)"},
|
||||
{
|
||||
"metavariable-regex": {
|
||||
"metavariable": "$ALGORITHM",
|
||||
"regex": "^[\"']?(" + "|".join(re.escape(literal) for literal in sorted(literals)) + ")[\"']?$",
|
||||
}
|
||||
},
|
||||
]
|
||||
}
|
||||
)
|
||||
return {
|
||||
"id": f"ousast.{language}.weak.{_ident(sink_id)}",
|
||||
"languages": dialects,
|
||||
"severity": "WARNING",
|
||||
"message": f"{sink_id}: a weak literal reaches {', '.join(sorted(calls))}",
|
||||
"metadata": {
|
||||
"family": _family_of(cwe),
|
||||
"cwe": cwe,
|
||||
"origin": f"sink:{language}:{sink_id}",
|
||||
"precision_estimate": "unmeasured",
|
||||
"kind": "weak_literal",
|
||||
"weak_literals": sorted(literals),
|
||||
},
|
||||
"pattern-either": alternatives,
|
||||
}
|
||||
|
||||
|
||||
def convert_facts(facts_dir=DEFAULT_FACTS_DIR):
|
||||
"""One taint rule per (language, sink id, CWE); the facts Semgrep has no word for are named, not dropped silently."""
|
||||
facts = load_facts(Path(facts_dir))
|
||||
converted, skipped, lossy = {}, [], []
|
||||
for language in sorted({fact.language for fact in facts.sinks}):
|
||||
scoped = facts.for_language(language)
|
||||
dialects = FACT_DIALECTS.get(language, [SEMGREP_LANGUAGES[language]])
|
||||
sources = [
|
||||
{"pattern": pattern} for fact in scoped.sources for token in fact.patterns for pattern in _source_patterns(token, language)
|
||||
]
|
||||
parameterized = set()
|
||||
sanitizers = []
|
||||
for fact in scoped.sanitizers:
|
||||
origin = f"sanitizer:{language}:{fact.id}"
|
||||
if fact.guard:
|
||||
skipped.append({"origin": origin, "reason": "guard: a check whose safety depends on dominance; Semgrep has no equivalent"})
|
||||
continue
|
||||
if fact.parameterized:
|
||||
parameterized.update(fact.calls)
|
||||
skipped.append(
|
||||
{"origin": origin, "reason": "parameterized: a call shape, not a cleansing; expressed as the single-argument sink form"}
|
||||
)
|
||||
continue
|
||||
if fact.literal_format_arg is not None:
|
||||
skipped.append(
|
||||
{
|
||||
"origin": origin,
|
||||
"reason": "literal_format_arg: a shape; a literal format is never tainted at the focused sink argument",
|
||||
}
|
||||
)
|
||||
continue
|
||||
if fact.quoted_only:
|
||||
loss = {"origin": origin, "loss": "quoted_only escape treated as a full sanitizer (silences the CVE-2024-1071 shape)"}
|
||||
if loss not in lossy:
|
||||
lossy.append(loss)
|
||||
sanitizers.extend({"pattern": pattern} for call in fact.calls for pattern in _call_patterns(call, language))
|
||||
for fact in scoped.dispatches:
|
||||
skipped.append(
|
||||
{
|
||||
"origin": f"dispatch:{language}:{fact.id}",
|
||||
"reason": "dispatch: string-keyed hook registry routing; no Semgrep equivalent",
|
||||
}
|
||||
)
|
||||
for fact in scoped.layouts:
|
||||
skipped.append({"origin": f"layout:{language}:{fact.id}", "reason": "layout: scope policy (vendored, tests), not a rule"})
|
||||
grouped = {}
|
||||
for fact in scoped.sinks:
|
||||
grouped.setdefault((fact.id, fact.cwe), []).append(fact)
|
||||
for (sink_id, cwe), rows in sorted(grouped.items()):
|
||||
origin = f"sink:{language}:{sink_id}"
|
||||
calls = sorted({call for row in rows for call in row.calls})
|
||||
literals = {literal for row in rows for literal in row.weak_literals}
|
||||
if literals:
|
||||
converted.setdefault(language, []).append(_weak_literal_rule(language, sink_id, cwe, calls, literals, dialects))
|
||||
continue
|
||||
entries, extra_sanitizers = [], []
|
||||
format_arg = next((row.format_arg for row in rows if row.format_arg is not None), None)
|
||||
for call in calls:
|
||||
if call == "v-html":
|
||||
skipped.append({"origin": f"{origin}:{call}", "reason": "template directive, not a call"})
|
||||
continue
|
||||
entries.extend(_sink_entries(call, language, format_arg=format_arg, single_argument=call in parameterized))
|
||||
for row in rows:
|
||||
if row.prefix_fixes_origin:
|
||||
lossy.append(
|
||||
{"origin": origin, "loss": "prefix_fixes_origin: a value appended after a fixed origin still counts as tainted"}
|
||||
)
|
||||
break
|
||||
for anchor in sorted({anchor for row in rows for anchor in row.origin_anchors}):
|
||||
extra_sanitizers.extend({"pattern": pattern} for pattern in _call_patterns(anchor, language))
|
||||
if not entries:
|
||||
skipped.append({"origin": origin, "reason": "no call form survives conversion"})
|
||||
continue
|
||||
rule = {
|
||||
"id": f"ousast.{language}.taint.{_ident(sink_id)}",
|
||||
"mode": "taint",
|
||||
"languages": dialects,
|
||||
"severity": "ERROR",
|
||||
"message": f"{sink_id}: untrusted input reaches {', '.join(calls)}",
|
||||
"metadata": {
|
||||
"family": _family_of(cwe),
|
||||
"cwe": cwe,
|
||||
"origin": origin,
|
||||
"precision_estimate": "unmeasured",
|
||||
"kind": "taint",
|
||||
"calls": calls,
|
||||
},
|
||||
"pattern-sources": [{"pattern-either": sources}],
|
||||
"pattern-sinks": [{"pattern-either": entries}],
|
||||
}
|
||||
if sanitizers or extra_sanitizers:
|
||||
rule["pattern-sanitizers"] = [{"pattern-either": [*sanitizers, *extra_sanitizers]}]
|
||||
converted.setdefault(language, []).append(rule)
|
||||
return converted, skipped, lossy
|
||||
|
||||
|
||||
def _obligation_tables(ruleset_dir):
|
||||
count = 0
|
||||
for path in sorted((Path(ruleset_dir) / "obligations").glob("*.toml")):
|
||||
payload = tomllib.loads(path.read_text())
|
||||
count += sum(len(value) for value in payload.values() if isinstance(value, list))
|
||||
return count
|
||||
|
||||
|
||||
def write_rules(out, *, ruleset_dir=DEFAULT_RULESET_DIR, facts_dir=DEFAULT_FACTS_DIR, min_precision=0.0, semgrep=None):
|
||||
"""Write one YAML per language under ``out`` plus ``conversion_report.json``; validate with Semgrep when present."""
|
||||
out = Path(out)
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
regex_rules, regex_skipped, rewrites = convert_regex_rules(ruleset_dir, min_precision=min_precision)
|
||||
taint_rules, fact_skipped, lossy = convert_facts(facts_dir)
|
||||
files = {}
|
||||
for language in sorted(set(regex_rules) | set(taint_rules)):
|
||||
rules = [*regex_rules.get(language, []), *taint_rules.get(language, [])]
|
||||
path = out / f"{language}.yaml"
|
||||
path.write_text(yaml.safe_dump({"rules": rules}, sort_keys=False, width=200, allow_unicode=True))
|
||||
files[path.name] = len(rules)
|
||||
obligations = _obligation_tables(ruleset_dir)
|
||||
report = {
|
||||
"regex": {
|
||||
"converted": sorted(rule["metadata"]["origin"] for rules in regex_rules.values() for rule in rules),
|
||||
"skipped": regex_skipped,
|
||||
"rewritten": rewrites,
|
||||
},
|
||||
"facts": {
|
||||
"converted": sorted(rule["metadata"]["origin"] for rules in taint_rules.values() for rule in rules),
|
||||
"skipped": fact_skipped,
|
||||
"lossy": lossy,
|
||||
},
|
||||
"obligations": {
|
||||
"skipped_tables": obligations,
|
||||
"reason": "an obligation finds a missing guard (an absence), which no pattern expresses",
|
||||
},
|
||||
"not_carried": [
|
||||
"comment skipping: the quick engine drops a regex match inside a comment; pattern-regex does not",
|
||||
"framework priors and fallback rules: the tagged rule is emitted, its language-level fallback is not",
|
||||
"min_evidence_level, ranking priority, the rung: post-match policy on our side",
|
||||
],
|
||||
"files": files,
|
||||
"validation": None,
|
||||
}
|
||||
binary = semgrep or shutil.which("semgrep")
|
||||
if binary:
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[binary, "--validate", "--config", str(out), "--metrics=off", "--disable-version-check"], capture_output=True, text=True
|
||||
)
|
||||
except OSError as error:
|
||||
report["validation"] = {"exit": None, "error": f"semgrep not found or not runnable: {type(error).__name__}"}
|
||||
else:
|
||||
report["validation"] = {"exit": completed.returncode, "stderr": completed.stderr[-4000:], "stdout": completed.stdout[-4000:]}
|
||||
(out / "conversion_report.json").write_text(json.dumps(report, indent=2, sort_keys=True) + "\n")
|
||||
return report
|
||||
|
||||
|
||||
# --- run ---------------------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def result_defect(result):
|
||||
"""One Semgrep result as a scorer defect; the family and CWE come from the rule's own metadata."""
|
||||
extra = result.get("extra", {})
|
||||
metadata = extra.get("metadata", {})
|
||||
path = result["path"]
|
||||
line = result.get("start", {}).get("line", 0)
|
||||
origin = metadata.get("origin", result["check_id"])
|
||||
return {
|
||||
"defect_id": f"{origin}:{path}:{line}",
|
||||
"family": metadata.get("family", "unknown"),
|
||||
"cwe": metadata.get("cwe"),
|
||||
"locations": [[path, line]],
|
||||
"source": "semgrep",
|
||||
"rule_id": result["check_id"],
|
||||
"origin": origin,
|
||||
"title": extra.get("message", ""),
|
||||
"rationale": extra.get("message", ""),
|
||||
"confidence": extra.get("severity"),
|
||||
"dataflow": bool(extra.get("dataflow_trace")),
|
||||
}
|
||||
|
||||
|
||||
def _write_json(path, value):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_suffix(".json.tmp")
|
||||
temporary.write_text(json.dumps(value, sort_keys=True, indent=2) + "\n")
|
||||
temporary.replace(path)
|
||||
|
||||
|
||||
def _record_path(out, identity):
|
||||
return out / (hashlib.sha256(identity.encode()).hexdigest() + ".json")
|
||||
|
||||
|
||||
def _inputs(out):
|
||||
path = out / "inputs.json"
|
||||
entries = json.loads(path.read_text()) if path.exists() else []
|
||||
for key in (lambda entry: entry["item_id"], lambda entry: entry["change"]["id"]):
|
||||
if len({key(entry) for entry in entries}) != len(entries):
|
||||
raise ValueError(f"duplicate ids in {path}")
|
||||
return entries
|
||||
|
||||
|
||||
def run_semgrep(binary, rules, root, targets, *, timeout=900):
|
||||
"""Semgrep over explicit targets inside ``root``; paths in the output stay repository-relative."""
|
||||
argv = [binary, "scan", "--config", str(rules), *SEMGREP_FLAGS, *targets]
|
||||
started = time.monotonic()
|
||||
completed = subprocess.run(argv, cwd=root, capture_output=True, text=True, timeout=timeout)
|
||||
return completed, time.monotonic() - started
|
||||
|
||||
|
||||
def _scan_change(change, repos, rules, *, binary):
|
||||
instrument = {
|
||||
"hook_exit": 0,
|
||||
"hook_bytes_read": 0,
|
||||
"changed_bytes": {},
|
||||
"head_verified": False,
|
||||
"base_verified": False,
|
||||
"failure": None,
|
||||
"suspect_fast": False,
|
||||
"semgrep": {"exit": None, "wall_seconds": None, "scanned": [], "errors": [], "results": 0},
|
||||
}
|
||||
record = {
|
||||
"id": change["id"],
|
||||
"done": True,
|
||||
"status": "semgrep_covered",
|
||||
"instrument": instrument,
|
||||
"push": {"admission": {"defects": [], "dispositions": []}, "quick_tier": []},
|
||||
"skipped": [],
|
||||
}
|
||||
try:
|
||||
if not all(SHA.fullmatch(change[side]) for side in ("base", "head")):
|
||||
raise InstrumentFailure("invalid_revision")
|
||||
git = local_git(Path(repos[change["repo"]]).expanduser().resolve())
|
||||
git.run("cat-file", "-e", change["base"] + "^{commit}")
|
||||
instrument["base_verified"] = True
|
||||
with tempfile.TemporaryDirectory(prefix="ousast-semgrep-") as temporary:
|
||||
root = Path(temporary) / "head"
|
||||
added = False
|
||||
try:
|
||||
git.run("-c", "core.hooksPath=/dev/null", "worktree", "add", "--detach", str(root), change["head"])
|
||||
added = True
|
||||
if local_git(root).run("rev-parse", "HEAD").strip() != change["head"]:
|
||||
raise InstrumentFailure("head_mismatch")
|
||||
instrument["head_verified"] = True
|
||||
targets = []
|
||||
for relative in dict.fromkeys(file["path"] for file in change.get("files", [])):
|
||||
path = clamp_repo_path(root, relative)
|
||||
try:
|
||||
path.stat()
|
||||
except FileNotFoundError:
|
||||
record["skipped"].append({"path": relative, "reason": "missing_at_head"})
|
||||
print(f"{change['id']} {relative}: skipped (missing_at_head)", flush=True)
|
||||
continue
|
||||
try:
|
||||
data = path.read_bytes()
|
||||
except OSError:
|
||||
print(f"{change['id']} {relative}: 0 bytes read", flush=True)
|
||||
raise InstrumentFailure(f"unreadable_target:{relative}") from None
|
||||
size = len(data)
|
||||
print(f"{change['id']} {relative}: {size} bytes read", flush=True)
|
||||
if not size:
|
||||
record["skipped"].append({"path": relative, "reason": "empty_at_head"})
|
||||
continue
|
||||
instrument["changed_bytes"][relative] = size
|
||||
instrument["hook_bytes_read"] += size
|
||||
targets.append(relative)
|
||||
if not targets:
|
||||
raise InstrumentFailure("no_readable_targets")
|
||||
try:
|
||||
completed, wall = run_semgrep(binary, rules, root, targets)
|
||||
except subprocess.TimeoutExpired:
|
||||
raise InstrumentFailure("semgrep_timeout") from None
|
||||
semgrep = instrument["semgrep"]
|
||||
semgrep.update(exit=completed.returncode, wall_seconds=round(wall, 3))
|
||||
instrument["hook_exit"] = completed.returncode
|
||||
if wall < MIN_PLAUSIBLE_SECONDS:
|
||||
instrument["suspect_fast"] = True
|
||||
if completed.returncode not in (0, 1):
|
||||
semgrep["stderr"] = completed.stderr[-2000:]
|
||||
raise InstrumentFailure(f"semgrep_exit_{completed.returncode}")
|
||||
try:
|
||||
payload = json.loads(completed.stdout)
|
||||
except json.JSONDecodeError:
|
||||
semgrep["stderr"] = completed.stderr[-2000:]
|
||||
raise InstrumentFailure("semgrep_output_unreadable") from None
|
||||
semgrep["scanned"] = sorted(payload.get("paths", {}).get("scanned", []))
|
||||
semgrep["errors"] = [
|
||||
{"type": str(error.get("type")), "path": error.get("path"), "message": str(error.get("message", ""))[:300]}
|
||||
for error in payload.get("errors", [])
|
||||
]
|
||||
results = payload.get("results", [])
|
||||
semgrep["results"] = len(results)
|
||||
record["push"]["admission"]["defects"] = [result_defect(result) for result in results]
|
||||
print(f"{change['id']}: semgrep exit {completed.returncode}, {len(results)} results, {wall:.1f}s", flush=True)
|
||||
finally:
|
||||
if added:
|
||||
git.run("worktree", "remove", "--force", str(root))
|
||||
except Exception as error: # noqa: BLE001
|
||||
reason = str(error) if isinstance(error, InstrumentFailure) else type(error).__name__
|
||||
instrument.update(hook_exit=2, failure=reason)
|
||||
record["status"] = "instrument_failure"
|
||||
record["push"]["admission"]["defects"] = []
|
||||
print(f"{change['id']}: instrument_failure ({reason})", file=sys.stderr, flush=True)
|
||||
return record
|
||||
|
||||
|
||||
def run(manifest, repos_path, rules, out, *, slice_id="1", only="all", semgrep=None):
|
||||
binary = semgrep or shutil.which("semgrep")
|
||||
if binary is None:
|
||||
print("No semgrep binary found on PATH.", file=sys.stderr)
|
||||
return 2
|
||||
rules = Path(rules).expanduser().resolve()
|
||||
if not rules.is_dir():
|
||||
raise ValueError(f"rules directory missing: {rules}")
|
||||
changes = load_slice(manifest, slice_id)
|
||||
if len({change["id"] for change in changes}) != len(changes):
|
||||
raise ValueError("duplicate change ids")
|
||||
changes = [change for change in changes if only == "all" or change["kind"] == only]
|
||||
entries = _inputs(out)
|
||||
indexed = {entry["change"]["id"]: entry for entry in entries}
|
||||
pending = []
|
||||
for change in changes:
|
||||
path = _record_path(out, change["id"])
|
||||
previous = json.loads(path.read_text()) if path.exists() else {}
|
||||
entry = indexed.get(change["id"])
|
||||
if entry is None:
|
||||
entry = {
|
||||
"item_id": change["id"],
|
||||
"change": change,
|
||||
"input_digest": previous.get("input_digest") or hashlib.sha256(canonical(change)).hexdigest(),
|
||||
}
|
||||
entries.append(entry)
|
||||
elif entry["change"] != change or entry["item_id"] != change["id"]:
|
||||
raise ValueError("existing semgrep inputs do not match frozen change")
|
||||
if previous.get("done") is True:
|
||||
print(f"{change['id']}: skipped (done)", flush=True)
|
||||
continue
|
||||
pending.append(entry)
|
||||
_write_json(out / "inputs.json", entries)
|
||||
if not pending:
|
||||
return 0
|
||||
repos = json.loads(Path(repos_path).expanduser().read_text())
|
||||
failed = False
|
||||
for entry in pending:
|
||||
record = _scan_change(entry["change"], repos, rules, binary=binary)
|
||||
record["input_digest"] = entry["input_digest"]
|
||||
_write_json(_record_path(out, entry["item_id"]), record)
|
||||
failed |= record["status"] == "instrument_failure"
|
||||
return 2 if failed else 0
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
commands = parser.add_subparsers(dest="command", required=True)
|
||||
rules = commands.add_parser("rules", help="convert the ruleset and the facts into Semgrep rules")
|
||||
rules.add_argument("--out", type=Path, required=True)
|
||||
rules.add_argument("--ruleset", type=Path, default=DEFAULT_RULESET_DIR)
|
||||
rules.add_argument("--facts", type=Path, default=DEFAULT_FACTS_DIR)
|
||||
rules.add_argument("--min-precision", type=float, default=0.0, help="the hook's min_emit_precision floor")
|
||||
rules.add_argument("--semgrep", help="semgrep binary (default: PATH)")
|
||||
scan = commands.add_parser("run", help="run Semgrep over each frozen change's changed files")
|
||||
scan.add_argument("manifest", type=Path)
|
||||
scan.add_argument("--slice", default="1")
|
||||
scan.add_argument("--repos", type=Path, required=True)
|
||||
scan.add_argument("--rules", type=Path, required=True)
|
||||
scan.add_argument("--out", type=Path, required=True)
|
||||
scan.add_argument("--only", choices=("vulnerability", "ordinary", "all"), default="all")
|
||||
scan.add_argument("--semgrep", help="semgrep binary (default: PATH)")
|
||||
args = parser.parse_args(argv)
|
||||
if args.command == "rules":
|
||||
report = write_rules(
|
||||
args.out, ruleset_dir=args.ruleset, facts_dir=args.facts, min_precision=args.min_precision, semgrep=args.semgrep
|
||||
)
|
||||
validation = report["validation"]
|
||||
print(
|
||||
f"regex converted {len(report['regex']['converted'])}, skipped {len(report['regex']['skipped'])}; "
|
||||
f"facts converted {len(report['facts']['converted'])}, skipped {len(report['facts']['skipped'])}, "
|
||||
f"lossy {len(report['facts']['lossy'])}; "
|
||||
f"obligations skipped {report['obligations']['skipped_tables']}; "
|
||||
f"validation {'not run' if validation is None else 'exit ' + str(validation['exit'])}"
|
||||
)
|
||||
return 0 if validation is None or validation["exit"] == 0 else 2
|
||||
return run(args.manifest, args.repos, args.rules, args.out.expanduser(), slice_id=args.slice, only=args.only, semgrep=args.semgrep)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user