feat(detection): language-scoped rules + multi-language benchmark corpora

Rebuild the quick-scan detection engine and ground-truth benchmarks to hit
>=90% recall / <10% false positives for C/C++, Python, JavaScript, and Java.

Engine (findings.py, preprocess.py):
- Scope each PatternRule by language (fixes cross-language false positives such
  as the Python eval/exec rule firing on JS .exec()).
- Add JavaScript/TypeScript rules (command exec, eval/Function, reflected XSS,
  SQLi, path traversal, SSRF, weak hash, deserialization); execFile/spawn-argv
  excluded as safe alternatives.
- Add Java rules (Runtime exec, SQL string concatenation, weak hash, readObject)
  and a Groovy server-side template rule (yieldUnescaped of a bound variable).
- Broaden C/C++ and Python coverage; precision guards for format strings and
  parameterised vs concatenated SQL; #-comment skipping; CWE in rationale.
- Register .java/.groovy/.tpl source extensions.

Benchmarks:
- Add realistic corpora modeled on Damn Vulnerable C/C++, PyGoat/DVPWA, and
  NodeGoat, plus the real kiview/damn-vulnerable-spring-boot-app vendored
  verbatim (MIT, attribution in LICENSE/NOTICE.md). Each manifest carries
  complete, independently-labeled ground truth including genuinely
  hard-to-detect vulns (integer overflow, second-order SQLi, prototype
  pollution) so recall is measured honestly, plus safe-API files for precision.
- Fix pre-existing smoke-manifest ground-truth errors (Python/JS line offsets,
  missing C/C++ strcpy entry, Java SQLi line).

benchmark.py:
- Fix sink matching to normalise the expected sink consistently so sinks with
  underscores (e.g. render_template_string) can match.

Tests:
- Add tests/test_detection_benchmarks.py enforcing the >=90% recall / <10% FP
  goal per language and overall.

Measured: C/C++ 93.3%, Python 93.3%, JavaScript 92.3%, Java 100%;
overall 93.6% recall, 0.0% false-positive rate.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-06-21 11:49:31 +02:00
co-authored by Claude Opus 4.8
parent be65aed96e
commit cf9f945992
38 changed files with 2420 additions and 15 deletions
@@ -0,0 +1,59 @@
// Vulnerable Express app modeled on NodeGoat / vulnerable JavaScript samples.
const express = require("express");
const childProcess = require("child_process");
const fs = require("fs");
const crypto = require("crypto");
const axios = require("axios");
const app = express();
app.get("/ping", (req, res) => {
childProcess.exec("ping -c 1 " + req.query.host, (e, out) => res.send(out)); // CWE-78 command injection
});
app.get("/build", (req, res) => {
childProcess.execSync("make " + req.query.target); // CWE-78 command injection via execSync
});
app.get("/calc", (req, res) => {
const result = eval(req.query.expr); // CWE-95 eval of user input
res.send(String(result));
});
app.get("/fn", (req, res) => {
const f = new Function("x", req.query.body); // CWE-95 Function constructor injection
res.send(String(f(1)));
});
app.get("/hello", (req, res) => {
res.send("<h1>Hello " + req.query.name + "</h1>"); // CWE-79 reflected XSS
});
app.get("/user", (req, res) => {
db.query("select * from users where name = '" + req.query.name + "'"); // CWE-89 SQL injection
});
app.get("/report", (req, res) => {
db.query(`select * from reports where id = ${req.query.id}`); // CWE-89 SQL injection template literal
});
app.get("/read", (req, res) => {
const data = fs.readFileSync(req.query.path); // CWE-22 path traversal
res.send(data);
});
app.get("/fetch", (req, res) => {
axios.get(req.query.url).then((r) => res.send(r.data)); // CWE-918 SSRF
});
app.get("/hash", (req, res) => {
const digest = crypto.createHash("md5").update(req.query.pw).digest("hex"); // CWE-327 weak hash
res.send(digest);
});
app.get("/merge", (req, res) => {
const target = {};
for (const k in req.query) target[k] = req.query[k]; // CWE-1321 prototype pollution (regex blind spot)
res.json(target);
});
module.exports = app;