feat(detection): language-scoped rules + multi-language benchmark corpora

Rebuild the quick-scan detection engine and ground-truth benchmarks to hit >=90% recall / <10% false positives for C/C++, Python, JavaScript, and Java. Engine (findings.py, preprocess.py): - Scope each PatternRule by language (fixes cross-language false positives such as the Python eval/exec rule firing on JS .exec()). - Add JavaScript/TypeScript rules (command exec, eval/Function, reflected XSS, SQLi, path traversal, SSRF, weak hash, deserialization); execFile/spawn-argv excluded as safe alternatives. - Add Java rules (Runtime exec, SQL string concatenation, weak hash, readObject) and a Groovy server-side template rule (yieldUnescaped of a bound variable). - Broaden C/C++ and Python coverage; precision guards for format strings and parameterised vs concatenated SQL; #-comment skipping; CWE in rationale. - Register .java/.groovy/.tpl source extensions. Benchmarks: - Add realistic corpora modeled on Damn Vulnerable C/C++, PyGoat/DVPWA, and NodeGoat, plus the real kiview/damn-vulnerable-spring-boot-app vendored verbatim (MIT, attribution in LICENSE/NOTICE.md). Each manifest carries complete, independently-labeled ground truth including genuinely hard-to-detect vulns (integer overflow, second-order SQLi, prototype pollution) so recall is measured honestly, plus safe-API files for precision. - Fix pre-existing smoke-manifest ground-truth errors (Python/JS line offsets, missing C/C++ strcpy entry, Java SQLi line). benchmark.py: - Fix sink matching to normalise the expected sink consistently so sinks with underscores (e.g. render_template_string) can match. Tests: - Add tests/test_detection_benchmarks.py enforcing the >=90% recall / <10% FP goal per language and overall. Measured: C/C++ 93.3%, Python 93.3%, JavaScript 92.3%, Java 100%; overall 93.6% recall, 0.0% false-positive rate. Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-16 03:49:26 +00:00 · 2026-06-21 11:49:31 +02:00
parent be65aed96e
commit cf9f945992
38 changed files with 2420 additions and 15 deletions
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2017 GDATA Advanced Analytics GmbH
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
@@ -0,0 +1 @@
+Vendored verbatim from https://github.com/kiview/damn-vulnerable-spring-boot-app (MIT, (c) 2017 GDATA Advanced Analytics GmbH) as a benchmark corpus for OpenUltraSAST. Source files unmodified; line numbers match upstream.
@@ -0,0 +1,13 @@
+package com.groovycoder.dvsba;
+
+import org.springframework.boot.SpringApplication;
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+
+@SpringBootApplication
+public class DvsbaApplication {
+
+    public static void main(String[] args) {
+        SpringApplication.run(DvsbaApplication.class, args);
+    }
+
+}
@@ -0,0 +1,14 @@
+package com.groovycoder.dvsba;
+
+import org.springframework.stereotype.Controller;
+import org.springframework.web.bind.annotation.GetMapping;
+import org.springframework.web.servlet.ModelAndView;
+
+@Controller
+public class HomeController {
+
+    @GetMapping("/")
+    public ModelAndView home() {
+        return new ModelAndView("views/home");
+    }
+}
@@ -0,0 +1,13 @@
+package com.groovycoder.dvsba.books;
+
+public class Book {
+    Long id;
+    String name;
+    String author;
+
+    public Book(Long id, String name, String author) {
+        this.id = id;
+        this.name = name;
+        this.author = author;
+    }
+}
@@ -0,0 +1,88 @@
+package com.groovycoder.dvsba.books;
+
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.jdbc.core.ResultSetExtractor;
+import org.springframework.stereotype.Controller;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestParam;
+import org.springframework.web.servlet.ModelAndView;
+
+import javax.annotation.PostConstruct;
+import java.util.*;
+
+@RequestMapping("/books")
+@Controller
+class BookController {
+
+    private final JdbcTemplate jdbcTemplate;
+
+    public BookController(JdbcTemplate jdbcTemplate) {
+        this.jdbcTemplate = jdbcTemplate;
+    }
+
+    @RequestMapping("/")
+    public ModelAndView home() {
+        List<Book> books = loadBooks();
+
+        Map<String, Object> model = new HashMap<>();
+        model.put("books", books);
+
+        return new ModelAndView("views/sql/home", model);
+    }
+
+    @RequestMapping("/detail")
+    public ModelAndView detail(@RequestParam(value = "id") String id) {
+        String sql = "SELECT * FROM books WHERE id=" + id;
+        final Book[] book = new Book[1];
+        jdbcTemplate.query(sql, (ResultSetExtractor) rs -> {
+            if (rs.next())
+                book[0] = new Book(rs.getLong(1), rs.getString(2), rs.getString(3));
+
+            return null;
+
+        });
+
+        Map<String, Object> model = new HashMap<>();
+        model.put("book", book[0]);
+
+        return new ModelAndView("views/sql/detail", model);
+    }
+
+    @PostConstruct
+    private void bootstrap() {
+        initDb();
+
+        List<Book> books = Arrays.asList(
+                new Book(1L, "Moby Dick", "Herman Melville"),
+                new Book(2L, "Unsichtbare Spuren", "Andreas Franz"),
+                new Book(3L, "Das Paket", "Sebastian Fitzek")
+        );
+        saveBooks(books);
+    }
+
+    private void saveBooks(List<Book> books) {
+        books.forEach(book ->
+                jdbcTemplate.update("INSERT INTO books (id, `name`, author) values (?, ?, ?)",
+                        book.id, book.name, book.author)
+        );
+    }
+
+    private List<Book> loadBooks() {
+
+        return jdbcTemplate.query("SELECT * FROM books", rs -> {
+            List<Book> books = new LinkedList<>();
+
+            while (rs.next()) {
+                books.add(new Book(rs.getLong(1), rs.getString(2), rs.getString(3)));
+            }
+
+            return books;
+        });
+
+    }
+
+    private void initDb() {
+        jdbcTemplate.execute("CREATE TABLE books (id NUMBER, name VARCHAR(255), author VARCHAR(255))");
+    }
+
+}
@@ -0,0 +1,47 @@
+package com.groovycoder.dvsba.comments;
+
+import org.springframework.stereotype.Controller;
+import org.springframework.web.bind.annotation.GetMapping;
+import org.springframework.web.bind.annotation.PostMapping;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestParam;
+import org.springframework.web.servlet.ModelAndView;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+@Controller
+@RequestMapping("/comments")
+public class CommentController {
+
+    private List<String> comments = new ArrayList<>(Arrays.asList(
+            "Hi super Webseite leider keine Mobile App deswegen nur einen Stern.",
+            "Ich muss darauf hinweisen, dass eine Webseite ohne Impressum zu betreiben !!!strafbar!!! ist!!!",
+            "Hallo, wir sinds! Wolltest du nicht die Fotos von Samstag hochladen? Grüße!"));
+
+    @GetMapping(path = "/")
+    public ModelAndView listComments() {
+        Map<String, Object> model = new HashMap<>();
+        model.put("comments", comments);
+
+        return new ModelAndView(
+                "views/comment/comment",
+                model);
+    }
+
+    @PostMapping("/")
+    public ModelAndView postComment(@RequestParam String input) {
+        comments.add(input);
+
+        Map<String, Object> model = new HashMap<>();
+        model.put("comments", comments);
+
+        return new ModelAndView(
+                "views/comment/comment",
+                model);
+    }
+
+}
@@ -0,0 +1,25 @@
+yieldUnescaped '<!DOCTYPE html>'
+html {
+    head {
+        title('Spring Boot - Groovy templates example')
+        link(rel: 'stylesheet', href: '/css/bootstrap.min.css')
+    }
+    body {
+        form(action: "/comments/", method: "POST") {
+            input(name: "input") {
+
+            }
+        }
+
+        comments.each { comment ->
+            div {
+                h3(style: "border-bottom: 1px solid black;") {
+                    yield "Comment by Anonymous"
+                }
+                div(style: "border-bottom: 1px solid black;") {
+                    yieldUnescaped comment
+                }
+            }
+        }
+    }
+}
@@ -0,0 +1,13 @@
+yieldUnescaped '<!DOCTYPE html>'
+html {
+    head {
+        title('Damn Vulnerable Spring Boot App - Home')
+    }
+    body {
+        h1 {
+            yield 'Hello and Welcome to the DVSBA!'
+        }
+        p ("Take a look around! We have a ${$a(href:'/books/', "catalouge")} of our favourite books which you can browse. If you like it, leave a comment in the ${ $a(href: '/comments/', "comment section")}!")
+
+    }
+}
@@ -0,0 +1,16 @@
+yieldUnescaped '<!DOCTYPE html>'
+html {
+    head {
+        title('Damn Vulnerable Spring Boot App')
+    }
+    body {
+        div(class: 'container') {
+            a(class: 'brand',
+                href: '/books/',
+                'List')
+            br()
+            br()
+            p "ID: ${book.id}, Name: ${book.name}, Author: ${book.author}"
+        }
+    }
+}
@@ -0,0 +1,38 @@
+yieldUnescaped '<!DOCTYPE html>'
+html {
+    head {
+        title('Damn Vulnerable Spring Boot App')
+    }
+    body {
+        div(class: 'container') {
+            form(method: "GET", action: "/books/detail") {
+                input(name: "id", placeholder: "Search for a book id!")
+            }
+            br
+            table(border:1) {
+                thead {
+                    tr {
+                        td 'ID'
+                        td 'Name'
+                        td 'Author'
+                      }
+                    }
+                tbody {
+                    books.each { book ->
+                        tr {
+                            td {
+                                a(href: "/books/detail?id=${book.id}", "${book.id}")
+                            }
+                            td {
+                                a(href: "/books/detail?id=${book.id}", "${book.name}")
+                            }
+                            td {
+                                a(href: "/books/detail?id=${book.id}", "${book.author}")
+                            }
+                        }
+                    }
+                }
+            }
+        }
+    }
+}
				`@@ -0,0 +1 @@`
				`Vendored verbatim from https://github.com/kiview/damn-vulnerable-spring-boot-app (MIT, (c) 2017 GDATA Advanced Analytics GmbH) as a benchmark corpus for OpenUltraSAST. Source files unmodified; line numbers match upstream.`