mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00

Files

T

Marius Ciepluch e9f185da59 feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

Add openultrasast[semantic] with official grammar wheels and one CST walker
into FileIR. quick stays extra-free. Host CLI is not overlay-ready. Default CI
stays extra-free; a semantic job runs pytest -m semantic.

2026-09-04 15:07:35 +02:00

2.2 KiB

Raw Blame History

Brief: tree-sitter-overlay-extra

Problem

AppSec scanning mixed working trees gets language_unsupported on JS/C/Java, and Python-only ast cannot follow the patterns those languages actually use. The overlay spec promised tree-sitter as the primary engine; the CLI probe never builds FileIR. Maintaining hand-rolled parsers would explode the codebase.

Current State

parse_tree_sitter_cli returns None even on a successful parse. Python falls back to stdlib ast. JS/C/Java stay unadjudicated. quick is correctly zero-dep. SAST overlay Youden on the 11-pair textbook slice is 0% vs +9% inventory — not because regex got worse, but because Python overlay fail-closes on ConfigParser-style heap and C/Java never parse.

Desired Outcome

With the extra installed, tree-sitter grammars produce FileIR for Python, JavaScript, C/C++, and Java. Existing taint/facts/dispositions consume that IR unchanged. Without the extra, behavior stays fail-closed unadjudicated. quick still has no parser wheels.

Approach

Add optional extra openultrasast[semantic] (tree-sitter + official language packages). One CST walker extracts binds and calls into the current FileIR. Delete the dead CLI sexp stub. Do not add a fourth language-specific AST visitor.

Scope

In: extra, probe that prefers the Python API when importable, CST → FileIR, tests with and without the extra.
Out: Joern queries, inter-file taint, CWE-121 buffer semantics, new pair corpora (owned by real-world-vfc-slice).

Boundary Candidates

Packaging extra vs core install
CST walker vs taint engine (taint stays in semantic/taint.py)

Out of Boundary

Pattern rules, CWE policy, sandbox, mechanism JSONL schema, pair catalog harvest

Upstream / Downstream

Upstream: propose-adjudicate-prove overlay records and taint
Downstream: real-world-vfc-slice overlay scoring on C/JS/Java pairs

Existing Spec Touchpoints

Extends: none (new spec). Does not reopen propose-adjudicate-prove tasks.
Adjacent: three-stage-scan modes; quick must remain zero-dep.

Constraints

No new required dependency. Missing extra → language_unsupported or Python ast, never crash. No LLM parse.

2.2 KiB Raw Blame History