Files
Open_Ultra_SAST/.kiro/specs/tree-sitter-overlay-extra/brief.md
T
Marius CiepluchandClaude Opus 4.8 39a1873e5b docs: plain-English pass over the active specs
Ran the humanizer rules over 79 prose files in the active specs (brief,
design, requirements, research and narrative docs). Removed em dashes,
significance inflation, forced rule-of-three, bold-label list items and
filler. Skipped tasks.md checklists and the retired specs (harnessx-*,
openrouter-sast-harness, unseen-repo-evaluation).

Every number, path, command, hash and identifier was preserved: checked
each file against HEAD for drift in numbers, inline code and code fences,
zero drift across all 79 files.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-10-12 18:16:41 +02:00

2.2 KiB

Brief: tree-sitter-overlay-extra

Problem

AppSec scanning mixed working trees gets language_unsupported on JS/C/Java, and Python-only ast cannot follow the patterns those languages actually use. The overlay spec promised tree-sitter as the primary engine; the CLI probe never builds FileIR. Maintaining hand-rolled parsers would explode the codebase.

Current State

parse_tree_sitter_cli returns None even on a successful parse. Python falls back to stdlib ast. JS/C/Java stay unadjudicated. quick is correctly zero-dep. SAST overlay Youden on the 11-pair textbook slice is 0% vs +9% inventory, not because regex got worse, but because Python overlay fail-closes on ConfigParser-style heap and C/Java never parse.

Desired Outcome

With the extra installed, tree-sitter grammars produce FileIR for Python, JavaScript, C/C++, and Java. Existing taint/facts/dispositions consume that IR unchanged. Without the extra, behavior stays fail-closed unadjudicated. quick still has no parser wheels.

Approach

Add optional extra openultrasast[semantic] (tree-sitter + official language packages). One CST walker extracts binds and calls into the current FileIR. Delete the dead CLI sexp stub. Do not add a fourth language-specific AST visitor.

Scope

  • In: extra, probe that prefers the Python API when importable, CST → FileIR, tests with and without the extra.
  • Out: Joern queries, inter-file taint, CWE-121 buffer semantics, new pair corpora (owned by real-world-vfc-slice).

Boundary Candidates

  • Packaging extra vs core install
  • CST walker vs taint engine (taint stays in semantic/taint.py)

Out of Boundary

  • Pattern rules, CWE policy, sandbox, mechanism JSONL schema, pair catalog harvest

Upstream / Downstream

  • Upstream: propose-adjudicate-prove overlay records and taint
  • Downstream: real-world-vfc-slice overlay scoring on C/JS/Java pairs

Existing Spec Touchpoints

  • Extends: none (new spec). Does not reopen propose-adjudicate-prove tasks.
  • Adjacent: three-stage-scan modes; quick must remain zero-dep.

Constraints

No new required dependency. Missing extra → language_unsupported or Python ast, never crash. No LLM parse.