.kiro/specs/tree-sitter-overlay-extra/brief.md

# Brief: tree-sitter-overlay-extra

## Problem

AppSec scanning mixed working trees gets `language_unsupported` on JS/C/Java, and Python-only `ast` cannot follow the patterns those languages actually use. The overlay spec promised tree-sitter as the primary engine; the CLI probe never builds `FileIR`. Maintaining hand-rolled parsers would explode the codebase.

## Current State

`parse_tree_sitter_cli` returns `None` even on a successful parse. Python falls back to stdlib `ast`. JS/C/Java stay unadjudicated. `quick` is correctly zero-dep. SAST overlay Youden on the 11-pair textbook slice is 0% vs +9% inventory, not because regex got worse, but because Python overlay fail-closes on ConfigParser-style heap and C/Java never parse.

## Desired Outcome

With the extra installed, tree-sitter grammars produce `FileIR` for Python, JavaScript, C/C++, and Java. Existing taint/facts/dispositions consume that IR unchanged. Without the extra, behavior stays fail-closed unadjudicated. `quick` still has no parser wheels.

## Approach

Add optional extra `openultrasast[semantic]` (`tree-sitter` + official language packages). One CST walker extracts binds and calls into the current `FileIR`. Delete the dead CLI sexp stub. Do not add a fourth language-specific AST visitor.

## Scope

- In: extra, probe that prefers the Python API when importable, CST → FileIR, tests with and without the extra.
- Out: Joern queries, inter-file taint, CWE-121 buffer semantics, new pair corpora (owned by `real-world-vfc-slice`).

## Boundary Candidates

- Packaging extra vs core install
- CST walker vs taint engine (taint stays in `semantic/taint.py`)

## Out of Boundary

- Pattern rules, CWE policy, sandbox, mechanism JSONL schema, pair catalog harvest

## Upstream / Downstream

- Upstream: `propose-adjudicate-prove` overlay records and taint
- Downstream: `real-world-vfc-slice` overlay scoring on C/JS/Java pairs

## Existing Spec Touchpoints

- Extends: none (new spec). Does not reopen `propose-adjudicate-prove` tasks.
- Adjacent: `three-stage-scan` modes; `quick` must remain zero-dep.

## Constraints

No new required dependency. Missing extra → `language_unsupported` or Python ast, never crash. No LLM parse.
feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra 2026-09-04 15:07:35 +02:00			`# Brief: tree-sitter-overlay-extra`

			`## Problem`

			AppSec scanning mixed working trees gets `language_unsupported` on JS/C/Java, and Python-only `ast` cannot follow the patterns those languages actually use. The overlay spec promised tree-sitter as the primary engine; the CLI probe never builds `FileIR`. Maintaining hand-rolled parsers would explode the codebase.

			`## Current State`

docs: plain-English pass over the active specs 2026-10-12 18:16:41 +02:00			`parse_tree_sitter_cli` returns `None` even on a successful parse. Python falls back to stdlib `ast`. JS/C/Java stay unadjudicated. `quick` is correctly zero-dep. SAST overlay Youden on the 11-pair textbook slice is 0% vs +9% inventory, not because regex got worse, but because Python overlay fail-closes on ConfigParser-style heap and C/Java never parse.
feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra 2026-09-04 15:07:35 +02:00
			`## Desired Outcome`

			With the extra installed, tree-sitter grammars produce `FileIR` for Python, JavaScript, C/C++, and Java. Existing taint/facts/dispositions consume that IR unchanged. Without the extra, behavior stays fail-closed unadjudicated. `quick` still has no parser wheels.

			`## Approach`

			Add optional extra `openultrasast[semantic]` (`tree-sitter` + official language packages). One CST walker extracts binds and calls into the current `FileIR`. Delete the dead CLI sexp stub. Do not add a fourth language-specific AST visitor.

			`## Scope`

docs: plain-English pass over the active specs 2026-10-12 18:16:41 +02:00			`- In: extra, probe that prefers the Python API when importable, CST → FileIR, tests with and without the extra.`
			- Out: Joern queries, inter-file taint, CWE-121 buffer semantics, new pair corpora (owned by `real-world-vfc-slice`).
feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra 2026-09-04 15:07:35 +02:00
			`## Boundary Candidates`

			`- Packaging extra vs core install`
			- CST walker vs taint engine (taint stays in `semantic/taint.py`)

			`## Out of Boundary`

			`- Pattern rules, CWE policy, sandbox, mechanism JSONL schema, pair catalog harvest`

			`## Upstream / Downstream`

docs: plain-English pass over the active specs 2026-10-12 18:16:41 +02:00			- Upstream: `propose-adjudicate-prove` overlay records and taint
			- Downstream: `real-world-vfc-slice` overlay scoring on C/JS/Java pairs
feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra 2026-09-04 15:07:35 +02:00
			`## Existing Spec Touchpoints`

docs: plain-English pass over the active specs 2026-10-12 18:16:41 +02:00			- Extends: none (new spec). Does not reopen `propose-adjudicate-prove` tasks.
			- Adjacent: `three-stage-scan` modes; `quick` must remain zero-dep.
feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra 2026-09-04 15:07:35 +02:00
			`## Constraints`

			No new required dependency. Missing extra → `language_unsupported` or Python ast, never crash. No LLM parse.