feat(propose-adjudicate-prove): overlay adjudicates inventory before prove

Regex inventory stays the proposer. MAP attaches promote/demote/unadjudicated/coverage from intra-file taint (tree-sitter when a grammar parses, Python ast as last resort). Deep sandboxes promotions only, ordered by the existing heuristic ranker plus optional OpenRouter mechanism recall. The SAST pair slice scores overlay outcomes; the stage-1 detection gate and local pair gate stay inventory-only.
2026-10-16 03:49:26 +00:00 · 2026-09-04 13:57:33 +02:00
parent 32aa92191a
commit 1ae41eea19
76 changed files with 6538 additions and 29 deletions
@@ -28,6 +28,8 @@ jobs:
        run: uv run python -m openultrasast.gate
      - name: Split-sink map gate (stage 2 beats pattern-hit ranking)
        run: uv run python -m openultrasast.map_gate
      - name: Pair-efficiency gate (local vuln vs fix; GitHub honesty dashboard)
        run: uv run python -m openultrasast.pair_gate
      - run: uv run python -m compileall src tests
  dagger:
@@ -0,0 +1,35 @@
 # Brief: regex proposes, semantics adjudicates, sandbox proves
 ## Problem
 OpenUltraSAST already has three scan stages (`quick` inventory, `standard` map, `deep` regression), but **only the first stage actually decides**. Pattern rules propose sinks; the map ranks files; the sandbox can run recipes. Nothing in the middle answers “does this sink receive attacker-controlled data, or is it a Juliet-style good twin / dead assignment / hardcoded command?”
 Calibration corpora (cheat-sheet fixtures, OWASP Benchmark true/false files, Juliet `bad`/`good`, CVE pairs) are **nice**: one CWE per file, named sinks, labeled counterparts. Real working trees are not: mixed languages, generated code, unparseable files, framework wrappers, comments, one-hop assignments (`sql = f"…{x}"; cur.execute(sql)`), and no ground-truth labels.
 Measured honesty on the labeled SAST slice is about **+9% Youden**. Widening regex to chase that number will fire on every `Runtime.exec` and leak on OWASP-false / Juliet `goodG2B`.
 ## Approach
 Keep three roles, never collapse them:
 1. **Propose** — pattern rules + optional SARIF. Stage 1. Does not decide worth-fixing.
 2. **Adjudicate** — tree-sitter IR (multi-language) + taint facts; optional Joern CPG sidecar; Python stdlib parse only as last-resort fallback. Promote, demote, or **unadjudicated**.
 3. **Budget** — ranker + **mechanism memory** (JSONL + OpenRouter embeddings, hard CWE/language/tag filters) **before** prove. Not a verdict. Not TF-IDF. Not raw file-slice embeddings.
 4. **Prove** — sandbox only on promoted, reachable, budgeted candidates.
 The primary in-scope target is a **messy working tree the operator is writing**, not a labeled benchmark. Known-vuln corpora remain calibration scoreboards (smoke 90/10, pair Youden, GitHub silent-on-fix). They do not define success on unlabeled code.
 ## Scope
 - Semantic overlay as MAP-stage adjudication; **tree-sitter is the prio engine** (CLI/API, fail-closed). Joern optional. Embeddings tune prove order.
 - Explicit `unadjudicated` when parse or flow fails — do not treat silence as safe.
 - Sandbox prove only after promote.
 - Pair eval scores the overlay on the sast slice; smoke gate stays inventory-only.
 ## Out of scope
 - Replacing pattern rules.
 - LLM as parse/taint engine.
 - Requiring Joern or embeddings for `quick`.
 - Using embedding similarity to silently drop promotions.
 - Auto-writing flow facts from the improve loop.
@@ -0,0 +1,349 @@
 # Design Document
 ## Overview
 **Purpose.** This feature makes the three-stage scan *mean* three jobs: pattern inventory **proposes** sinks; a flow overlay **adjudicates** those proposals on real files; the sandbox **proves** only what was promoted. Operators scanning a working tree they are writing get honest `unadjudicated` instead of fake cleanliness.
 **Users.** Developers get `quick` as cheap inventory on WIP code. AppSec gets `standard` dispositions (promote / demote / unadjudicated / coverage). DevOps gets `deep` budget spent on promoted reachable sinks.
 **Impact.** Today `quick_scan_findings` is the de facto decision. After this spec, inventory is necessary but not sufficient. MAP writes overlay artifacts. REGRESS candidate selection reads promotions, not demotions.
 ### Goals
 - Map every current scan module onto propose, adjudicate, budget, or prove — rank/embed is not a verdict.
 - Treat messy unlabeled trees as the default; labeled corpora as calibration.
 - Fail closed: cannot parse or cannot follow flow → `unadjudicated`, not demote.
 - **Prioritize the semantic engine now**: tree-sitter IR for in-scope languages; Joern optional CPG sidecar; stdlib Python parse only as last resort.
 - Tunable embeddings + existing ranker order **prove** candidates.
 - SAST pair slice scores the overlay; smoke gate stays inventory-only.
 ### Non-Goals
 - Replacing pattern rules or `LANGUAGE_MANIFESTS`.
 - LLM-as-adjudicator or LLM-as-parser.
 - Requiring Joern or an embedding API for `quick`.
 - Using retrieval similarity to drop promotions.
 - Auto-authoring source/sink/sanitizer facts from `ousast improve`.
 - Buffer-size / memory-safety semantics (Juliet CWE-121 goodG2B stays a known leak until a later spec).
 ## Boundary Commitments
 ### This Spec Owns
 - Overlay dispositions and flow-fact records.
 - Source / sink / sanitizer **facts as data** (not CWE severity).
 - Tree-sitter probe/runner (CLI-first, like Docker) and last-resort Python stdlib parse.
 - Optional Joern probe/runner (CLI sidecar).
 - Intra-file taint on the tree-sitter IR.
 - Prove-budget: heuristic rank + OpenRouter embeddings over the mechanism log.
 - Calibration scoring of overlay on the sast pair slice; unlabeled-scan honesty.
 ### Out of Boundary
 - Pattern text, rule `enabled|shadow|disabled`, CWE TSV, project score formula (`three-stage-scan` / ruleset / policy).
 - Complexity-map identity keys, Docker sandbox flags, hunter suspicion contract (`three-stage-scan`).
 - Pair catalog schema except scoring **which** findings count (overlay vs inventory).
 - External analyzer execution (SARIF ingest remains a proposal input).
 - MCP exposing `deep`.
 ### Allowed Dependencies
 - Preprocess, mapping, rank (`heuristic_rank` 0.5/0.2/0.3), `quick_scan_findings`, complexity map, `ousast index` / json-local vectorstore, tool hunter, sandbox runner, pair eval, reports.
 - Host binaries when present: `tree-sitter`, `joern` / `joern-parse` (probed, never required for inventory).
 - Optional embedding model from existing `[embeddings]` config.
 - Clearwing as an **oracle** for tree-sitter callgraph + taint-pattern tables + mechanism retrieval — not a dependency.
 ### Revalidation Triggers
 - Adding a fourth disposition that implies “safe.”
 - Demoting when flow is incomplete.
 - Stamping overlay above `static_corroboration`.
 - Selecting sandbox candidates from demoted or unadjudicated sets.
 - Demoting because retrieval score is low.
 - Folding overlay Youden into `python -m openultrasast.gate`.
 - Making tree-sitter or Joern a hard dependency of `quick`.
 ## Architecture
 ### Existing Architecture Analysis
 Concrete mapping of **what exists today** onto the three roles:
 | Role | Stage / mode | Module today | What it actually does | After this spec |
 |------|----------------|--------------|------------------------|-----------------|
 | Propose | STATIC / `quick` | `findings.quick_scan_findings` | Pattern rules decide the report | **Proposal list only** |
 | Propose | STATIC | `mapping.ingest_sarif` | Operator SARIF | Still proposals |
 | Propose | STATIC | `mapping.analyze_entry_points` | Routes / CLI | **Sources** for overlay |
 | Budget | STATIC | `rank.heuristic_rank` 0.5/0.2/0.3 | File budget | Same formula; later **prove order** |
 | Budget | `ousast index` | `vectorstore` json-local | Hunter retrieval | **Prove retrieval** on promotions |
 | Adjudicate? | MAP | `complexity.map` | Hotspot band | Attention only |
 | Adjudicate | MAP | **new** tree-sitter IR + taint | — | **Primary semantic engine** |
 | Adjudicate | MAP | **new** Joern CLI sidecar | — | Optional CPG facts; never required |
 | Adjudicate fallback | MAP | stdlib `ast` | Routes only today | Python last resort if tree-sitter missing |
 | Suspicion | MAP | `tool_hunter` | LLM + grep | May query overlay; never adjudicate |
 | Prove | REGRESS / `deep` | `regress` + sandbox | Recipes on ranked findings | Recipes on **promoted then budgeted** findings |
 | Calibrate | CI | `gate.py` / `pairs` | Cheat-sheet vs Youden | Inventory vs overlay, unchanged split |
 **Selected pattern.** Propose → adjudicate → **budget** → prove. Clearwing’s tree-sitter callgraph + taint tables + mechanism memory are the oracle, not a library import. Embeddings never demote.
 ```mermaid
 flowchart TB
  subgraph propose["PROPOSE inventory"]
    PRE[preprocess]
    EP[entry points as sources]
    RX[pattern inventory]
    SARIF[optional SARIF]
  end
  subgraph adjudicate["ADJUDICATE overlay"]
    TS[tree-sitter IR]
    JOERN[optional Joern CPG]
    FALLBACK[Python stdlib parse last resort]
    FACTS[source sink sanitizer facts]
    TAINT[intra-file taint]
    DISP[promote demote unadjudicated coverage]
  end
  subgraph budget["BUDGET not a verdict"]
    RANK[heuristic and optional model ranker]
    VS[json-local embeddings]
  end
  subgraph prove["PROVE sandbox"]
    CAND[promoted reachable then budgeted]
    BOX[existing Docker runner]
    VERD[triggerable or not]
  end
  PRE --> EP
  PRE --> RX
  RX --> DISP
  SARIF --> DISP
  EP --> TAINT
  TS --> TAINT
  JOERN --> TAINT
  FALLBACK --> TAINT
  FACTS --> TAINT
  TAINT --> DISP
  DISP --> RANK
  DISP --> VS
  RANK --> CAND
  VS --> CAND
  CAND --> BOX
  BOX --> VERD
 ```
 ### Technology Stack
 | Layer | Choice | Role |
 |-------|--------|------|
 | CLI / modes | existing `quick` `standard` `deep` | Role gating |
 | Propose | pattern rules + SARIF | Inventory |
 | Semantic IR | `tree-sitter` CLI or API extra; probe like Docker | Primary adjudicator parse |
 | Semantic CPG | Joern CLI when present | Optional inter-file facts |
 | Fallback parse | stdlib `ast` | Python only, last resort |
 | Facts | TOML next to the ruleset | Sources sinks sanitizers |
 | Budget | existing ranker + json-local vectorstore | Prove order, tunable |
 | Prove | existing Docker CLI sandbox | Unchanged runner |
 `quick` stays zero-dep. `standard`/`deep` degrade to unadjudicated when parsers are missing.
 ## File Structure Plan
 ```
 src/openultrasast/semantic/
  __init__.py
  facts.py              # source sink sanitizer facts
  engines.py            # probe tree-sitter, joern, ast fallback
  ir.py                 # FileIR from the first engine that works
  taint.py              # worklist; incomplete -> Unadjudicated
  overlay.py            # dispositions
  prove_budget.py       # rank + optional retrieval over promotions
 src/openultrasast/ruleset/semantic/
  python.toml javascript.toml c.toml java.toml
 ```
 ### Modified Files
 - `stages.py` / `cli.py` — overlay in MAP; prove-budget before regress.
 - `findings.py` — inventory unchanged.
 - `regress/candidate.py` — `promote` only, then budget order.
 - `rank.py` / `index.py` / `vectorstore.py` — reuse; do not invent a second store.
 - `reports.py` — disposition columns.
 - `pairs.py` — sast slice uses overlay.
 - `mcp.py` — still no `deep`.
 ## System Flows
 ```mermaid
 stateDiagram-v2
  [*] --> Proposed
  Proposed --> Unadjudicated: parse fail or flow incomplete or language unsupported
  Proposed --> Promoted: source reaches sink without sanitizer
  Proposed --> Demoted: constant or sanitizer dominates
  Proposed --> Coverage: flow to uninventoried sink
  Promoted --> Proven: sandbox triggerable
  Promoted --> Unproven: sandbox skip or unavailable
  Unadjudicated --> [*]
  Demoted --> [*]
  Coverage --> Promoted: same flow rule
 ```
 **Key decisions**
 - Demote only with a positive reason (constant or sanitizer on that path).
 - Unadjudicated is the default for messy real files.
 - Coverage findings enter the same state machine as proposals.
 ### Unlabeled working-tree flow
 No expected.toml. Success is: every inventory item has a disposition; unadjudicated reasons are explicit; `deep` does not execute demoted sinks; `--fail-on worth-fixing` ignores unadjudicated.
 ## Requirements Traceability
 | Requirement | Summary | Components | Flows |
 |-------------|---------|------------|-------|
 | 1.1–1.5 | Role split by mode | Stage plan, overlay, prove budget, regress | propose → adjudicate → budget → prove |
 | 8.1–8.6 | Semantic engines | engines.py, ir.py, taint.py | tree-sitter then Joern then ast |
 | 9.1–9.5 | Prove budget | prove_budget.py, rank, vectorstore | retrieval never demotes |
 | 2.1–2.5 | Messy trees, unadjudicated | IR fail, taint incomplete | unlabeled scan |
 | 3.1–3.5 | Promote demote coverage | taint, overlay | state diagram |
 | 4.1–4.5 | Calibration vs product | gate.py, pairs.py | sast overlay score |
 | 5.1–5.5 | Evidence honesty | overlay evidence, hunter | ladder |
 | 6.1–6.5 | Prove promoted only | candidate filter, sandbox | deep |
 | 7.1–7.5 | Artifacts and fail-on | reports, overlay.json | operator |
 ## Components and Interfaces
 | Component | Domain | Intent | Req Coverage | Dependencies | Contracts |
 |-----------|--------|--------|--------------|--------------|-----------|
 | InventoryProposer | Propose | Existing pattern + SARIF list | 1.1 | findings, mapping | State |
 | FactStore | Adjudicate | Sources sinks sanitizers as data | 3, 5.5 | ruleset/semantic | State |
 | FileIR | Adjudicate | Parse one file or fail | 2.1 | ast | Service |
 | TaintEngine | Adjudicate | Intra-file paths or incomplete | 2.2, 3 | FileIR, FactStore | Service |
 | Overlay | Adjudicate | Dispositions on proposals | 1.2, 3, 5 | TaintEngine | State |
 | ProveFilter | Prove | Candidates from promote | 1.3, 6 | Overlay, regress | Service |
 | OverlayPairEval | Calibrate | SAST slice uses dispositions | 4 | Overlay, pairs | Batch |
 ### Adjudicate layer
 #### OverlayRecord
 | Field | Detail |
 |-------|--------|
 | Intent | One disposition per proposal or coverage item |
 | Requirements | 3.5, 7.1 |
 **State contract**
 ```
 proposal_id: str
 path: str
 line: int | None
 disposition: promote | demote | unadjudicated | coverage
 reason: str
 cwe: str
 sources: list[str]
 sinks: list[str]
 sanitizers: list[str]
 evidence_level: static_corroboration | suspicion
 ```
 **Invariants**
 - `demote` requires a recorded constant or sanitizer.
 - `unadjudicated` requires a reason in {`parse_failed`, `language_unsupported`, `flow_incomplete`, `facts_unavailable`}.
 - `promote` and `coverage` do not exceed `static_corroboration`.
 #### TaintEngine service
 ```
 analyze_file(path, text, language, facts) -> FileFlow | Unadjudicated
 adjudicate(proposals, flows, facts) -> list[OverlayRecord]
 ```
 - Engine order: tree-sitter grammar → Joern file export if probed and useful → Python stdlib parse → unadjudicated.
 - Joern may add paths; it must not demote a tree-sitter promote, and a Joern miss must not demote.
 - Postconditions: every inventory proposal appears exactly once; extras are `coverage` only.
 ### Prove layer
 #### ProveFilter
 Inbound: overlay + reachability from mapping. Outbound: existing `regress` candidates.
 - Drop `demote`.
 - Do not sandbox `unadjudicated` as a negative proof.
 - Keep `promote` that are reachable or inferred-file-surface.
 - Order survivors with heuristic rank, then **mechanism recall**: hard filter (language, CWE, tags) then OpenRouter dense cosine on the mechanism text. Low similarity never drops a promotion.
 - Write mechanisms only after prove/triggerable. JSONL log is authoritative; json-local `mechanisms` namespace is a rebuildable embedding cache via `OpenRouterEmbeddingClient`.
 ## Data Models
 **Facts (data, versioned with the ruleset)**
 - `source`: identifier (e.g. request parameter, header, stdin, argv)
 - `sink`: call shape + CWE (aligned to existing rule ids when possible)
 - `sanitizer`: call shape that stops taint (parameterized execute, `printf` with literal format, `safe_load`, list-form subprocess)
 **FileFlow**
 - assignments and call arguments in one function
 - one-hop name copies (`bar = param`, `sql = f"...{bar}"`)
 - no types, no inter-file, no may-alias heap in v1
 **Mechanism (Clearwing reference architecture)**
 Identity is the *pattern*, not a file slice:
 | Field | Role |
 |-------|------|
 | id | stable uuid |
 | summary | one sentence, no path/line |
 | cwe, language | filters |
 | tags, keywords | hard filters, not a ranker |
 | what_made_it_exploitable | why a near-miss became a bug |
 | source_finding_id, source_repo | provenance only |
 Persistence: append-only JSONL (portable). Recall is **not** Clearwing’s TF-IDF ladder.
 1. **Filter** — language, CWE, tags (exact overlap). This is a hard prefilter, not a ranker.  
 2. **Rank** — OpenRouter `/embeddings` (`OpenRouterEmbeddingClient`, existing `[embeddings].model`) over `summary + what_made_it_exploitable + tags`. Cosine against the promotion’s overlay text. Cache vectors in json-local `mechanisms` namespace, keyed by mechanism id + model id.  
 3. **Degrade** — no key / no model / embed failure → heuristic ranker only, degradation recorded. Never bag-of-words cosine.
 Query for a promotion: embed the overlay rationale (sources, sink, CWE). Result is a **boost to prove order**, never a demote.
 Repo-code chunks (`ousast index` 80-line slices) stay for hunter/verifier retrieval. They are **not** the prove-budget store. Textbook fixtures must not fill the mechanism log unless they were actually proven in the sandbox.
 ## Error Handling
 | Condition | Disposition | Operator sees |
 |-----------|-------------|---------------|
 | Syntax error / parse fail | unadjudicated parse_failed | proposal kept, marked unresolved |
 | No tree-sitter grammar and not Python | unadjudicated language_unsupported | same |
 | Joern missing | ignore sidecar | standard still overlays via tree-sitter |
 | Embedding model missing | heuristic prove order | deep still runs |
 | Callee in another file | unadjudicated flow_incomplete | same |
 | Fact file missing | unadjudicated facts_unavailable; `quick` still emits inventory | degradation in manifest |
 | Sandbox missing | promoted remains unproven | `sandbox_unavailable` |
 ## Testing Strategy
 - Unit: parse fail → unadjudicated; constant `exec("ls")` → demote; `eval(request.args)` → promote; `sql = f"…{x}"; execute(sql)` → coverage or promote.
 - Pair: `--slice sast` uses overlay; `juliet-c-cwe134-printf` stays PASS; `owasp-python-deser` stays PASS; Juliet `goodG2B` exec/strcpy must **demote or unadjudicated**, not inventory-leak counted as pair fail for overlay.
 - Unlabeled: tmp tree with a syntax-error file next to a clean `eval(request.args)` — error file unadjudicated, clean file promoted; no expected.toml required.
 - Gate: `python -m openultrasast.gate` still inventory-only.
 - Deep: candidate list excludes demoted.
 ## Security
 - Overlay does not execute target code.
 - Facts cannot override CWE severity.
 - Hunter still cannot stamp corroboration.
 ## Performance
 - Overlay is per-file, MAP stage, same file budget as rank.
 - Skip files already unreadable at preprocess.
 ## Migration
 - Default `standard` runs overlay when facts load; if facts missing, degrade as Requirement 7.5.
 - Existing `quick` CI unchanged.
 - Pair gate local 100% remains inventory-or-overlay-equivalent on fixture counterparts (those files are still “nice”; unlabeled tests are additional).
@@ -0,0 +1,127 @@
 # Requirements Document
 ## Introduction
 OpenUltraSAST users scan **working trees they are writing**: mixed files, incomplete edits, generated code, framework wrappers, and no labels. Today the harness still *decides* with pattern inventory. The complexity map ranks; the sandbox can run; neither adjudicates “this sink is fed by a source” versus “this sink is hardcoded, sanitized, or unreadable.”
 This feature splits the scan into three operator-visible roles that must not collapse: **regex proposes, semantics adjudicates, sandbox proves.** Labeled suites (cheat-sheet fixtures, OWASP Benchmark, Juliet, CVE pairs) remain **calibration**. They are not the definition of in-scope code.
 ## Boundary Context
 - **In scope**: Role split on `quick` / `standard` / `deep`; overlay dispositions; tree-sitter (and optional Joern) as the semantic engine; tunable retrieval/rank **before prove**; `unadjudicated` on messy files; sandbox only after promote; calibration vs unlabeled working trees.
 - **Out of scope**: Deleting pattern rules; making an LLM the adjudicator; rewriting CWE policy or project-score formula; requiring Joern or an embedding model for `quick`; auto-generated flow facts from the improve loop; treating benchmark Youden as a merge gate that can unblock a smoke-gate breach. Embedding rank is a **budget**, not a fourth verdict.
 - **Adjacent expectations**: Stage plan, complexity map, Docker sandbox, pair catalog, and 90/10 detection gate from `three-stage-scan` keep their contracts. This spec overlays adjudication on that pipeline; it does not replace stage accounting, sandbox flags, or `LANGUAGE_MANIFESTS`.
 ## Requirements
 ### Requirement 1: Three roles stay distinct
 **Objective:** As an operator, I want every finding to show whether it is a proposal, an adjudication, or a proof, so that a sink hit is never mistaken for a worth-fixing verdict.
 #### Acceptance Criteria
 1. When OpenUltraSAST completes a `quick` scan, the system shall emit inventory proposals only and shall not emit adjudication dispositions or sandbox verdicts.
 2. When OpenUltraSAST completes a `standard` scan, the system shall attach an adjudication disposition to each inventory proposal it considered and shall not execute target-derived code to obtain that disposition.
 3. When OpenUltraSAST completes a `deep` scan, the system shall offer sandbox proof only for candidates that adjudication promoted and that remain reachable or inferred-file-surface, ordered by the tunable rank/retrieval budget.
 4. The system shall not treat an inventory proposal as worth-fixing.
 5. The system shall not treat an adjudication promote as crash-reproduced, exploit-demonstrated, or patch-validated.
 ### Requirement 2: Messy working trees are the primary target
 **Objective:** As a developer writing ordinary code, I want the harness to degrade honestly on files that are not labeled benchmarks, so that silence is not reported as safety.
 #### Acceptance Criteria
 1. When a scanned file is incomplete, generated, minified, or otherwise not interpretable as structured source, the system shall leave matching inventory proposals **unadjudicated** and shall record that reason on the finding.
 2. When no available semantic engine can follow data from a source to a sink for that file (parse failure, missing grammar, cross-file call, dynamic dispatch, or missing callee), the system shall leave the proposal unadjudicated rather than demote it.
 3. When OpenUltraSAST scans a working tree with no benchmark manifest and no expected-finding labels, the system shall still emit proposals, dispositions, and (in `deep`) proofs without requiring a counterpart “fixed” tree.
 4. If the operator reads a report for an unadjudicated proposal, then the system shall present it as unresolved inventory, not as a verified vulnerability and not as a cleared finding.
 5. The system shall not require a file to resemble an OWASP Benchmark or Juliet test case in order to produce a proposal.
 ### Requirement 3: Adjudication promote, demote, coverage
 **Objective:** As an AppSec reviewer, I want the overlay to keep, drop, or add findings based on flow, so that hardcoded and sanitized sinks stop looking like attacks.
 #### Acceptance Criteria
 1. When a proposal’s sink is reachable from an identified untrusted source without an identified sanitizer in the adjudicated region, the system shall **promote** the proposal.
 2. When a proposal’s sink is fed only by a constant, a sanitizer, or an identified non-attacker source in the adjudicated region, the system shall **demote** the proposal and shall not list it as a promoted finding.
 3. When adjudication finds an untrusted source flowing to a dangerous sink that inventory did not propose, the system shall emit a **coverage** finding at corroboration-or-lower evidence, tagged as overlay-originated.
 4. When both a source-to-sink path and a sanitizer appear to apply, the system shall not promote unless the sanitizer does not dominate that path.
 5. The system shall persist promote, demote, unadjudicated, and coverage outcomes in the scan artifacts with the proposal identifier they refer to.
 ### Requirement 4: Calibration corpora do not define product success
 **Objective:** As a maintainer, I want labeled suites to measure the overlay without becoming the only in-scope code, so that we do not overfit textbook cases.
 #### Acceptance Criteria
 1. When the stage-1 detection gate runs, the system shall score inventory only on the existing cheat-sheet corpus and shall not fold overlay Youden or unlabeled-scan counts into that gate.
 2. When the pair-efficiency evaluation runs on the SAST slice, the system shall score **adjudicated** outcomes (promote on the vulnerable side, demote or silent on the false or good side), not raw inventory hits.
 3. When overlay changes raise SAST Youden and also cause the stage-1 detection gate to fail, the system shall treat the detection-gate failure as blocking.
 4. The system shall keep labeled pair fixtures and SAST-slice cases out of the stage-1 cheat-sheet corpus map.
 5. The system shall not require a labeled counterpart tree in order to scan an operator working tree.
 ### Requirement 5: Evidence and models stay honest
 **Objective:** As a reviewer, I want overlay and hunter output to stay on the evidence ladder, so that a model cannot mint proof.
 #### Acceptance Criteria
 1. The system shall keep inventory and successful intra-file adjudication at `static_corroboration` or below.
 2. When the tool hunter emits a finding, the system shall keep that finding at `suspicion` unless an independent corroborating stage has already raised it.
 3. If an overlay or hunter result claims an evidence level above corroboration without a sandbox or verifier step that earned it, then the system shall reject that claim.
 4. The system shall not use a language model as the adjudication engine that decides promote or demote.
 5. Where CWE policy applies, the system shall resolve severity from the central CWE policy, not from the overlay or the proposing rule’s private severity string.
 ### Requirement 6: Sandbox proves only promoted work
 **Objective:** As a DevOps engineer, I want isolation to spend budget on promoted sinks, so that `deep` does not execute every regex hit.
 #### Acceptance Criteria
 1. When `deep` runs and the sandbox is available, the system shall select regression candidates from promoted findings, not from demoted findings.
 2. When a finding is unadjudicated, the system shall not treat a sandbox skip as proof that the finding is safe.
 3. If the sandbox is unavailable, then the system shall keep promoted findings as unproven corroboration, record `sandbox_unavailable`, and shall not invent a worth-fixing verdict.
 4. When sandbox proof is `triggerable` on a promoted reachable finding, the system shall allow `worth_fixing` to follow the existing worth-fixing rule from the three-stage scan.
 5. The system shall not execute target-derived code outside the existing isolated sandbox path.
 ### Requirement 7: Operator-visible artifacts and fail-on
 **Objective:** As an operator, I want the report to show the three-role pipeline on real trees, so that I can act on promoted and worth-fixing items without trusting unlabeled silence.
 #### Acceptance Criteria
 1. When a `standard` or `deep` scan finishes, the system shall write overlay artifacts that list each considered proposal, its disposition, and any flow facts or unadjudicated reason.
 2. When the markdown or SARIF report is written, the system shall distinguish promoted, demoted, unadjudicated, and coverage items.
 3. Where `--fail-on worth-fixing` is used, the system shall fail only on worth-fixing as already defined, not on unadjudicated inventory.
 4. The system shall keep `quick` usable without overlay configuration and without a model.
 5. If overlay fact data is missing or invalid, then the system shall degrade to inventory-only for that scan, mark proposals unadjudicated with that reason, and shall not abort `quick`.
 ### Requirement 8: Semantic engines are multi-language and optional-at-runtime
 **Objective:** As an AppSec engineer scanning mixed-language trees, I want adjudication to use a real parser family, so that Python-only stdlib parsing is not the product engine.
 #### Acceptance Criteria
 1. When a tree-sitter grammar for the file’s language is available, the system shall use that parse as the primary intra-file IR for adjudication.
 2. When tree-sitter is unavailable for that language, the system shall try a stdlib parse only for Python and shall otherwise leave the proposal unadjudicated with language_unsupported or parse_failed.
 3. Where Joern is installed and the language is one Joern can export, the system shall be allowed to attach Joern flow facts as **additional** corroboration; Joern absence shall not fail `quick` or `standard`.
 4. If Joern and tree-sitter disagree or Joern returns incomplete paths, then the system shall not demote on that disagreement; it shall promote only on a complete dominating path or leave the item unadjudicated.
 5. The system shall not require Joern or tree-sitter to be present for a `quick` inventory scan.
 6. The system shall not use a language model as the parse or taint engine.
 ### Requirement 9: Mechanism memory ranks prove budget, not truth
 **Objective:** As an operator on a large messy tree, I want proven-bug *mechanisms* (not raw file slices) to decide **what to prove next**, so that the sandbox follows patterns that have worked on real bugs.
 #### Acceptance Criteria
 1. When `deep` has more promoted candidates than the sandbox budget, the system shall order them using the existing ranker axes plus OpenRouter embedding similarity over mechanism records, after hard filters on language, CWE, and tags.
 2. When no embedding model or API key is configured, the system shall order candidates with the heuristic ranker only, record that degradation, and shall not skip prove.
 3. The system shall not demote or drop a promotion solely because mechanism similarity is low.
 4. When the embedding index is missing or stale, the system shall rebuild it from the mechanism log via OpenRouter embeddings; if that rebuild fails, the system shall record the degradation and continue on heuristic order.
 5. The system shall keep hard filters operator-tunable (language, CWE, tags) without changing CWE severity policy.
 6. When a finding is sandbox-proven triggerable (or otherwise verified at corroboration-or-higher with a prove step), the system shall append an abstract mechanism record to the mechanism log: summary, CWE, language, tags, what made it exploitable — without file:line as the identity.
 7. The system shall treat the append-only mechanism log as the source of truth; any vector index shall be a rebuildable cache of that log.
 8. The system shall not persist mechanisms from inventory-only, demoted, or unadjudicated proposals.
@@ -0,0 +1,105 @@
 # Research & Design Decisions
 ## Summary
 - **Feature**: `propose-adjudicate-prove`
 - **Discovery Scope**: Extension of `three-stage-scan` (MAP grows an overlay; STATIC and REGRESS stay)
 - **Key Findings**:
  - Inventory already decides; MAP ranks; REGRESS can run. Nothing adjudicates flow.
  - Labeled SAST slice Youden is about +9%. Leaks are OWASP-false / Juliet `goodG2B` that still contain the sink. Misses are one-hop assignments and sink aliases.
  - Clearwing already does the split we want: tree-sitter callgraph + pattern taint tables, then ranker (surface/influence/reachability 0.5/0.2/0.3), then mechanism-memory embeddings. OpenUltraSAST already copied the rank formula and has `json-local` vectors; they are not wired as prove-budget.
 - Tree-sitter CLI/API covers C/C++/Python/JS/Go/Rust (Clearwing grammars); Joern is the CPG sidecar when present. Stdlib `ast` is last-resort Python only. LLM hunter remains suspicion-only.
  - Real working trees will often fail parse or cross a file boundary. Those cases must stay **unadjudicated**, not demoted.
 ## Research Log
 ### Labeled vs unlabeled code
 - **Context**: Operator: known-vuln corpora are “nice”; written code will not be.
 - **Sources Consulted**: In-tree `LANGUAGE_MANIFESTS`, `benchmarks/pairs/`, OWASP Benchmark true/false files, Juliet `bad`/`goodG2B`/`goodB2G`, CWE-Bench-Java catalog.
 - **Findings**:
  - Cheat-sheets: one CWE, named sink, often same-line source.
  - OWASP: true and false files both often contain `exec` / `eval`; the difference is source constancy or infeasible branches.
  - Juliet `goodG2B`: hardcoded source, **same sink**.
  - Juliet `goodB2G`: same source, **sanitized sink** (`printf("%s")`).
  - Working trees: syntax errors, generated bundles, wrappers (`helpers.db_sqlite.execute`), no labels.
 - **Implications**: Overlay scored on sast slice; unlabeled scans must not pretend demote. Unadjudicated is a first-class disposition.
 ### Engine choice
 - **Context**: Operator prefers tree-sitter + Joern look, embeddings before prove, still v1-focused.
 - **Sources Consulted**: `clearwing/sourcehunt/callgraph.py`, `taint.py`, `ranker.py`, `mechanism_memory.py`; `openultrasast/rank.py`, `vectorstore.py`, `index.py`.
 - **Findings**:
  - Clearwing tree-sitter is optional: missing grammar disables callgraph, does not crash.
  - Clearwing taint is intra-procedural, pattern tables for sources/sinks — same shape as our fact TOML.
  - Clearwing ranker fills reachability from the callgraph; taint_paths boost surface. Embeddings retrieve *mechanisms*, they do not decide taint.
  - Our `heuristic_rank` already uses surface·0.5 + influence·0.2 + reachability·0.3. Vector store bakeoff selected `json-local`. `VALID_NAMESPACES` already includes `mechanisms` but nothing writes Clearwing-style records into it.
 ### Clearwing vector-store reference (real-world bugs)
 - **Context**: Operator: Clearwing’s store is effective on real bugs, not textbook files.
 - **Sources Consulted**: `clearwing/sourcehunt/mechanism_memory.py`, `data/memory/semantic_memory.py`, `docs/architecture.md` steps 7–8; `openultrasast/index.py` namespaces.
 - **Findings**:
  - Store **mechanisms**, not findings and not 80-line file windows. Example: “length field trusted before alloc; 16-bit value widened to size_t.”
  - JSONL is the source of truth (`~/.clearwing/sourcehunt/mechanisms.jsonl`). Chroma/sqlite embeddings are ephemeral caches rebuilt from JSONL.
  - Write path is **verified findings only** (after adversarial verify / crash). Suspicion never seeds the store.
  - Clearwing’s JSONL + verified-write + metadata filters are the reference. Its TF-IDF default is **not**: bag-of-words will not match paraphrased real bugs (`length field trusted before alloc` vs `user-controlled size_t in malloc`).
  - OpenUltraSAST already has `OpenRouterEmbeddingClient` and json-local cosine. Use that as the only semantic ranker; CWE/language/tags are hard filters. No TF-IDF, no Chroma, no local MiniLM.
  - Query is language + file tags + optional free text; same-language boost +0.5. Top-N injects hunter context and variant-grep seeds — it does not decide taint.
  - Variant loop then greps the *messy* tree for structural matches of that mechanism. That is how it leaves Juliet behind.
 - **Implications**: Copy JSONL + verified-write + metadata filters. Rank with OpenRouter embeddings only. Do not copy TF-IDF. Do not embed 80-line file windows as the prove store. Seed from sandbox-proven mechanisms, not OWASP `BenchmarkTest00007`.
  - Joern CLI (`joern-parse` / scripts) is the only practical CPG without a Scala runtime in-process.
 - **Implications**: Probe tree-sitter like Docker. Joern sidecar cannot demote. Embeddings order prove only.
 ### Decision: Python intra-file first
 - **Context**: Zero-dep core vs multi-language trees.
 - **Selected**: **Superseded.** Tree-sitter is the prio engine; stdlib `ast` is Python last resort; `quick` stays zero-dep.
 - **Rationale**: Operator working trees are mixed-language; Python-only adjudication would leave JS/C/Java as perpetual unadjudicated.
 - **Trade-offs**: `standard` needs a grammar or CLI on PATH; missing tools → unadjudicated, not skip the scan.
 ### Evidence ladder
 - **Context**: Three-stage-scan forbids hunter JSON as corroboration.
 - **Sources Consulted**: `verification.EvidenceLevel`, `tool_hunter` suspicion rule, `worth_fixing` gate.
 - **Findings**: Overlay can justify promote at `static_corroboration`. It cannot skip to crash or exploit. Worth-fixing stays sandbox + reachability.
 - **Implications**: Disposition is not an evidence rung. Do not add a seventh ladder value.
 ## Architecture Pattern Evaluation
 | Option | Description | Strengths | Risks | Notes |
 |--------|-------------|-----------|-------|-------|
 | Replace regex with CodeQL | External engine is the detector | Real dataflow | Breaks zero-dep, smoke gate, ledger shadows | Rejected |
 | LLM hunter as adjudicator | Model reads file and votes | Flexible on messy code | Unstable Youden; violates suspicion rule | Rejected |
 | Overlay on inventory | Regex proposes; tree-sitter taint adjudicates; embed ranks prove | Fits stages; multi-lang | Parser must be probed | **Selected** |
 | Whole-program taint v1 | Inter-file | Closes wrappers | Too large; false demote risk | Defer; stay unadjudicated |
 ## Design Decisions
 ### Decision: Unadjudicated is not demote
 - **Context**: Messy files and cross-file flow.
 - **Alternatives**: Silent drop; treat as safe; force demote.
 - **Selected**: Explicit `unadjudicated` with reason.
 - **Rationale**: Operator writing code must not read silence as “clean.”
 - **Trade-offs**: Reports stay noisy until languages and inter-file land.
 - **Follow-up**: Count unadjudicated rate on unlabeled trees as an honesty metric.
 ### Decision: Score overlay on sast slice, inventory on smoke gate
 - **Context**: +9% Youden vs 93% cheat-sheet recall.
 - **Selected**: Two gates stay separate; overlay Youden cannot unblock smoke failure.
 - **Rationale**: Same as detection-gate vs score in the improve loop.
 ## Risks & Mitigations
 - False demote on real bugs (infeasible-branch OWASP tricks, wrappers) — never demote when flow is incomplete; only demote when the engine *has* a dominating constant or sanitizer.
 - Youden chase via extra sinks — each coverage rule needs a false/good counterpart that must stay demoted or silent.
 - Parse failures on operator WIP — unadjudicated, `quick` still works.
 ## References
 - `.kiro/specs/three-stage-scan/design.md` — stage plan, sandbox, hunter suspicion
 - `benchmarks/pairs/sast/README.md` — OWASP/Juliet slice
 - NIST Juliet `goodG2B` vs `goodB2G` templates
 - OWASP Benchmark expectedresults true/false protocol
@@ -0,0 +1,22 @@
 {
  "feature_name": "propose-adjudicate-prove",
  "created_at": "2026-09-04T10:58:28Z",
  "updated_at": "2026-09-04T12:00:00Z",
  "language": "en",
  "phase": "implementation",
  "approvals": {
    "requirements": {
      "generated": true,
      "approved": true
    },
    "design": {
      "generated": true,
      "approved": true
    },
    "tasks": {
      "generated": true,
      "approved": true
    }
  },
  "ready_for_implementation": true
 }
@@ -0,0 +1,142 @@
 # Implementation Plan
 - [x] 1. Foundation: overlay vocabulary and fact data
 - [x] 1.1 Define overlay records and fail-closed reasons
  - Dispositions are promote, demote, unadjudicated, coverage only.
  - Unadjudicated reasons are parse_failed, language_unsupported, flow_incomplete, facts_unavailable.
  - Demote is illegal without a recorded constant or sanitizer in the record.
  - Tests construct records and reject demote-without-reason.
  - Observable: a unit test fails if a demote record omits the dominating fact.
  - _Requirements: 3.5, 2.1, 2.2, 5.1_
  - _Boundary: OverlayRecord_
 - [x] 1.2 (P) Load source, sink, and sanitizer facts as versioned data
  - Facts bind to CWE ids that already exist in policy.
  - Invalid or missing fact files yield facts_unavailable, not a crash of `quick`.
  - Observable: loading a broken fact file returns a structured error the overlay can attach to every proposal.
  - _Requirements: 5.5, 7.5_
  - _Boundary: FactStore_
 - [x] 2. Semantic engines: tree-sitter first
 - [x] 2.1 Probe tree-sitter like Docker and parse in-scope languages
  - Primary IR comes from tree-sitter when a grammar exists (Python, JS, C/C++, Java if present).
  - Missing CLI or grammar does not crash `quick`; those files become unadjudicated.
  - Stdlib Python parse is last resort only when tree-sitter is absent.
  - Observable: a JS `eval(req.query.x)` file yields IR when tree-sitter JS is available, and language_unsupported when it is not.
  - _Requirements: 8.1, 8.2, 8.5, 8.6, 2.1_
  - _Boundary: FileIR_
  - _Depends: 1.1_
 - [x] 2.2 (P) Probe Joern as an optional CPG sidecar
  - Joern facts may add paths; Joern missing or incomplete must not demote.
  - `quick` and `standard` succeed without Joern.
  - Observable: with Joern absent, overlay still runs; a Joern-only miss does not turn a tree-sitter promote into demote.
  - _Requirements: 8.3, 8.4, 8.5_
  - _Boundary: JoernSidecar_
 - [x] 2.3 Follow one-hop assignments and call arguments on the IR
  - Cover `bar = param`, f-string or concat into `sql`, then `execute(sql)`, in Python and the equivalent JS/C patterns the facts describe.
  - Incomplete callees mark flow_incomplete, not safe.
  - Observable: split-sink Python fixture yields a source-to-sink path; a call into a missing module does not.
  - _Requirements: 2.2, 3.1, 3.4, 2.5_
  - _Boundary: TaintEngine_
  - _Depends: 2.1, 1.2_
 - [x] 3. Adjudicate inventory proposals
 - [x] 3.1 Promote, demote, and leave unadjudicated
  - Every inventory proposal for a considered file appears exactly once.
  - `eval(request.args)` promotes; `eval("1")` demotes; unreadable file stays unadjudicated.
  - Observable: overlay output length equals proposal count for that file, plus coverage items only.
  - _Requirements: 1.2, 3.1, 3.2, 2.4, 3.5_
  - _Boundary: Overlay_
  - _Depends: 2.3_
 - [x] 3.2 Emit coverage findings for uninventoried sinks with flow
  - Coverage stays at corroboration or below and is tagged overlay-originated.
  - No coverage item is stamped as worth-fixing or sandbox-proven.
  - Observable: `hashlib.new('md5')` with tainted input appears as coverage when facts include that sink and inventory missed it.
  - _Requirements: 3.3, 1.4, 1.5, 5.1_
  - _Boundary: Overlay_
 - [x] 4. Integration: stages, prove filter, reports
 - [x] 4.1 Run overlay in standard, not in quick
  - `quick` artifacts contain no dispositions.
  - `standard` attaches a disposition to each considered proposal without executing target code.
  - Observable: a `quick` run on a tiny tree has no overlay file; `standard` writes overlay artifacts.
  - _Requirements: 1.1, 1.2, 7.1, 7.4_
  - _Boundary: StagePlan, Overlay_
  - _Depends: 3.1_
 - [x] 4.2 Select deep candidates from promotions only
  - Demoted findings are absent from the sandbox candidate list.
  - Unadjudicated findings are not used as negative proof when the sandbox is skipped.
  - Sandbox unavailable keeps promoted items unproven and records sandbox_unavailable.
  - Order remaining promotions with the heuristic ranker; if embeddings are configured, retrieval may reorder but must not drop.
  - Observable: a tree with one demoted `exec("ls")` and one promoted `eval(request.args)` offers only the latter to regression when Docker is probed; with embeddings off, heuristic order is used.
  - _Requirements: 1.3, 6.1, 6.2, 6.3, 6.4, 6.5, 9.1, 9.2, 9.3, 9.4, 9.5_
  - _Boundary: ProveFilter_
  - _Depends: 4.1_
 - [x] 4.3 Distinguish dispositions in markdown and SARIF
  - Promoted, demoted, unadjudicated, and coverage are visible without reading JSON.
  - `--fail-on worth-fixing` does not fail on unadjudicated inventory.
  - Observable: report.md contains the four labels; fail-on worth-fixing on an unadjudicated-only tree exits 0.
  - _Requirements: 7.2, 7.3, 2.4_
  - _Boundary: Reports_
  - _Depends: 4.1_
 - [x] 4.4 Mechanism memory with OpenRouter embeddings as the only semantic ranker
  - Append-only mechanism log is source of truth; json-local `mechanisms` namespace is a rebuildable OpenRouter vector cache.
  - Hard-filter by language, CWE, tags; rank remaining by OpenRouter cosine on mechanism text vs promotion overlay text.
  - No TF-IDF. Missing key or embed failure → heuristic order plus a recorded degradation; prove still runs.
  - Write mechanisms only from sandbox-proven findings; never from unadjudicated inventory.
  - Observable: after one triggerable finding, a paraphrased second promotion ranks above an unrelated CWE; with OPENROUTER_API_KEY unset, both remain candidates in heuristic order.
  - _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8_
  - _Boundary: ProveBudget_
  - _Depends: 4.2_
 - [x] 5. Evidence honesty
 - [x] 5.1 Keep hunter and overlay off the proof rungs
  - Overlay promote remains static_corroboration or below.
  - Hunter output remains suspicion unless an independent stage already raised it.
  - A fixture that tries to stamp overlay as exploit_demonstrated is rejected.
  - Observable: existing hunter suspicion tests still pass; overlay records never contain crash_reproduced or higher.
  - _Requirements: 5.1, 5.2, 5.3, 5.4_
  - _Boundary: Overlay, ToolHunter_
  - _Depends: 3.1_
 - [x] 6. Calibration vs unlabeled product
 - [x] 6.1 Score the sast pair slice on overlay outcomes
  - Vulnerable side counts promote; false or good side counts demote or silent promoted-empty.
  - Stage-1 detection gate still uses inventory on cheat-sheet manifests only.
  - Overlay Youden cannot mark a failed detection gate as passed.
  - Sast and pair fixtures remain outside LANGUAGE_MANIFESTS.
  - Observable: `ousast pairs --slice sast` changes when overlay demotes Juliet goodG2B exec; `python -m openultrasast.gate` still ignores that slice.
  - _Requirements: 4.1, 4.2, 4.3, 4.4_
  - _Boundary: OverlayPairEval_
  - _Depends: 3.1, 4.1_
 - [x] 6.2 Scan an unlabeled messy tree without expected findings
  - A syntax-broken file next to a clear request-to-eval file: broken file unadjudicated, eval promoted.
  - No counterpart fixed tree and no manifest required.
  - Observable: `ousast scan` on that tmp tree writes both dispositions.
  - _Requirements: 2.3, 2.5, 4.5, 7.4_
  - _Boundary: Overlay_
  - _Depends: 4.1_
 - [x] 7. End-to-end validation
 - [x] 7.1 Regression of three-stage contracts
  - Quick still has no model and no target execution.
  - Deep still does not run outside the existing sandbox.
  - MCP still rejects deep.
  - Observable: existing three-stage-scan tests for modes, sandbox, and MCP remain green.
  - _Requirements: 1.1, 6.5, 7.4_
  - _Depends: 4.2, 5.1_
 ## Implementation Notes
 - Python adjudication uses stdlib `ast` when tree-sitter has no grammar; JS/C/Java stay `language_unsupported` until a grammar actually parses.
 - Pair eval on the sast slice uses overlay when at least one promote/demote/coverage exists; otherwise it falls back to inventory so C/Java Juliet cases do not collapse without grammars.
 - Intra-file taint is one-hop names plus source patterns in expressions. Heap channels (ConfigParser get/set) stay `flow_incomplete`.
 - OpenRouter embeddings reorder promotions only; missing key records `embeddings_unavailable` and prove still runs.
 - Hunter findings stay off the sandbox candidate list; only overlay promotions are proven.
@@ -30,6 +30,11 @@ uv run ousast index /path/to/repo
 # Score the detector against a benchmark manifest
 uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
 # Differential pair eval: fire on vuln, stay silent on the fix (local + real GitHub VFCs)
 uv run ousast pairs
 uv run ousast pairs --slice sast   # OWASP Benchmark Java/Python + Juliet
 uv run python -m openultrasast.pair_gate
 # Run the bounded self-improvement loop over the ruleset (benchmark-driven)
 uv run ousast improve benchmarks/manifests/python-vulnerable.toml --dry-run
 ```
@@ -75,6 +80,15 @@ removes cross-language false positives):
 | JavaScript / TS | command injection (78), code injection (95), reflected/DOM XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) |
 | Java + Groovy templates | command injection (78), SQLi via concatenation (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
 Cheat-sheet fixtures live under `benchmarks/fixtures/` and feed the 90/10 smoke
 gate. **Efficiency** is measured separately on isolated vuln-vs-fixed pairs
 (`ousast pairs`, `python -m openultrasast.pair_gate`): the harness must fire on
 the vulnerable snapshot and stay silent after the patch. Local fixture
 counterparts are a CI gate; vendored GitHub VFCs (SVEN, CWE-Bench-Java, NVD
 patterns) are the honesty dashboard and the miss/fp signals for `ousast improve`.
 Dataset pointers (CVEfixes, DiverseVul, PrimeVul, MegaVul, MoreFixes, Vul4J,
 GitHub Advisory Database, OSV) are in `benchmarks/pairs/datasets.toml`.
 ## Project scores and central CWE policy
 Severity used to be whatever string a rule set on itself. It is now governed centrally: one CWE, one severity, decided by policy — not by the rule that fired.
@@ -402,5 +416,6 @@ uv run ruff check .           # lint
 uv run ruff format --check .  # format
 uv run mypy src/openultrasast # types
 uv run python -m openultrasast.gate  # standalone detection gate
 uv run python -m openultrasast.pair_gate  # local vuln-vs-fix efficiency + GitHub honesty
 uv run python dagger/ci.py    # full containerized CI pipeline
 ```
@@ -0,0 +1,51 @@
 # Vuln vs fixed pair corpus
 This corpus measures harness **efficiency**, not cheat-sheet recall.
 The stage-1 smoke gate (`python -m openultrasast.gate`) scans one tree that
 already contains the planted sinks. A pair eval instead scans **two isolated
 trees**:
 1. **vuln** — only the vulnerable file. The harness should fire (true positive).
 2. **fixed** — only the patched file, same relative path. The harness should stay
   silent on that CWE (true negative).
 `pair_correct` = detected on vuln **and** silent on the fix. That is the signal
 the improvement loop consumes (`miss` on the vuln side, `fp` on the fix side).
 ## Layout
 | Path | Role |
 | --- | --- |
 | `catalog.toml` | Runnable offline pairs (local fixtures + vendored GitHub functions) |
 | `datasets.toml` | Pointers to public VFC datasets — do not vendor the dumps |
 | `datasets/cwe-bench-java-project_info.csv` | 120 vetted Java CVE buggy/fix commits |
 | `github/` | Isolated function excerpts with provenance headers |
 Pairs in this directory **must not** join `LANGUAGE_MANIFESTS`.
 ## Slices
 - **local** — in-tree fixture counterparts (`app.py` vs `safe.py`, …). CI gate:
  every local pair must pass (`python -m openultrasast.pair_gate`).
 - **github** — real commit/function pairs (SVEN, CWE-Bench-Java, NVD patterns).
  Measured, not a 90% smoke gate. Misses here are the honesty / evolve-the-harness
  dashboard.
 - **sast** — OWASP Benchmark Java/Python true-vs-false files and Juliet
  bad-vs-good sinks (`benchmarks/pairs/sast/`). Score is Youden (TPR − FPR).
  Honesty dashboard, not a CI fail.
 ## Commands
 ```bash
 uv run ousast pairs
 uv run ousast pairs --slice github --json
 uv run python -m openultrasast.pair_gate
 ```
 ## Growing the corpus
 Prefer function-level before/after from a reviewed source (SVEN jsonl, a
 CWE-Bench-Java `fix_info.csv` row, a GHSA with a tiny patch). Isolate the
 changed function into `vuln` / `fixed` files with the same `relpath`. Leave
 whole-repo clones to `--fetch` later; CI stays offline.
@@ -0,0 +1,181 @@
 # Vuln-vs-fixed pair catalog. Isolated trees: the scanner sees only the vuln
 # file or only the fixed file, never both. Do not add these names to
 # LANGUAGE_MANIFESTS (stage-1 90% smoke gate stays on cheat-sheet fixtures).
 [[pair]]
 name = "local-python-vulnerable"
 slice = "local"
 language = "python"
 origin = "in-tree-fixture"
 license = "in-tree"
 vuln = "../fixtures/python-vulnerable/app.py"
 fixed = "../fixtures/python-vulnerable/safe.py"
 relpath = "app.py"
 expected_from = "../manifests/python-vulnerable.toml"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair]]
 name = "local-javascript-vulnerable"
 slice = "local"
 language = "javascript"
 origin = "in-tree-fixture"
 license = "in-tree"
 vuln = "../fixtures/javascript-vulnerable/src/app.js"
 fixed = "../fixtures/javascript-vulnerable/src/safe.js"
 relpath = "src/app.js"
 expected_from = "../manifests/javascript-vulnerable.toml"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair]]
 name = "local-cpp-buffer"
 slice = "local"
 language = "c_cpp"
 origin = "in-tree-fixture"
 license = "in-tree"
 vuln = "../fixtures/cpp-damn-vulnerable/src/buffer.cpp"
 fixed = "../fixtures/cpp-damn-vulnerable/src/safe.cpp"
 relpath = "src/buffer.cpp"
 expected_from = "../manifests/cpp-damn-vulnerable.toml"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair]]
 name = "sven-cwe-078-os-system"
 slice = "github"
 language = "python"
 origin = "sven"
 license = "original-project"
 repo = "Internet-of-People/titania-os"
 commit = "9b7805119938343fcac9dc929d8882f1d97cf14a"
 commit_url = "https://github.com/Internet-of-People/titania-os/commit/9b7805119938343fcac9dc929d8882f1d97cf14a"
 vuln = "github/sven-cwe-078-os-system/vuln.py"
 fixed = "github/sven-cwe-078-os-system/fixed.py"
 relpath = "vuedj/configtitania/views.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-78"
 class = "command injection"
 path = "vuedj/configtitania/views.py"
 rule_id = "python-os-command"
 sink = "os.system"
 evidence = "os.system concatenates attacker-controlled username into a shell command"
 [[pair]]
 name = "sven-cwe-078-shell-true"
 slice = "github"
 language = "python"
 origin = "sven"
 license = "Apache-2.0"
 repo = "google/mobly"
 commit = "3862e8ba359040fbdd6e1a6d36e51d07cda8e1ee"
 commit_url = "https://github.com/google/mobly/commit/3862e8ba359040fbdd6e1a6d36e51d07cda8e1ee"
 vuln = "github/sven-cwe-078-shell-true/vuln.py"
 fixed = "github/sven-cwe-078-shell-true/fixed.py"
 relpath = "mobly/controllers/android_device_lib/adb.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-78"
 class = "command injection"
 path = "mobly/controllers/android_device_lib/adb.py"
 rule_id = "python-shell-true"
 sink = "subprocess"
 evidence = "subprocess.Popen runs the command with shell=True"
 [[pair]]
 name = "sven-cwe-089-execute-percent"
 slice = "github"
 language = "python"
 origin = "sven"
 license = "original-project"
 repo = "russ-lewis/ttt_-_python_cgi"
 commit = "6096f43fd4b2d91211eec4614b7960c0816900da"
 commit_url = "https://github.com/russ-lewis/ttt_-_python_cgi/commit/6096f43fd4b2d91211eec4614b7960c0816900da"
 vuln = "github/sven-cwe-089-execute-percent/vuln.py"
 fixed = "github/sven-cwe-089-execute-percent/fixed.py"
 relpath = "cgi/common.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-89"
 class = "sql injection"
 path = "cgi/common.py"
 rule_id = "python-sql-injection"
 sink = "execute"
 evidence = "execute percent-formats attacker-controlled input into SQL"
 [[pair]]
 name = "sven-cwe-089-query-concat"
 slice = "github"
 language = "python"
 origin = "sven"
 license = "original-project"
 repo = "pukkapies/urop2019"
 commit = "3ca2e2c291d2d5fe262d20a8e0520bdfb622432b"
 commit_url = "https://github.com/pukkapies/urop2019/commit/3ca2e2c291d2d5fe262d20a8e0520bdfb622432b"
 vuln = "github/sven-cwe-089-query-concat/vuln.py"
 fixed = "github/sven-cwe-089-query-concat/fixed.py"
 relpath = "modules/query_lastfm.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-89"
 class = "sql injection"
 path = "modules/query_lastfm.py"
 sink = "SELECT"
 evidence = "SQL query concatenates attacker-controlled tag_num"
 [[pair]]
 name = "hutool-cve-2018-17297"
 slice = "github"
 language = "java"
 origin = "cwe-bench-java"
 license = "MulanPSL-2.0"
 repo = "dromara/hutool"
 cve = "CVE-2018-17297"
 cwe = "CWE-22"
 commit = "8d7d0b7fb5ea4f7447b40131bffc1ec506a6528e"
 parent = "fd77bb8718648f69a587046b735488a80123cc26"
 commit_url = "https://github.com/dromara/hutool/commit/8d7d0b7fb5ea4f7447b40131bffc1ec506a6528e"
 vuln = "github/hutool-cve-2018-17297/vuln/ZipUtil.java"
 fixed = "github/hutool-cve-2018-17297/fixed/ZipUtil.java"
 relpath = "cn/hutool/core/util/ZipUtil.java"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-22"
 class = "path traversal"
 path = "cn/hutool/core/util/ZipUtil.java"
 sink = "ZipEntry"
 evidence = "zip entry name is joined onto the extract directory without a zip-slip check"
 [[pair]]
 name = "node-serialize-cve-2017-5941"
 slice = "github"
 language = "javascript"
 origin = "nvd"
 license = "public-advisory-pattern"
 cve = "CVE-2017-5941"
 cwe = "CWE-95"
 commit_url = "https://nvd.nist.gov/vuln/detail/CVE-2017-5941"
 vuln = "github/node-serialize-cve-2017-5941/vuln.js"
 fixed = "github/node-serialize-cve-2017-5941/fixed.js"
 relpath = "index.js"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-95"
 class = "code injection"
 path = "index.js"
 rule_id = "js-eval"
 sink = "eval"
 evidence = "eval executes attacker-controlled serialized input"
@@ -0,0 +1,124 @@
 # External vulnerability-fixing-commit (VFC) datasets. Metadata only — these
 # dumps are too large / too noisy to vendor. Use them to grow the pair catalog.
 # Label accuracy on auto-mined function datasets is often well below 100%
 # (DiverseVul sampled ~60% correct "vulnerable" labels). Prefer manually
 # vetted sets (CWE-Bench-Java, Vul4J, SVEN) when evolving the harness.
 [[dataset]]
 name = "OWASP Benchmark Java"
 url = "https://github.com/OWASP-Benchmark/BenchmarkJava"
 data = "benchmarks/pairs/sast/owasp-java"
 level = "file"
 languages = ["Java"]
 notes = "Official SAST/DAST/IAST scorecard. 2740 labeled true/false cases, 11 CWEs. Score = TPR - FPR (Youden). Vendored slice under sast/owasp-java."
 [[dataset]]
 name = "OWASP Benchmark Python"
 url = "https://github.com/OWASP-Benchmark/BenchmarkPython"
 data = "benchmarks/pairs/sast/owasp-python"
 level = "file"
 languages = ["Python"]
 notes = "Python counterpart of OWASP Benchmark (expectedresults-0.1.csv, Flask testcode/). Same true/false scoring protocol. Vendored slice under sast/owasp-python."
 [[dataset]]
 name = "Juliet Test Suite C/C++"
 url = "https://github.com/arichardson/juliet-test-suite-c"
 data = "benchmarks/pairs/sast/juliet-c"
 level = "function"
 languages = ["C/C++"]
 notes = "NIST SARD Juliet 1.3. Each case has bad() and goodG2B/goodB2G. CC0. The arichardson mirror is a memory-safety subset; CWE-78 lives in the full NIST zip / Juliet Java."
 [[dataset]]
 name = "Juliet Test Suite Java"
 url = "https://github.com/UnitTestBot/juliet-java-test-suite"
 data = "benchmarks/pairs/sast/juliet-java"
 level = "function"
 languages = ["Java"]
 notes = "NIST Juliet Java 1.3, 112 CWEs including CWE-78 and CWE-89. CC0. Vendored slice extracts bad vs goodG2B sinks."
 [[dataset]]
 name = "CVEfixes"
 url = "https://github.com/secureIT-project/CVEfixes"
 data = "https://doi.org/10.5281/zenodo.4476563"
 level = "commit/file/function"
 languages = ["multi"]
 notes = "NVD-linked CVE fix commits. v1.0.8: 11873 CVEs, 12107 commits, 4249 projects. CC BY 4.0 data."
 [[dataset]]
 name = "MoreFixes"
 url = "https://github.com/secureIT-project/MoreFixes"
 level = "commit/file/method"
 languages = ["C/C++", "Java", "Python", "JavaScript"]
 notes = "Largest language-independent CVE-fix set (~26617 CVEs, ~31883 commits). Method-level CSVs also on Kaggle."
 [[dataset]]
 name = "DiverseVul"
 url = "https://github.com/wagner-group/diversevul"
 level = "function"
 languages = ["C/C++"]
 notes = "18945 vuln functions / 150 CWEs. Sampled label accuracy ~60%. Do not treat as ground truth without review."
 [[dataset]]
 name = "PrimeVul"
 url = "https://github.com/DLVulDet/PrimeVul"
 level = "function"
 languages = ["C/C++"]
 notes = "Relabeled DiverseVul/CVEFixes-style pairs with stricter commit-function alignment."
 [[dataset]]
 name = "MegaVul"
 url = "https://github.com/Icyrockton/MegaVul"
 level = "function"
 languages = ["C/C++"]
 notes = "Large C/C++ function pair set (vuln vs after-fix)."
 [[dataset]]
 name = "CWE-Bench-Java"
 url = "https://github.com/iris-sast/cwe-bench-java"
 data = "benchmarks/pairs/datasets/cwe-bench-java-project_info.csv"
 level = "project/commit"
 languages = ["Java"]
 notes = "120 manually vetted CVEs (CWE-22/78/79/94) with buggy_commit_id and fix_commit_ids. IRIS paper."
 [[dataset]]
 name = "Vul4J"
 url = "https://github.com/tuhh-softsec/vul4j"
 level = "project"
 languages = ["Java"]
 notes = "Reproducible Java vulns with human patches and PoV tests. CC-BY-4.0 dataset."
 [[dataset]]
 name = "SVEN"
 url = "https://github.com/eth-sri/sven"
 data = "https://github.com/eth-sri/sven/tree/master/data_train_val"
 level = "function"
 languages = ["Python", "C/C++"]
 notes = "Manually curated before/after functions (CWE-22/78/79/89/...). Train jsonl is the pair source for several vendored cases."
 [[dataset]]
 name = "vfc_datasets"
 url = "https://github.com/UzL-ITS/vfc_datasets"
 level = "toolkit"
 languages = ["multi"]
 notes = "Unified loader over 22 commit-level and 8 function-level VFC datasets (Devign, BigVul, CrossVul, PatchDB, CleanVul, SecVulEval, ...)."
 [[dataset]]
 name = "GitHub Advisory Database"
 url = "https://github.com/github/advisory-database"
 level = "advisory"
 languages = ["multi"]
 notes = "GHSA JSON with affected packages; pair with OSV/NVD for fix commits. Not code itself."
 [[dataset]]
 name = "OSV"
 url = "https://github.com/google/osv.dev"
 level = "advisory"
 languages = ["multi"]
 notes = "Cross-ecosystem advisories with introduced/fixed commit ranges when present."
 [[dataset]]
 name = "security-patches-dataset"
 url = "https://github.com/tqrg/security-patches-dataset"
 level = "commit"
 languages = ["multi"]
 notes = "Ground-truth security patches mined from NVD, CVE Details, OSV."
@@ -0,0 +1,121 @@
 id,project_slug,cve_id,cwe_id,cwe_name,github_username,github_repository_name,github_tag,github_url,advisory_id,buggy_commit_id,fix_commit_ids
 1,DSpace__DSpace_CVE-2016-10726_4.4,CVE-2016-10726,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),DSpace,DSpace,4.4,https://github.com/DSpace/DSpace,GHSA-4m9r-5gqp-7j82,ca4c86b1baa4e0b07975b1da86a34a6e7170b3b7,4239abd2dd2ae0dedd7edc95a5c9f264fdcf639d
 2,perwendel__spark_CVE-2018-9159_2.7.1,CVE-2018-9159,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),perwendel,spark,2.7.1,https://github.com/perwendel/spark,GHSA-76qr-mmh8-cp8f,5316c0d0f057daaf556c3907c20df975f7bf8a8a,030e9d00125cbd1ad759668f85488aba1019c668;a221a864db28eb736d36041df2fa6eb8839fc5cd;ce9e11517eca69e58ed4378d1e47a02bd06863cc
 3,perwendel__spark_CVE-2016-9177_2.5.1,CVE-2016-9177,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),perwendel,spark,2.5.1,https://github.com/perwendel/spark,GHSA-89gc-6cw6-4vch,ce57dfb949ed183405f75dee8cf3262b45c9b3b5,efcb46c710e3f56805b9257a63d1306882f4faf9
 4,wildfly__wildfly_CVE-2018-1047_11.0.0.Final,CVE-2018-1047,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),wildfly,wildfly,11.0.0.Final,https://github.com/wildfly/wildfly,GHSA-fmr4-w67p-vh8x,4fd7bffaf2ee73201910684f2674aa1bced7fe81,735c77ce8b3bab6aa5072fb8575dc96db52d01ab;21c358acf0b06f38ec46034404c7a3d0bbbc132d;9ff55ed2da845a6f604b2edd1bc3bd311f1b776c
 5,vert-x3__vertx-web_CVE-2018-12542_3.5.3.CR1,CVE-2018-12542,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),vert-x3,vertx-web,3.5.3.CR1,https://github.com/vert-x3/vertx-web,GHSA-h39x-m55c-v55h,3fe3873cc28b4d833c90f8989ff797a7b463da2c,57a65dce6f4c5aa5e3ce7288685e7f3447eb8f3b
 6,apache__camel_CVE-2018-8041_2.20.3,CVE-2018-8041,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,camel,2.20.3,https://github.com/apache/camel,GHSA-jv74-f9pj-xp3f,32bfda73ddd1ea8576bcb53dac496af9e0825f1a,4580e4d6c65cfd544c1791c824b5819477c583cc;4f401c09d22c45c94fa97746dc31905e06b19e3;63c7c080de4d18f9ceb25843508710df2c2c6d4;a0d25d9582c6ee85e9567fa39413df0b4f02ef7
 7,dromara__hutool_CVE-2018-17297_4.1.11,CVE-2018-17297,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),dromara,hutool,4.1.11,https://github.com/dromara/hutool,GHSA-rhq2-2574-78mc,7687720c5125b29386d3bb9c7c2931da79664b73,8d7d0b7fb5ea4f7447b40131bffc1ec506a6528e
 8,apache__tika_CVE-2018-11762_1.18,CVE-2018-11762,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,tika,1.18,https://github.com/apache/tika,GHSA-w6g3-v46q-5p28,38ff2a986af24ee255f1f91d654ea402f4016696,a09d853dbed712f644e274b497cce254f3189d57;5c78eb78893fa077a4e58be4484eab4717667d52
 9,square__retrofit_CVE-2018-1000850_2.4.0,CVE-2018-1000850,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),square,retrofit,2.4.0,https://github.com/square/retrofit,GHSA-8p8g-f9vg-r7xr,7158698314daa138e993fac6a590ed19d78a8599,b9a7f6ad72073ddd40254c0058710e87a073047d
 10,apache__jspwiki_CVE-2019-0225_2.11.0.M2,CVE-2019-0225,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,jspwiki,2.11.0.M2,https://github.com/apache/jspwiki,GHSA-pffw-p2q5-w6vh,d48bcef107a14dd07ecd830ab817b8df796082a4,88d89d6523802c044cfcb7930cba40d8eeb21da2
 11,apache__camel_CVE-2019-0194_2.21.4,CVE-2019-0194,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,camel,2.21.4,https://github.com/apache/camel,GHSA-4wjq-69rc-8wcp,757bc836d1d9ed11d11de73c5dc72fc53617d6a0,5b64969d37cf2906efd4623cfd473041ce5132fd
 12,asf__tapestry-5_CVE-2019-0207_5.4.4,CVE-2019-0207,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,tapestry-5,5.4.4,https://github.com/apache/tapestry-5,GHSA-89r3-rcpj-h7w6,d29cc6778eb0c75e45139274005bba1c4f58907f,6dd7219fd4707467a36091e2f7f5f6460c619099
 13,spring-cloud__spring-cloud-config_CVE-2020-5410_2.1.8.RELEASE,CVE-2020-5410,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),spring-cloud,spring-cloud-config,2.1.8.RELEASE,https://github.com/spring-cloud/spring-cloud-config,GHSA-32xf-jwmv-9hf3,91843a9ddc7489275b679568bfce1e4c37af71e1,03eaadc25323d8c8c89f528e8585033ed90c50f6
 14,spring-cloud__spring-cloud-config_CVE-2020-5405_2.1.6.RELEASE,CVE-2020-5405,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),spring-cloud,spring-cloud-config,2.1.6.RELEASE,https://github.com/spring-cloud/spring-cloud-config,GHSA-g86w-v5vg-9gxf,24e7292e5930367396461354d3f9d8ba8b94771c,651f458919c40ef9a5e93e7d76bf98575910fad0
 15,apache__rocketmq_CVE-2019-17572_4.6.0,CVE-2019-17572,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,rocketmq,4.6.0,https://github.com/apache/rocketmq,GHSA-5x3v-2gxr-59m2,d1b4e47c26576cf4fa543bbdd8c6f09643220bed,f8f6fbe4aa7f5dee937e688322628c366b12a552
 16,joniles__mpxj_CVE-2020-35460_8.3.4,CVE-2020-35460,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),joniles,mpxj,8.3.4,https://github.com/joniles/mpxj,GHSA-p9j6-4pjr-gp48,87ea0ba33e901b87c95a18e3703ac51b59e2ee04,8eaf4225048ea5ba7e59ef4556dab2098fcc4a1d
 17,apache__flink_CVE-2020-17519_1.11.2,CVE-2020-17519,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,flink,1.11.2,https://github.com/apache/flink,GHSA-395w-qhqr-9fr6,fe3613574f76201a8d55d572a639a4ce7e18a9db,b561010b0ee741543c3953306037f00d7a9f0801
 18,kubernetes-client__java_CVE-2020-8570_client-java-parent-9.0.1,CVE-2020-8570,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),kubernetes-client,java,client-java-parent-9.0.1,https://github.com/kubernetes-client/java,GHSA-cghx-9gcr-r42x,796c74f4244cacd90d99393640ccaad0c72f8893,858316ae8bc1145005a0310e1f65f95d2389a589
 19,asf__commons-io_CVE-2021-29425_2.6,CVE-2021-29425,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,commons-io,2.6,https://github.com/apache/commons-io,GHSA-gwrp-pvrq-jmwv,2ae025fe5c4a7d2046c53072b0898e37a079fe62,fe7543eee5cd4b2f9e78aa44c31031b68eba204d
 20,asf__karaf_CVE-2022-22932_4.3.5,CVE-2022-22932,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,karaf,4.3.5,https://github.com/apache/karaf,GHSA-544x-2jx9-4pfg,d0439e5b0dde1c7e3ff0207c557aa701093acd25,36a2bc4
 21,asf__james-project_CVE-2022-22931_3.6.0,CVE-2022-22931,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,james-project,3.6.0,https://github.com/apache/james-project,GHSA-v84g-cf5j-xjqx,34f9b0b52ab8c84378d3a34ac43a0fef5aa4e8fe,8f9512637d374ccdf84d61617a2878557bf07532
 22,vert-x3__vertx-web_CVE-2019-17640_3.9.3,CVE-2019-17640,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),vert-x3,vertx-web,3.9.3,https://github.com/vert-x3/vertx-web,GHSA-vjw7-6gfq-6wf5,2146b7240096e25b40bb1acc083fa7ec79330989,b12f7c4c6576a8adc3f683b2899de4b0e7d099e4
 23,ESAPI__esapi-java-legacy_CVE-2022-23457_2.2.3.1,CVE-2022-23457,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),ESAPI,esapi-java-legacy,2.2.3.1,https://github.com/ESAPI/esapi-java-legacy,GHSA-8m5h-hrqm-pxm2,2e8694c6beb3bdbb2645b882eba72ce41bc63242,a0d67b75593878b1b6e39e2acc1773b3effedb2a
 24,xwiki__xwiki-commons_CVE-2022-24897_12.6.6,CVE-2022-24897,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),xwiki,xwiki-commons,12.6.6,https://github.com/xwiki/xwiki-commons,GHSA-cvx5-m8vg-vxgc,4f33e1efd74016fa9eb31b3c2fcd01bc6becd551,215951cfb0f808d0bf5b1097c9e7d1e503449ab8
 25,srikanth-lingala__zip4j_CVE-2018-1002202_1.3.2,CVE-2018-1002202,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),srikanth-lingala,zip4j,1.3.2,https://github.com/iris-sast/zip4j,GHSA-2rpm-4x8c-pvqg,d87ffa2d64ffb3a0a1cf0c7a69c7b19d7015bfde,59aeea594ff885e06c25751dc6334dfea1bed8c6
 26,alibaba__one-java-agent_CVE-2022-25842_0.0.1,CVE-2022-25842,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),alibaba,one-java-agent,0.0.1,https://github.com/alibaba/one-java-agent,GHSA-9hr3-j9mc-xmq2,b911af2bb779917c84a016350700745dc31c642a,359603b63fc6c59d8b57e061c171954bab3433bf;e0149bb36488fede2a12cb1ab3c2095e56dd7571
 27,apache__myfaces_CVE-2011-4367_2.0.11,CVE-2011-4367,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,myfaces,2.0.11,https://github.com/apache/myfaces,GHSA-gjfx-9wx3-j6r7,bedac49daffc3cb792fa19884620af01ab5a01ab,236cd211327c557825c7fce091bcc68f57a88e8e;99ef1250ca3e28213e85eac14817e1ff93fc9e6a;c07c3bf93ad0e0cf530ddece0642783f1553ad89;9e7e996a4ef83c4d8e673ee6e4431348e598e031;b9b2e00bb53d3eb435256c5a140b83f7d6251b55
 28,undertow-io__undertow_CVE-2014-7816_1.0.16.Final,CVE-2014-7816,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),undertow-io,undertow,1.0.16.Final,https://github.com/undertow-io/undertow,GHSA-h6p6-fc4w-cqhx,4aa98c80fa9962235c94f8666c30793d0018f95d,28f244e63f558ba99a197813cfd5eee461b52b4c
 29,jeremylong__DependencyCheck_CVE-2018-12036_3.1.2,CVE-2018-12036,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),jeremylong,DependencyCheck,3.1.2,https://github.com/jeremylong/DependencyCheck,GHSA-hcwx-7q5v-vc67,c3b5b3c0d7fe7283b4cd421149d6fb620476f67d,c106ca919aa343b95cca0ffff0a0b5dc20b2baf7
 30,codehaus-plexus__plexus-archiver_CVE-2018-1002200_3.5,CVE-2018-1002200,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),codehaus-plexus,plexus-archiver,3.5,https://github.com/codehaus-plexus/plexus-archiver,GHSA-hcxq-x77q-3469,b9f9a425865eb47fb3665b3144ee4ca11f402704,f8f4233508193b70df33759ae9dc6154d69c2ea8
 31,zeroturnaround__zt-zip_CVE-2018-1002201_1.12,CVE-2018-1002201,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),zeroturnaround,zt-zip,1.12,https://github.com/zeroturnaround/zt-zip,GHSA-qcf3-9vmh-xw4r,bd3f3d77f3709f901fd5180a493ef32e4dfebfa6,759b72f33bc8f4d69f84f09fcb7f010ad45d6fff
 32,whitesource__curekit_CVE-2022-23082_1.1.3,CVE-2022-23082,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),whitesource,CureKit,1.1.3,https://github.com/whitesource/CureKit,GHSA-m9vj-44f3-78xw,7b275a67a5992165deb186b2b3f7764ddd062d26,af35e870ed09411d2f1fae6db1b04598cd1a31b6
 33,aws__aws-sdk-java_CVE-2022-31159_1.12.260,CVE-2022-31159,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),aws,aws-sdk-java,1.12.260,https://github.com/aws/aws-sdk-java,GHSA-c28r-hw5m-5gv3,5555d847a882d4c3094b00ac7cec21a16cf01a3e,cb66c50c885566d9a7fce837a68dc48ddced8a9a
 34,jlangch__venice_CVE-2022-36007_1.10.16,CVE-2022-36007,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),jlangch,venice,1.10.16,https://github.com/jlangch/venice,GHSA-4mmh-5vw7-rgvj,9306c93686100a6bc52ad0c8cb617ed199b5f75c,215ae91bb964013b0a2d70718a692832d561ae0a;c942c73136333bc493050910f171a48e6f575b23
 35,DSpace__DSpace_CVE-2022-31195_5.10,CVE-2022-31195,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),DSpace,DSpace,5.10,https://github.com/DSpace/DSpace,GHSA-8rmh-55h4-93h5,eca7968be7d6b9f8f5f302c9fc09f8186ed4809e,56e76049185bbd87c994128a9d77735ad7af0199;7af52a0883a9dbc475cf3001f04ed11b24c8a4c0
 36,payara__Payara_CVE-2022-37422_5.2022.2,CVE-2022-37422,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),payara,Payara,5.2022.2,https://github.com/payara/Payara,GHSA-h28c-453m-h9xm,7680990e55c04d628f16a591f950a2b3a4dcb401,6490d4918c65ba6df8344424cb9b8ab583d90b1a
 37,DSpace__DSpace_CVE-2022-31194_5.10,CVE-2022-31194,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),DSpace,DSpace,5.10,https://github.com/DSpace/DSpace,GHSA-qp5m-c3m9-8q2p,eca7968be7d6b9f8f5f302c9fc09f8186ed4809e,7569c6374aefeafb996e202cf8d631020eda5f24;d1dd7d23329ef055069759df15cfa200c8e3
 38,diffplug__goomph_CVE-2022-26049_3.37.1,CVE-2022-26049,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),diffplug,goomph,3.37.1,https://github.com/diffplug/goomph,GHSA-p2f7-9cv7-jjf6,fe2083196f0d0a75885aea402baaa972254173d5,25f04f67ba62d9a14104bee13a0a0f2517afb8c8
 39,apache__dolphinscheduler_CVE-2022-26884_2.0.5,CVE-2022-26884,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,dolphinscheduler,2.0.5,https://github.com/apache/dolphinscheduler,GHSA-vpgf-fgm8-gxr2,f31328276c12821c21d1ce07e824bc74146c19b2,9717da6c255414eaef1c9ac943b0eb8989c1a85d
 40,apache__dolphinscheduler_CVE-2022-34662_2.0.9,CVE-2022-34662,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,dolphinscheduler,2.0.9,https://github.com/apache/dolphinscheduler,GHSA-fp35-xrrr-3gph,23302054559410093573fb169336bafeb02db97f,63f835715f8ca8bff79c0e7177ebfa5917ebb3bd
 41,testng-team__testng_CVE-2022-4065_7.5,CVE-2022-4065,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),cbeust,testng,7.5,https://github.com/cbeust/testng,GHSA-rc2q-x9mf-w3vf,d8adcef70a9e0b8fcef5eab92ca87a8d398cfb89,9150736cd2c123a6a3b60e6193630859f9f0422b
 42,apache__uima-uimaj_CVE-2022-32287_3.3.0,CVE-2022-32287,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,uima-uimaj,3.3.0,https://github.com/apache/uima-uimaj,GHSA-xgqr-5wqw-9fpv,f1f557a415ae38e383bf050587a510211515c21d,1088f615154b385dd163965b8d2ef78e87111ab1
 43,keycloak__keycloak_CVE-2022-3782_20.0.1,CVE-2022-3782,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),keycloak,keycloak,20.0.1,https://github.com/keycloak/keycloak,GHSA-g8q8-fggx-9r3q,cce8ee2b1ab7c925bfbb62d8f12a33e93d943040,da8d61616399d5c4e4c470c16dd6b0bd98f2b656
 44,eclipse-ee4j__glassfish_CVE-2022-2712_6.2.5,CVE-2022-2712,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),eclipse-ee4j,glassfish,6.2.5,https://github.com/eclipse-ee4j/glassfish,GHSA-3g5w-6pw7-6hrp,0159b68b362c7f4be78d1fa75aeaf2ec0b997f1d,6982a7ca7e7fb9f02276dc12ab61832daf4002bd
 45,hapifhir__org.hl7.fhir.core_CVE-2023-24057_5.6.91,CVE-2023-24057,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),hapifhir,org.hl7.fhir.core,5.6.91,https://github.com/hapifhir/org.hl7.fhir.core,GHSA-jqh6-9574-5x22,7b22d0355b1b12448e50b9cbfe2a8ae63f2ee828,b50aec59124416b7315a49220cfc3999223414cc
 46,hapifhir__org.hl7.fhir.core_CVE-2023-28465_5.6.105,CVE-2023-28465,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),hapifhir,org.hl7.fhir.core,5.6.105,https://github.com/hapifhir/org.hl7.fhir.core,GHSA-9654-pr4f-gh6m,0b8e2c39841b51def04e29d3b745602517602d5b,909f7e64fea6532a00b3eb8904bf620fc3cae6d1
 47,Graylog2__graylog2-server_CVE-2023-41044_5.1.2,CVE-2023-41044,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),Graylog2,graylog2-server,5.1.2,https://github.com/Graylog2/graylog2-server,GHSA-2q4p-f6gf-mqr5,d970230b6f3ecb83e07dd43df56428d1421c40e9,02b8792e6f4b829f0c1d87fcbf2d58b73458b938
 48,apache__mina-sshd_CVE-2023-35887_2.9.2,CVE-2023-35887,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,mina-sshd,2.9.2,https://github.com/apache/mina-sshd,GHSA-mjmq-gwgm-5qhm,fbbbb7b6af7fdf1765f297a96dfec3f198f5fd7d,10de190e7d3f9189deb76b8d08c72334a1fe2df0;a61e93035f06bff8fc622ad94870fb773d48b9f0;c20739b43aab0f7bf2ccad982a6cb37b9d5a8a0b
 49,apache__shiro_CVE-2023-34478_1.11.0,CVE-2023-34478,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,shiro,1.11.0,https://github.com/apache/shiro,GHSA-pmhc-2g4f-85cg,adb56c885210eeca710b2df17f6b3aeda85e4f35,c3ede3f94efb442acb0795714a022c2c121d1da0
 50,codehaus-plexus__plexus-archiver_CVE-2023-37460_4.7.1,CVE-2023-37460,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),codehaus-plexus,plexus-archiver,4.7.1,https://github.com/codehaus-plexus/plexus-archiver,GHSA-wh3p-fphp-9h2m,e7cbadba1b5c78192677da95b355dd790f3a4535,54759839fbdf85caf8442076f001d5fd64e0dcb2
 51,codehaus-plexus__plexus-utils_CVE-2022-4244_3.0.23,CVE-2022-4244,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),codehaus-plexus,plexus-utils,3.0.23,https://github.com/codehaus-plexus/plexus-utils,GHSA-g6ph-x5wf-g337,6176f91bb843eecddac857591ed20bda38e01470,33a2853df8185b4519b1b8bfae284f03392618ef
 52,yamcs__yamcs_CVE-2023-45278_5.8.6,CVE-2023-45278,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),yamcs,yamcs,5.8.6,https://github.com/yamcs/yamcs,GHSA-43fw-536j-w37j,9443a67d2aeda20fb73f68c5d571b515d68c05e4,b6db35e432e62fdff0447cf513369f0b8a848abf
 53,yamcs__yamcs_CVE-2023-45277_5.8.6,CVE-2023-45277,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),yamcs,yamcs,5.8.6,https://github.com/yamcs/yamcs,GHSA-w4m2-qmh3-2g8f,9443a67d2aeda20fb73f68c5d571b515d68c05e4,b6db35e432e62fdff0447cf513369f0b8a848abf
 54,apache__shiro_CVE-2023-46749_1.12.0,CVE-2023-46749,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,shiro,1.12.0,https://github.com/apache/shiro,GHSA-jc7h-c423-mpjc,c8a294ab32b29a6e9d4b03bbaa6bae11c42daea1,8400d08d5eac0bc4fae99d28c5adc82dd8a86eda
 55,apache__sling-org-apache-sling-servlets-resolver_CVE-2024-23673_2.10.0,CVE-2024-23673,CWE-022,Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'),apache,sling-org-apache-sling-servlets-resolver,2.10.0,https://github.com/apache/sling-org-apache-sling-servlets-resolver,GHSA-h2rq-qhr7-53gm,cb80c5e80144fcd9f6ac283ffb564f3b863cfbb2,b54d4e6693e0bcd63a97a0328f4f065b8a81b75e
 56,x-stream__xstream_CVE-2013-7285_1.4.6,CVE-2013-7285,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),x-stream,xstream,1.4.6,https://github.com/x-stream/xstream,GHSA-f554-x222-wgf7,768c6e417a75e7732fc591bee844e5e81af56a7d,6344867dce6767af7d0fe34fb393271a6456672d
 57,x-stream__xstream_CVE-2020-26217_1.4.14-java7,CVE-2020-26217,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),x-stream,xstream,1.4.14-java7,https://github.com/x-stream/xstream,GHSA-mw36-7c6c-q4q2,d03f6987b793fac71ab89a31d7aa633c366c5289,0fec095d534126931c99fd38e9c6d41f5c685c1a
 58,x-stream__xstream_CVE-2021-21345_1.4.15,CVE-2021-21345,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),x-stream,xstream,1.4.15,https://github.com/x-stream/xstream,GHSA-hwpc-8xqv-jvj4,f04bbec461f2c2a6f1e2cf41770f42c64aae24a4,d5e51177634afea7213b9dc2d21f101d2e258db9
 59,jenkinsci__docker-commons-plugin_CVE-2022-20617_1.17,CVE-2022-20617,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,docker-commons-plugin,1.17,https://github.com/jenkinsci/docker-commons-plugin,GHSA-jpxj-vgq5-prjc,2f0fda49452a7653c33de24220c908786f1aa405,c069b79c31c5aa80a01b0c462f0dc6b41751f059
 60,jenkinsci__workflow-cps-plugin_CVE-2022-25173_2646.v6ed3b5b01ff1,CVE-2022-25173,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,workflow-cps-plugin,2646.v6ed3b5b01ff1,https://github.com/jenkinsci/workflow-cps-plugin,GHSA-4m7p-55jm-3vwv,6ed3b5b01ff16dc4f6f9f4b209dbc21394b13913,f7ae7b75a457976853539bff1db52373b85fdb85
 61,jenkinsci__workflow-cps-global-lib-plugin_CVE-2022-25174_544.vff04fa68714d,CVE-2022-25174,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,workflow-cps-global-lib-plugin,544.vff04fa68714d,https://github.com/jenkinsci/workflow-cps-global-lib-plugin,GHSA-g9fx-6j5c-grmw,ff04fa68714d2cc277e8a87512461bdf3a39e186,ace0de3c2d691662021ea10306eeb407da6b6365
 62,jenkinsci__workflow-multibranch-plugin_CVE-2022-25175_706.vd43c65dec013,CVE-2022-25175,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,workflow-multibranch-plugin,706.vd43c65dec013,https://github.com/jenkinsci/workflow-multibranch-plugin,GHSA-pj84-qjm3-77mg,d43c65dec013812b084fecbb4a310d856eb6b89a,71c3f0a6ccdb2ba43f43686826b0d62160df85e8
 63,apache__activemq_CVE-2014-3576_5.10.1,CVE-2014-3576,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),apache,activemq,5.10.1,https://github.com/apache/activemq,GHSA-3wfj-vh84-732p,8938d14d434447193b02ba635606aa0fb7a80353,f07e6a53216f9388185ac2b39f366f3bfd6a8a55
 64,codehaus-plexus__plexus-utils_CVE-2017-1000487_3.0.15,CVE-2017-1000487,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),codehaus-plexus,plexus-utils,3.0.15,https://github.com/codehaus-plexus/plexus-utils,GHSA-8vhq-qq4p-grq3,3b271b43f9e5021b4232fc2ffdfb4e469c7f8685,b38a1b3a4352303e4312b2bb601a0d7ec6e28f41
 65,jenkinsci__git-client-plugin_CVE-2019-10392_2.8.4,CVE-2019-10392,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,git-client-plugin,2.8.4,https://github.com/jenkinsci/git-client-plugin,GHSA-hw6x-2qwv-rxr7,72add81da4cbeabc95504ebdb9007d47ac1c2b0b,899123fa2eb9dd2c37137aae630c47c6be6b4b02
 66,jenkinsci__perfecto-plugin_CVE-2020-2261_1.17,CVE-2020-2261,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,perfecto-plugin,1.17,https://github.com/jenkinsci/perfecto-plugin,GHSA-jq84-6fmm-6qv6,adabcd9da98628e063bb91198e30ff39a34dc921,52c09ad2e26416211e4009b095c4f16508b27b8a
 67,apache__nifi_CVE-2022-33140_1.16.2,CVE-2022-33140,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),apache,nifi,1.16.2,https://github.com/apache/nifi,GHSA-77hf-23pq-2g7c,06f04958272dafc30ce357c4c4edcaf470050b52,a1306a19f78e11c2d809e9aef24336669461403a
 68,jenkinsci__script-security-plugin_CVE-2023-24422_1228.vd93135a_2fb_25,CVE-2023-24422,CWE-078,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'),jenkinsci,script-security-plugin,1228.vd93135a_2fb_25,https://github.com/jenkinsci/script-security-plugin,GHSA-76qj-9gwh-pvv3,d93135a2fb250c04cc361b0bb1efd809bc63ad08,4880bbe905a6783d80150c8b881d0127430d4a73
 69,nahsra__antisamy_CVE-2016-10006_1.5.3,CVE-2016-10006,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),nahsra,antisamy,1.5.3,https://github.com/nahsra/antisamy,GHSA-683w-6h9j-57wq,8bebe1eb2ec1ac23e34111e9d06024d7dab7fa25,90550e42b39972fce0db4bcab0e1ce4359b94661;7313931dc3c0d1377b010f07faef2063dd359a36;5f4982e95f1894669550bb3f6a639573dd4e3756;5a783b4116a21a12ab10f57cd1c88616902af633
 70,nahsra__antisamy_CVE-2017-14735_1.5.6,CVE-2017-14735,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),nahsra,antisamy,1.5.6,https://github.com/nahsra/antisamy,GHSA-q44v-xc3g-v7jq,1369d4c904e1f021d3616580daf9e1d8e43e0ddb,82da009e733a989a57190cd6aa1b6824724f6d36
 71,apache__jspwiki_CVE-2019-10077_2.11.0.M3,CVE-2019-10077,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,jspwiki,2.11.0.M3,https://github.com/apache/jspwiki,GHSA-cj6j-32rg-45r2,2ea7973c1c7fdf2771a9743dbc8ecb27c9bb162d,87c89f0405d6b31fc165358ce5d5bc4536e32a8a
 72,apache__jspwiki_CVE-2019-10076_2.11.0.M3,CVE-2019-10076,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,jspwiki,2.11.0.M3,https://github.com/apache/jspwiki,GHSA-cxx2-fp39-rf3r,2ea7973c1c7fdf2771a9743dbc8ecb27c9bb162d,60a9cb84ce36a0392fa5a01afcf28c11857c2f9e
 73,apache__jspwiki_CVE-2019-10078_2.11.0.M3,CVE-2019-10078,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,jspwiki,2.11.0.M3,https://github.com/apache/jspwiki,GHSA-hp5r-mhgp-56c9,2ea7973c1c7fdf2771a9743dbc8ecb27c9bb162d,46cd981dfb431730da3f9249f5db858aacf11e52
 74,apache__jspwiki_CVE-2019-10089_2.11.0.M4,CVE-2019-10089,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,jspwiki,2.11.0.M4,https://github.com/apache/jspwiki,GHSA-3rx2-x6mx-grj3,6e1dc66adf3faa22e7f1f60189611a3d8ddb0726,2956ccb307dd4b23b25c8ddeae8d7e7b301c6ff3
 75,hibernate__hibernate-validator_CVE-2019-10219_6.0.17.Final,CVE-2019-10219,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),hibernate,hibernate-validator,6.0.17.Final,https://github.com/hibernate/hibernate-validator,GHSA-m8p2-495h-ccmh,7694cff9ed6325006382390c9bb2d8e4f3efedb6,124b7dd6d9a4ad24d4d49f74701f05a13e56ceee;20d729548511ac5cff6fd459f93de137195420fe
 76,asf__cxf_CVE-2019-17573_3.2.11,CVE-2019-17573,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,cxf,3.2.11,https://github.com/apache/cxf,GHSA-f93p-f762-vr53,da2d27d97f9e9abd7d307e2224e5e0338b767ee2,a02e96ba1095596bef481919f16a90c5e80a92c8
 77,xuxueli__xxl-job_CVE-2020-29204_2.2.0,CVE-2020-29204,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xuxueli,xxl-job,2.2.0,https://github.com/xuxueli/xxl-job,GHSA-wc73-w5r9-x9pc,1874c4854f81d6af4a9471d3fe111cdbf713f336,227628567354d3c156951009d683c6fec3006e0e
 78,OWASP__json-sanitizer_CVE-2020-13973_1.2.0,CVE-2020-13973,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),OWASP,json-sanitizer,1.2.0,https://github.com/OWASP/json-sanitizer,GHSA-g8jj-899q-8x3j,bb41f80ad575cb90ea6535976d7c17accb4b4c87,f3512bab078af3f5c533dacc2d410add8c0c1124
 79,eclipse__hawkbit_CVE-2020-27219_0.3.0M6,CVE-2020-27219,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),eclipse,hawkbit,0.3.0M6,https://github.com/eclipse/hawkbit,GHSA-rcvx-rmvf-mxch,f3659f01425ad0162f92fa73357f8c507058bcb2,94b7c12cde1b38eda5414bd88d6d068008cfb9f9
 80,alibaba__nacos_CVE-2021-44667_2.0.3,CVE-2021-44667,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),alibaba,nacos,2.0.3,https://github.com/alibaba/nacos,GHSA-4gr7-qw2q-jxh6,5a4d433970be9b8402f57e65bffa497027f21e30,cd6d7e33b94f24814701f3faf8b632e5e85444c5
 81,nahsra__antisamy_CVE-2022-28367_1.6.5,CVE-2022-28367,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),nahsra,antisamy,1.6.5,https://github.com/nahsra/antisamy,GHSA-3pqg-4rqg-pg9g,b7c151112dd19928e700ee48639942fd1079d1e3,0199e7e194dba5e7d7197703f43ebe22401e61ae
 82,ESAPI__esapi-java-legacy_CVE-2022-24891_2.2.3.1,CVE-2022-24891,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),ESAPI,esapi-java-legacy,2.2.3.1,https://github.com/ESAPI/esapi-java-legacy,GHSA-q77q-vx4q-xx6q,2e8694c6beb3bdbb2645b882eba72ce41bc63242,096aa60d6441fce4c632b36b36fdf5f53799c29a
 83,nahsra__antisamy_CVE-2022-29577_1.6.6.1,CVE-2022-29577,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),nahsra,antisamy,1.6.6.1,https://github.com/nahsra/antisamy,GHSA-vp37-2f9p-3vr3,b6e76de85a07549f2c97941437dd73430d7287a8,32e273507da0e964b58c50fd8a4c94c9d9363af0
 84,rhuss__jolokia_CVE-2018-1000129_1.4.0,CVE-2018-1000129,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),rhuss,jolokia,1.4.0,https://github.com/rhuss/jolokia,GHSA-hfpg-gqjw-779m,10727cab59a8fc2ae053bec6b2f26f48f1c4245c,5895d5c137c335e6b473e9dcb9baf748851bbc5f
 85,keycloak__keycloak_CVE-2014-3656_1.0.5.Final,CVE-2014-3656,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),keycloak,keycloak,1.0.5.Final,https://github.com/keycloak/keycloak,GHSA-px42-mr8m-cpgh,3b071b83aa0444749eba851fdbe0f2ed47932410,63b41e2548cbc20bd3758e34a82d880e177bf24c
 86,asf__cxf_CVE-2016-6812_3.0.11,CVE-2016-6812,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,cxf,3.0.11,https://github.com/apache/cxf,GHSA-vw2c-5wph-v92r,6c7df35913ce3fc6ec8f5dc6c6074b187f751aa6,1be97cb13aef121b799b1be4d9793c0e8b925a12;1f824d8039c7a42a4aa46f844e6c800e1143c7e7;32e89366e2daa5670ac7a5c5c19f0bf9329a4c1e;a30397b0
 87,apache__sling-org-apache-sling-xss_CVE-2016-5394_1.0.8,CVE-2016-5394,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,sling-org-apache-sling-xss,1.0.8,https://github.com/apache/sling-org-apache-sling-xss,GHSA-xwf4-88xr-hx2j,e23ca68878dd6504066015601b6b76e8fe19b752,de32b144ad2be3367559f6184d560db42a220529
 88,DSpace__DSpace_CVE-2022-31192_5.10,CVE-2022-31192,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),DSpace,DSpace,5.10,https://github.com/DSpace/DSpace,GHSA-4wm8-c2vv-xrpq,eca7968be7d6b9f8f5f302c9fc09f8186ed4809e,28eb8158210d41168a62ed5f9e044f754513bc37;f7758457b7ec3489d525e39aa753cc70809d9ad9
 89,keycloak__keycloak_CVE-2022-4137_20.0.3,CVE-2022-4137,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),keycloak,keycloak,20.0.3,https://github.com/keycloak/keycloak,GHSA-9hhc-pj4w-w5rv,02d113ab71de67817890b655ae792580a4fb8ee2,30d0e9d22dae51392e5a3748a1c68c116667359a
 90,keycloak__keycloak_CVE-2022-1274_20.0.3,CVE-2022-1274,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),keycloak,keycloak,20.0.3,https://github.com/keycloak/keycloak,GHSA-m4fv-gm5m-4725,02d113ab71de67817890b655ae792580a4fb8ee2,fc3c61235fa30132123c17ed8702ff7b3a672fe9
 91,xwiki__xwiki-commons_CVE-2023-29201_14.5,CVE-2023-29201,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xwiki,xwiki-commons,14.5,https://github.com/xwiki/xwiki-commons,GHSA-m3jr-cvhj-f35j,3780c8f6836c7d380f649562e9718a0868112945,4a185e0594d90cd4916d60aa60bb4333dc5623b2;b11eae9d82cb53f32962056b5faa73f3720c6182
 92,xwiki__xwiki-commons_CVE-2023-29528_14.9-rc-1,CVE-2023-29528,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xwiki,xwiki-commons,14.9-rc-1,https://github.com/xwiki/xwiki-commons,GHSA-x37v-36wv-6v6h,7796f76096a3e94b7a0a96bf12b3b026922e3573,8ff1a9d7e5d7b45b690134a537d53dc05cae04ab
 93,xwiki__xwiki-rendering_CVE-2023-32070_14.5,CVE-2023-32070,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xwiki,xwiki-rendering,14.5,https://github.com/xwiki/xwiki-rendering,GHSA-6gf5-c898-7rxp,0057b2db748674446ceeacf3a540d42b7295a524,c40e2f5f9482ec6c3e71dbf1fff5ba8a5e44cdc1
 94,xwiki__xwiki-commons_CVE-2023-31126_14.10.3,CVE-2023-31126,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xwiki,xwiki-commons,14.10.3,https://github.com/xwiki/xwiki-commons,GHSA-pv7v-ph6g-3gxv,011dee864805aec0b24151f69f230a42e40087da,0b8e9c45b7e7457043938f35265b2aa5adc76a68
 95,apache__jspwiki_CVE-2022-46907_2.11.3,CVE-2022-46907,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),apache,jspwiki,2.11.3,https://github.com/apache/jspwiki,GHSA-qvq8-cw7f-m7m4,515dff39c66c43318056fa76a20117662d2a0e8d,0b9a0149032170063f22d65e335dfd317db815ea;46e1ef7a595ca5cabf5ef184139910413f2024fc;484c6a133e397693991b7c9a9b62ef3ca48ce707;75019d337f1d0033b1f65428e75f43baeffd99dd;82be08904a6d8bd22fa2d4e5a7e85f43408724d3;9d6dbf911d52d724297e4e46c4b80649fb028ff9;df20770f251a8d7431047e556b144ef24ee6a3a7
 96,keycloak__keycloak_CVE-2022-4361_21.1.1,CVE-2022-4361,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),keycloak,keycloak,21.1.1,https://github.com/keycloak/keycloak,GHSA-3p62-6fjh-3p5h,0ca0a5fa4459d97c4d769fec15b4ba99c3824314,a1cfe6e24e5b34792699a00b8b4a8016a5929e3a
 97,xwiki__xwiki-commons_CVE-2023-36471_14.10.5,CVE-2023-36471,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xwiki,xwiki-commons,14.10.5,https://github.com/xwiki/xwiki-commons,GHSA-6pqf-c99p-758v,8dfb9fcb4f456c70f82ea670ca0e2f3dee373ee4,99484d48e899a68a1b6e33d457825b776c6fe8c3
 98,jstachio__jstachio_CVE-2023-33962_1.0.0,CVE-2023-33962,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),jstachio,jstachio,1.0.0,https://github.com/jstachio/jstachio,GHSA-gwxv-jv83-6qjr,9ce20009d6bf726086fc528fceb174933077bff4,7b2f78377d1284df14c580be762a25af5f8dcd66
 99,xwiki__xwiki-rendering_CVE-2023-37908_14.10.3,CVE-2023-37908,CWE-079,Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'),xwiki,xwiki-rendering,14.10.3,https://github.com/xwiki/xwiki-rendering,GHSA-663w-2xp3-5739,0b0286c4caeab4cc1fdd617ec5bf0a91e6f46762,f4d5acac451dccaf276e69f0b49b72221eef5d2f
 100,SpringSource__spring-security-oauth_CVE-2018-1260_2.3.2.RELEASE,CVE-2018-1260,CWE-094,Improper Control of Generation of Code ('Code Injection'),spring-attic,spring-security-oauth,2.3.2.RELEASE,https://github.com/spring-attic/spring-security-oauth,GHSA-rrpm-pj7p-7j9q,97e39dde7e88aae802be98de084a382886ca4255,70e5ba84ca98b7ab42c1900bdd9fa51b393d611f
 101,apache__activemq_CVE-2019-0222_5.15.8,CVE-2019-0222,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,activemq,5.15.8,https://github.com/apache/activemq,GHSA-jpv3-g4cc-6vfx,bcad7e1f6a6dd078d87787a03b56560e995ef773,98b9f2e;f78c0962ffb46fae3397eed6b7ec1e6e15045031
 102,jmrozanec__cron-utils_CVE-2021-41269_9.1.5,CVE-2021-41269,CWE-094,Improper Control of Generation of Code ('Code Injection'),jmrozanec,cron-utils,9.1.5,https://github.com/jmrozanec/cron-utils,GHSA-p9m8-27x8-rg87,34493c66edb490396202edad66c5f8cc5717d494,cfd2880f80e62ea74b92fa83474c2aabdb9899da;d6707503ec2f20947f79e38f861dba93b39df9da
 103,apache__struts_CVE-2020-17530_2.5.25,CVE-2020-17530,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,struts,2.5.25,https://github.com/apache/struts,GHSA-jc35-q369-45pv,b3a9d82d5830ef9cd7811cfa3f86a373ae52fada,45667346629455f7ea125bff36bf9b763b7e8463
 104,apache__activemq_CVE-2020-11998_5.15.12,CVE-2020-11998,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,activemq,5.15.12,https://github.com/apache/activemq,GHSA-wqfh-9m4g-7x6x,cebbd7fdea0633e6a0cbbef427f9b753949fa531,0d6e5f2;88b78d0;aa8900c
 105,spring-projects__spring-framework_CVE-2022-22965_5.2.19.RELEASE,CVE-2022-22965,CWE-094,Improper Control of Generation of Code ('Code Injection'),spring-projects,spring-framework,5.2.19.RELEASE,https://github.com/spring-projects/spring-framework,GHSA-36p3-wjmg-h94x,925b07826067d57a8a057837a1b5973cb6fad186,002546b3e4b8d791ea6acccb81eb3168f51abb15
 106,spring-cloud__spring-cloud-gateway_CVE-2022-22947_3.0.6,CVE-2022-22947,CWE-094,Improper Control of Generation of Code ('Code Injection'),spring-cloud,spring-cloud-gateway,3.0.6,https://github.com/spring-cloud/spring-cloud-gateway,GHSA-3gx9-37ww-9qw6,da0f9014bcc9131d8f15141be74011b89a01420f,1b868d2159e8fb71c22d583e424ae699f90532e3
 107,apache__dubbo_CVE-2021-30180_2.7.9,CVE-2021-30180,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,dubbo,2.7.9,https://github.com/apache/dubbo,GHSA-7wfc-x4f7-gg2x,f0483b80a32e813a4393879744e30d0084c3eafd,041f8bb0a98a912bd363560d5a2339198dcf79a5
 108,apache__incubator-dubbo_CVE-2021-30181_2.6.8,CVE-2021-30181,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,dubbo,2.6.8,https://github.com/apache/dubbo,GHSA-qmfc-6www-fjqw,5a6a069986f3627f2d50720c50024df31a3dc38a,9b7575aa1c949959c86d29c47625575b58a9dfa9
 109,spring-projects__spring-security_CVE-2011-2732_2.0.6.RELEASE,CVE-2011-2732,CWE-094,Improper Control of Generation of Code ('Code Injection'),spring-projects,spring-security,2.0.6.RELEASE,https://github.com/spring-projects/spring-security,GHSA-5xm9-rf63-wj7h,beb0ec4ba95265a7e3481b3eb8d87f3b922733ae,a087e828a63edf0932e4eecf174cf816cbe6a58a;5238ba0e2615ef733a906b264c6c06a4446a886b
 110,fabric8io__kubernetes-client_CVE-2021-4178_5.0.2,CVE-2021-4178,CWE-094,Improper Control of Generation of Code ('Code Injection'),fabric8io,kubernetes-client,5.0.2,https://github.com/fabric8io/kubernetes-client,GHSA-98g7-rxmf-rrxm,1d1b3d404836871c4d0fc0a35dafe5d40369b519,445103004d1ed3153d5abb272473451d05891e39
 111,asf__commons-text_CVE-2022-42889_1.9,CVE-2022-42889,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,commons-text,1.9,https://github.com/apache/commons-text,GHSA-599f-7c49-w659,cb85bed468e99d34b88d0c81fe20eb3b1615660e,b9b40b903e2d1f9935039803c9852439576780ea
 112,ff4j__ff4j_CVE-2022-44262_1.8.13,CVE-2022-44262,CWE-094,Improper Control of Generation of Code ('Code Injection'),ff4j,ff4j,1.8.13,https://github.com/ff4j/ff4j,GHSA-65hj-9ppw-77xc,7c34409d320ed1c917c4ec4ac72a99661c4e446f,62d03140f3bfbd380d0f4b4e5a94dcb8baa9d9c6
 113,codecentric__spring-boot-admin_CVE-2022-46166_2.6.9,CVE-2022-46166,CWE-094,Improper Control of Generation of Code ('Code Injection'),codecentric,spring-boot-admin,2.6.9,https://github.com/codecentric/spring-boot-admin,GHSA-w3x5-427h-wfq6,650db9e89d80f54b41ebd18f8c27389e8741516d,c14c3ec12533f71f84de9ce3ce5ceb7991975f75
 114,xerial__sqlite-jdbc_CVE-2023-32697_3.41.2.1,CVE-2023-32697,CWE-094,Improper Control of Generation of Code ('Code Injection'),xerial,sqlite-jdbc,3.41.2.1,https://github.com/xerial/sqlite-jdbc,GHSA-6phf-6h5g-97j2,a3322b2ae802b28624bcd4e2e20c55558e46cf99,edb4b8adc2447bc04e05b9b908195a4bc7926242
 115,asf__nifi_CVE-2023-34468_1.21.0,CVE-2023-34468,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,nifi,1.21.0,https://github.com/apache/nifi,GHSA-xm2m-2q6h-22jw,892f822107da84ca0dcfefdb5c91c5bc11dc3dc1,4faf3ea59895e7e153db3f8f61147ff70a254361
 116,apache__rocketmq_CVE-2023-37582_4.9.6,CVE-2023-37582,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,rocketmq,4.9.6,https://github.com/apache/rocketmq,GHSA-gpq8-963w-8qc9,db4ba459bec9392ac37647ff1a4d24d30466814d,fb1c67d536c95ec7bd5904e61b9d97c4c2ee5a3d
 117,asf__nifi_CVE-2023-36542_1.22.0,CVE-2023-36542,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,nifi,1.22.0,https://github.com/apache/nifi,GHSA-r969-8v3h-23v9,71e3ea9f2c58d0cf2ce6824c388f2bd3e917dfc8,532578799c
 118,apache__rocketmq_CVE-2023-33246_5.1.0,CVE-2023-33246,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,rocketmq,5.1.0,https://github.com/apache/rocketmq,GHSA-x3cq-8f32-5f63,b8e9334d2a1ed8868a3b15b63b8ef3198fcb81ce,9d411cf04a695e7a3f41036e8377b0aa544d754d;c3ada731405c5990c36bf58d50b3e61965300703
 119,apache__dolphinscheduler_CVE-2023-51770_3.2.0,CVE-2023-51770,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,dolphinscheduler,3.2.0,https://github.com/apache/dolphinscheduler,GHSA-ff2w-wm48-jhqj,f64b254809f21f811eaef780d489a43b490ca327,309c8c97a9b9003244496dedc7861a5026f9441a
 120,apache__dolphinscheduler_CVE-2023-49109_3.2.0,CVE-2023-49109,CWE-094,Improper Control of Generation of Code ('Code Injection'),apache,dolphinscheduler,3.2.0,https://github.com/apache/dolphinscheduler,GHSA-qwxx-xww6-8q8m,f64b254809f21f811eaef780d489a43b490ca327,8c01fcf14b199e2c1ba86c17351e1ea30614f484
@@ -0,0 +1,27 @@
 package cn.hutool.core.util;
 import java.io.File;
 import java.nio.charset.Charset;
 import java.util.Enumeration;
 import java.util.zip.ZipEntry;
 import java.util.zip.ZipFile;
 /**
 * Excerpt of hutool ZipUtil.unzip after CVE-2018-17297 (Zip Slip / CWE-22).
 * https://github.com/dromara/hutool/commit/8d7d0b7fb5ea4f7447b40131bffc1ec506a6528e
 */
 public class ZipUtil {
    public static File unzip(File zipFile, File outFile, Charset charset) throws Exception {
        ZipFile zipFileObj = new ZipFile(zipFile, charset);
        Enumeration<? extends ZipEntry> em = zipFileObj.entries();
        while (em.hasMoreElements()) {
            ZipEntry zipEntry = em.nextElement();
            File outItemFile = FileUtil.file(outFile, zipEntry.getName());
            if (zipEntry.isDirectory()) {
                outItemFile.mkdirs();
            }
        }
        zipFileObj.close();
        return outFile;
    }
 }
@@ -0,0 +1,28 @@
 package cn.hutool.core.util;
 import java.io.File;
 import java.nio.charset.Charset;
 import java.util.Enumeration;
 import java.util.zip.ZipEntry;
 import java.util.zip.ZipFile;
 /**
 * Excerpt of hutool ZipUtil.unzip before CVE-2018-17297 (Zip Slip / CWE-22).
 * https://github.com/dromara/hutool/commit/8d7d0b7fb5ea4f7447b40131bffc1ec506a6528e
 * parent fd77bb8718648f69a587046b735488a80123cc26
 */
 public class ZipUtil {
    public static File unzip(File zipFile, File outFile, Charset charset) throws Exception {
        ZipFile zipFileObj = new ZipFile(zipFile, charset);
        Enumeration<? extends ZipEntry> em = zipFileObj.entries();
        while (em.hasMoreElements()) {
            ZipEntry zipEntry = em.nextElement();
            File outItemFile = new File(outFile, zipEntry.getName());
            if (zipEntry.isDirectory()) {
                outItemFile.mkdirs();
            }
        }
        zipFileObj.close();
        return outFile;
    }
 }
@@ -0,0 +1,6 @@
 // Fixed counterpart of the node-serialize CVE-2017-5941 eval pattern (CWE-95).
 function parsePayload(obj) {
  return JSON.parse(obj);
 }
 module.exports = { parsePayload };
@@ -0,0 +1,7 @@
 // Pattern from npm node-serialize CVE-2017-5941 (CWE-95): unserialize via eval.
 // Public advisory: https://nvd.nist.gov/vuln/detail/CVE-2017-5941
 function parsePayload(obj) {
  return eval("(" + obj + ")");
 }
 module.exports = { parsePayload };
@@ -0,0 +1,10 @@
 # Provenance: SVEN train CWE-78 function pair from
 # https://github.com/Internet-of-People/titania-os/commit/9b7805119938343fcac9dc929d8882f1d97cf14a
 # file vuedj/configtitania/views.py function add_user
 import crypt
 import subprocess
 def add_user(username, password):
    encPass = crypt.crypt(password, "22")
    subprocess.call(["useradd", "-G", "docker,wheel", "-p", encPass, username])
@@ -0,0 +1,10 @@
 # Provenance: SVEN train CWE-78 function pair from
 # https://github.com/Internet-of-People/titania-os/commit/9b7805119938343fcac9dc929d8882f1d97cf14a
 # file vuedj/configtitania/views.py function add_user
 import crypt
 import os
 def add_user(username, password):
    encPass = crypt.crypt(password, "22")
    os.system("useradd -G docker,wheel -p " + encPass + " " + username)
@@ -0,0 +1,22 @@
 # Provenance: SVEN train CWE-78 function pair from
 # https://github.com/google/mobly/commit/3862e8ba359040fbdd6e1a6d36e51d07cda8e1ee
 # file mobly/controllers/android_device_lib/adb.py function _exec_cmd
 import logging
 import subprocess
 class AdbError(Exception):
    def __init__(self, **kwargs):
        super().__init__(kwargs)
 class Adb:
    def _exec_cmd(self, args, shell):
        """Executes adb commands."""
        proc = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=shell)
        (out, err) = proc.communicate()
        ret = proc.returncode
        logging.debug("cmd: %s, stdout: %s, stderr: %s, ret: %s", args, out, err, ret)
        if ret == 0:
            return out
        raise AdbError(cmd=args, stdout=out, stderr=err, ret_code=ret)
@@ -0,0 +1,22 @@
 # Provenance: SVEN train CWE-78 function pair from
 # https://github.com/google/mobly/commit/3862e8ba359040fbdd6e1a6d36e51d07cda8e1ee
 # file mobly/controllers/android_device_lib/adb.py function _exec_cmd
 import logging
 import subprocess
 class AdbError(Exception):
    def __init__(self, **kwargs):
        super().__init__(kwargs)
 class Adb:
    def _exec_cmd(self, cmd):
        """Executes adb commands in a new shell."""
        proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
        (out, err) = proc.communicate()
        ret = proc.returncode
        logging.debug("cmd: %s, stdout: %s, stderr: %s, ret: %s", cmd, out, err, ret)
        if ret == 0:
            return out
        raise AdbError(cmd=cmd, stdout=out, stderr=err, ret_code=ret)
@@ -0,0 +1,22 @@
 # Provenance: SVEN train CWE-89 function pair from
 # https://github.com/russ-lewis/ttt_-_python_cgi/commit/6096f43fd4b2d91211eec4614b7960c0816900da
 # file cgi/common.py function build_board
 def build_board(conn, game, size):
    board = []
    for i in range(size):
        board.append([""] * size)
    cursor = conn.cursor()
    cursor.execute("SELECT x,y,letter FROM moves WHERE gameID = %d;", (game,))
    counts = {"X": 0, "O": 0}
    for move in cursor.fetchall():
        (x, y, letter) = move
        x = int(x)
        y = int(y)
        board[x][y] = letter
        counts[letter] += 1
    cursor.close()
    return (board, counts)
@@ -0,0 +1,22 @@
 # Provenance: SVEN train CWE-89 function pair from
 # https://github.com/russ-lewis/ttt_-_python_cgi/commit/6096f43fd4b2d91211eec4614b7960c0816900da
 # file cgi/common.py function build_board
 def build_board(conn, game, size):
    board = []
    for i in range(size):
        board.append([""] * size)
    cursor = conn.cursor()
    cursor.execute("SELECT x,y,letter FROM moves WHERE gameID = %d;" % game)
    counts = {"X": 0, "O": 0}
    for move in cursor.fetchall():
        (x, y, letter) = move
        x = int(x)
        y = int(y)
        board[x][y] = letter
        counts[letter] += 1
    cursor.close()
    return (board, counts)
@@ -0,0 +1,14 @@
 # Provenance: SVEN train CWE-89 function pair from
 # https://github.com/pukkapies/urop2019/commit/3ca2e2c291d2d5fe262d20a8e0520bdfb622432b
 # file modules/query_lastfm.py function tag_num_to_tag
 class QueryStore:
    def tag_num_to_tag(self, tag_num):
        """Returns tag given tag_num."""
        q = "SELECT tag FROM tags WHERE rowid = ?"
        self.query(q, tag_num)
        return self.c.fetchone()[0]
    def query(self, q, *args):
        return q
@@ -0,0 +1,14 @@
 # Provenance: SVEN train CWE-89 function pair from
 # https://github.com/pukkapies/urop2019/commit/3ca2e2c291d2d5fe262d20a8e0520bdfb622432b
 # file modules/query_lastfm.py function tag_num_to_tag
 class QueryStore:
    def tag_num_to_tag(self, tag_num):
        """Returns tag given tag_num."""
        q = "SELECT tag FROM tags WHERE rowid = '" + str(tag_num) + "'"
        self.query(q)
        return self.c.fetchone()[0]
    def query(self, q):
        return q
@@ -0,0 +1,21 @@
 # SAST bake-off slice (OWASP + Juliet + Python)
 Labeled true/false cases from the suites SAST papers actually quote. Not
 cheat-sheet fixtures and not the 90% smoke gate.
 | Suite | Upstream | License | Protocol |
 | --- | --- | --- | --- |
 | OWASP Benchmark Java v1.2 | [OWASP-Benchmark/BenchmarkJava](https://github.com/OWASP-Benchmark/BenchmarkJava) | GPL-2.0 | true file = TP if we fire; false file = FP if we fire. Score = TPR − FPR. |
 | OWASP Benchmark Python v0.1 | [OWASP-Benchmark/BenchmarkPython](https://github.com/OWASP-Benchmark/BenchmarkPython) | GPL-3.0 | Same CSV protocol (`expectedresults-0.1.csv`). |
 | Juliet C 1.3 | [arichardson/juliet-test-suite-c](https://github.com/arichardson/juliet-test-suite-c) | CC0 | `bad()` vs `goodB2G()` / `goodG2B()`. |
 | Juliet Java 1.3 | [UnitTestBot/juliet-java-test-suite](https://github.com/UnitTestBot/juliet-java-test-suite) | CC0 | `bad()` vs `goodG2B()` (same sink, hardcoded source). |
 This directory vendors a **representative slice**, not the full 2,740 / 180k cases.
 ```bash
 uv run ousast pairs --slice sast
 ```
 `slice=sast` is an honesty dashboard: CI still only fails the local fixture pairs.
 Juliet `goodG2B` often keeps the dangerous sink (`strcpy`, `Runtime.exec`); a
 regex harness is *supposed* to keep firing there.
@@ -0,0 +1,253 @@
 # Labeled SAST bake-off slice. OWASP true/false files and Juliet bad/good sinks.
 # slice = "sast" — honesty dashboard, not the local 100% CI gate.
 # Do not add these names to LANGUAGE_MANIFESTS.
 # --- OWASP Benchmark for Python v0.1 ---
 [[pair]]
 name = "owasp-python-codeinj"
 slice = "sast"
 language = "python"
 origin = "owasp-benchmark-python"
 license = "GPL-3.0"
 repo = "OWASP-Benchmark/BenchmarkPython"
 cwe = "CWE-94"
 vuln = "owasp-python/BenchmarkTest00074.py"
 fixed = "owasp-python/BenchmarkTest00073.py"
 relpath = "testcode/BenchmarkTest00074.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-94"
 class = "code injection"
 path = "testcode/BenchmarkTest00074.py"
 rule_id = "python-unsafe-eval"
 sink = "exec"
 evidence = "exec executes attacker-controlled input"
 [[pair]]
 name = "owasp-python-deser"
 slice = "sast"
 language = "python"
 origin = "owasp-benchmark-python"
 license = "GPL-3.0"
 repo = "OWASP-Benchmark/BenchmarkPython"
 cwe = "CWE-502"
 vuln = "owasp-python/BenchmarkTest00078.py"
 fixed = "owasp-python/BenchmarkTest00079.py"
 relpath = "testcode/BenchmarkTest00078.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-502"
 class = "insecure deserialization"
 path = "testcode/BenchmarkTest00078.py"
 rule_id = "python-unsafe-deserialization"
 sink = "pickle"
 evidence = "pickle.loads deserializes untrusted input"
 [[pair]]
 name = "owasp-python-cmdi"
 slice = "sast"
 language = "python"
 origin = "owasp-benchmark-python"
 license = "GPL-3.0"
 repo = "OWASP-Benchmark/BenchmarkPython"
 cwe = "CWE-78"
 vuln = "owasp-python/BenchmarkTest00165.py"
 fixed = "owasp-python/BenchmarkTest00350.py"
 relpath = "testcode/BenchmarkTest00165.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-78"
 class = "command injection"
 path = "testcode/BenchmarkTest00165.py"
 rule_id = "python-shell-true"
 sink = "subprocess"
 evidence = "subprocess runs an attacker-controlled command"
 [[pair]]
 name = "owasp-python-sqli"
 slice = "sast"
 language = "python"
 origin = "owasp-benchmark-python"
 license = "GPL-3.0"
 repo = "OWASP-Benchmark/BenchmarkPython"
 cwe = "CWE-89"
 vuln = "owasp-python/BenchmarkTest00099.py"
 fixed = "owasp-python/BenchmarkTest00011.py"
 relpath = "testcode/BenchmarkTest00099.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-89"
 class = "sql injection"
 path = "testcode/BenchmarkTest00099.py"
 rule_id = "python-sql-injection"
 sink = "execute"
 evidence = "execute runs a SQL string built from request input"
 [[pair]]
 name = "owasp-python-hash"
 slice = "sast"
 language = "python"
 origin = "owasp-benchmark-python"
 license = "GPL-3.0"
 repo = "OWASP-Benchmark/BenchmarkPython"
 cwe = "CWE-327"
 vuln = "owasp-python/BenchmarkTest00054.py"
 fixed = "owasp-python/BenchmarkTest00055.py"
 relpath = "testcode/BenchmarkTest00054.py"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-327"
 class = "weak cryptography"
 path = "testcode/BenchmarkTest00054.py"
 rule_id = "python-weak-hash"
 sink = "md5"
 evidence = "md5 is a weak hash"
 # --- OWASP Benchmark Java v1.2 ---
 [[pair]]
 name = "owasp-java-cmdi"
 slice = "sast"
 language = "java"
 origin = "owasp-benchmark-java"
 license = "GPL-2.0"
 repo = "OWASP-Benchmark/BenchmarkJava"
 cwe = "CWE-78"
 vuln = "owasp-java/BenchmarkTest00007.java"
 fixed = "owasp-java/BenchmarkTest00090.java"
 relpath = "org/owasp/benchmark/testcode/BenchmarkTest00007.java"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-78"
 class = "command injection"
 path = "org/owasp/benchmark/testcode/BenchmarkTest00007.java"
 rule_id = "java-command-exec"
 sink = "exec"
 evidence = "Runtime.exec runs an attacker-controlled command"
 [[pair]]
 name = "owasp-java-sqli"
 slice = "sast"
 language = "java"
 origin = "owasp-benchmark-java"
 license = "GPL-2.0"
 repo = "OWASP-Benchmark/BenchmarkJava"
 cwe = "CWE-89"
 vuln = "owasp-java/BenchmarkTest00008.java"
 fixed = "owasp-java/BenchmarkTest00052.java"
 relpath = "org/owasp/benchmark/testcode/BenchmarkTest00008.java"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-89"
 class = "sql injection"
 path = "org/owasp/benchmark/testcode/BenchmarkTest00008.java"
 rule_id = "java-sql-injection"
 sink = "prepareCall"
 evidence = "SQL is concatenated with request input"
 [[pair]]
 name = "owasp-java-hash"
 slice = "sast"
 language = "java"
 origin = "owasp-benchmark-java"
 license = "GPL-2.0"
 repo = "OWASP-Benchmark/BenchmarkJava"
 cwe = "CWE-327"
 vuln = "owasp-java/BenchmarkTest00003.java"
 fixed = "owasp-java/BenchmarkTest00009.java"
 relpath = "org/owasp/benchmark/testcode/BenchmarkTest00003.java"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-327"
 class = "weak cryptography"
 path = "org/owasp/benchmark/testcode/BenchmarkTest00003.java"
 rule_id = "java-weak-hash"
 sink = "MessageDigest"
 evidence = "MessageDigest uses a weak hash algorithm"
 # --- Juliet C/C++ 1.3 (CC0) ---
 [[pair]]
 name = "juliet-c-cwe134-printf"
 slice = "sast"
 language = "c_cpp"
 origin = "juliet-c"
 license = "CC0-1.0"
 repo = "arichardson/juliet-test-suite-c"
 cwe = "CWE-134"
 vuln = "juliet-c/cwe134-printf-vuln.c"
 fixed = "juliet-c/cwe134-printf-fixed.c"
 relpath = "CWE134_char_console_printf_01.c"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-134"
 class = "format string"
 path = "CWE134_char_console_printf_01.c"
 rule_id = "c-unsafe-format"
 sink = "printf"
 evidence = "printf uses attacker-controlled format string"
 [[pair]]
 name = "juliet-c-cwe121-strcpy"
 slice = "sast"
 language = "c_cpp"
 origin = "juliet-c"
 license = "CC0-1.0"
 repo = "arichardson/juliet-test-suite-c"
 cwe = "CWE-120"
 vuln = "juliet-c/cwe121-strcpy-vuln.c"
 fixed = "juliet-c/cwe121-strcpy-fixed.c"
 relpath = "CWE121_char_declare_cpy_01.c"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-120"
 class = "buffer overflow"
 path = "CWE121_char_declare_cpy_01.c"
 rule_id = "c-unsafe-copy"
 sink = "strcpy"
 evidence = "strcpy copies into a buffer without a null terminator"
 # --- Juliet Java 1.3 (CC0) ---
 [[pair]]
 name = "juliet-java-cwe78-exec"
 slice = "sast"
 language = "java"
 origin = "juliet-java"
 license = "CC0-1.0"
 repo = "UnitTestBot/juliet-java-test-suite"
 cwe = "CWE-78"
 vuln = "juliet-java/cwe78-exec-vuln.java"
 fixed = "juliet-java/cwe78-exec-fixed.java"
 relpath = "CWE78_OS_Command_Injection__console_readLine_01.java"
 min_recall = 1.0
 fix_policy = "silent"
 [[pair.expected]]
 cwe = "CWE-78"
 class = "command injection"
 path = "CWE78_OS_Command_Injection__console_readLine_01.java"
 rule_id = "java-command-exec"
 sink = "exec"
 evidence = "Runtime.exec concatenates attacker-controlled input into a command"
@@ -0,0 +1,99 @@
 /* TEMPLATE GENERATED TESTCASE FILE
 Filename: CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01.c
 Label Definition File: CWE121_Stack_Based_Buffer_Overflow__CWE193.label.xml
 Template File: sources-sink-01.tmpl.c
 */
 /*
 * @description
 * CWE: 121 Stack Based Buffer Overflow
 * BadSource:  Point data to a buffer that does not have space for a NULL terminator
 * GoodSource: Point data to a buffer that includes space for a NULL terminator
 * Sink: cpy
 *    BadSink : Copy string to data using strcpy()
 * Flow Variant: 01 Baseline
 *
 * */
 #include "std_testcase.h"
 #ifndef _WIN32
 #include <wchar.h>
 #endif
 /* MAINTENANCE NOTE: The length of this string should equal the 10 */
 #define SRC_STRING "AAAAAAAAAA"
 #ifndef OMITBAD
 void CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01_bad()
 {
    char * data;
    char dataBadBuffer[10];
    char dataGoodBuffer[10+1];
    /* FLAW: Set a pointer to a buffer that does not leave room for a NULL terminator when performing
     * string copies in the sinks  */
    data = dataBadBuffer;
    data[0] = '\0'; /* null terminate */
    {
        char source[10+1] = SRC_STRING;
        /* POTENTIAL FLAW: data may not have enough space to hold source */
        strcpy(data, source);
        printLine(data);
    }
 }
 #endif /* OMITBAD */
 #ifndef OMITGOOD
 /* goodG2B uses the GoodSource with the BadSink */
 static void goodG2B()
 {
    char * data;
    char dataBadBuffer[10];
    char dataGoodBuffer[10+1];
    /* FIX: Set a pointer to a buffer that leaves room for a NULL terminator when performing
     * string copies in the sinks  */
    data = dataGoodBuffer;
    data[0] = '\0'; /* null terminate */
    {
        char source[10+1] = SRC_STRING;
        /* POTENTIAL FLAW: data may not have enough space to hold source */
        strcpy(data, source);
        printLine(data);
    }
 }
 void CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01_good()
 {
    goodG2B();
 }
 #endif /* OMITGOOD */
 /* Below is the main(). It is only used when building this testcase on
 * its own for testing or for building a binary to use in testing binary
 * analysis tools. It is not used when compiling all the testcases as one
 * application, which is how source code analysis tools are tested.
 */
 #ifdef INCLUDEMAIN
 int main(int argc, char * argv[])
 {
    /* seed randomness */
    srand( (unsigned)time(NULL) );
 #ifndef OMITGOOD
    printLine("Calling good()...");
    CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01_good();
    printLine("Finished good()");
 #endif /* OMITGOOD */
 #ifndef OMITBAD
    printLine("Calling bad()...");
    CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01_bad();
    printLine("Finished bad()");
 #endif /* OMITBAD */
    return 0;
 }
 #endif
@@ -0,0 +1,143 @@
 /* TEMPLATE GENERATED TESTCASE FILE
 Filename: CWE134_Uncontrolled_Format_String__char_console_printf_01.c
 Label Definition File: CWE134_Uncontrolled_Format_String.label.xml
 Template File: sources-sinks-01.tmpl.c
 */
 /*
 * @description
 * CWE: 134 Uncontrolled Format String
 * BadSource: console Read input from the console
 * GoodSource: Copy a fixed string into data
 * Sinks: printf
 *    GoodSink: printf with "%s" as the first argument and data as the second
 *    BadSink : printf with only data as an argument
 * Flow Variant: 01 Baseline
 *
 * */
 #include "std_testcase.h"
 #ifndef _WIN32
 #include <wchar.h>
 #endif
 #ifndef OMITBAD
 void CWE134_Uncontrolled_Format_String__char_console_printf_01_bad()
 {
    char * data;
    char dataBuffer[100] = "";
    data = dataBuffer;
    {
        /* Read input from the console */
        size_t dataLen = strlen(data);
        /* if there is room in data, read into it from the console */
        if (100-dataLen > 1)
        {
            /* POTENTIAL FLAW: Read data from the console */
            if (fgets(data+dataLen, (int)(100-dataLen), stdin) != NULL)
            {
                /* The next few lines remove the carriage return from the string that is
                 * inserted by fgets() */
                dataLen = strlen(data);
                if (dataLen > 0 && data[dataLen-1] == '\n')
                {
                    data[dataLen-1] = '\0';
                }
            }
            else
            {
                printLine("fgets() failed");
                /* Restore NUL terminator if fgets fails */
                data[dataLen] = '\0';
            }
        }
    }
    /* POTENTIAL FLAW: Do not specify the format allowing a possible format string vulnerability */
    printf(data);
 }
 #endif /* OMITBAD */
 #ifndef OMITGOOD
 /* goodG2B uses the GoodSource with the BadSink */
 static void goodG2B()
 {
    char * data;
    char dataBuffer[100] = "";
    data = dataBuffer;
    /* FIX: Use a fixed string that does not contain a format specifier */
    strcpy(data, "fixedstringtest");
    /* POTENTIAL FLAW: Do not specify the format allowing a possible format string vulnerability */
    printf(data);
 }
 /* goodB2G uses the BadSource with the GoodSink */
 static void goodB2G()
 {
    char * data;
    char dataBuffer[100] = "";
    data = dataBuffer;
    {
        /* Read input from the console */
        size_t dataLen = strlen(data);
        /* if there is room in data, read into it from the console */
        if (100-dataLen > 1)
        {
            /* POTENTIAL FLAW: Read data from the console */
            if (fgets(data+dataLen, (int)(100-dataLen), stdin) != NULL)
            {
                /* The next few lines remove the carriage return from the string that is
                 * inserted by fgets() */
                dataLen = strlen(data);
                if (dataLen > 0 && data[dataLen-1] == '\n')
                {
                    data[dataLen-1] = '\0';
                }
            }
            else
            {
                printLine("fgets() failed");
                /* Restore NUL terminator if fgets fails */
                data[dataLen] = '\0';
            }
        }
    }
    /* FIX: Specify the format disallowing a format string vulnerability */
    printf("%s\n", data);
 }
 void CWE134_Uncontrolled_Format_String__char_console_printf_01_good()
 {
    goodG2B();
    goodB2G();
 }
 #endif /* OMITGOOD */
 /* Below is the main(). It is only used when building this testcase on
   its own for testing or for building a binary to use in testing binary
   analysis tools. It is not used when compiling all the testcases as one
   application, which is how source code analysis tools are tested. */
 #ifdef INCLUDEMAIN
 int main(int argc, char * argv[])
 {
    /* seed randomness */
    srand( (unsigned)time(NULL) );
 #ifndef OMITGOOD
    printLine("Calling good()...");
    CWE134_Uncontrolled_Format_String__char_console_printf_01_good();
    printLine("Finished good()");
 #endif /* OMITGOOD */
 #ifndef OMITBAD
    printLine("Calling bad()...");
    CWE134_Uncontrolled_Format_String__char_console_printf_01_bad();
    printLine("Finished bad()");
 #endif /* OMITBAD */
    return 0;
 }
 #endif
@@ -0,0 +1,10 @@
 /* Juliet CWE-121 goodG2B still copies with strcpy (larger buffer, same sink).
 * Regex SAST is expected to keep firing — that is the Juliet goodG2B trap.
 * Upstream: CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01.c
 */
 #include <string.h>
 void CWE121_char_declare_cpy_01_goodG2B(char *data, char *source)
 {
    strcpy(data, source);
 }
@@ -0,0 +1,11 @@
 /* Juliet CWE-121 flow variant 01 — bad sink only.
 * Upstream: CWE121_Stack_Based_Buffer_Overflow__CWE193_char_declare_cpy_01.c
 * https://github.com/arichardson/juliet-test-suite-c (NIST Juliet 1.3, CC0)
 */
 #include <string.h>
 void CWE121_char_declare_cpy_01_bad(char *data, char *source)
 {
    /* POTENTIAL FLAW: data may not have enough space to hold source */
    strcpy(data, source);
 }
@@ -0,0 +1,11 @@
 /* Juliet CWE-134 flow variant 01 — goodB2G sink only.
 * Upstream: CWE134_Uncontrolled_Format_String__char_console_printf_01.c
 * https://github.com/arichardson/juliet-test-suite-c (NIST Juliet 1.3, CC0)
 */
 #include <stdio.h>
 void CWE134_char_console_printf_01_goodB2G(char *data)
 {
    /* FIX: Specify the format disallowing a format string vulnerability */
    printf("%s\n", data);
 }
@@ -0,0 +1,12 @@
 /* Juliet CWE-134 flow variant 01 — bad sink only.
 * Upstream: testcases/CWE134_Uncontrolled_Format_String/s01/
 *   CWE134_Uncontrolled_Format_String__char_console_printf_01.c
 * https://github.com/arichardson/juliet-test-suite-c (NIST Juliet 1.3, CC0)
 */
 #include <stdio.h>
 void CWE134_char_console_printf_01_bad(char *data)
 {
    /* POTENTIAL FLAW: Do not specify the format allowing a format string vulnerability */
    printf(data);
 }
@@ -0,0 +1,143 @@
 /* TEMPLATE GENERATED TESTCASE FILE
 Filename: CWE78_OS_Command_Injection__console_readLine_01.java
 Label Definition File: CWE78_OS_Command_Injection.label.xml
 Template File: sources-sink-01.tmpl.java
 */
 /*
 * @description
 * CWE: 78 OS Command Injection
 * BadSource: console_readLine Read data from the console using readLine()
 * GoodSource: A hardcoded string
 * BadSink: exec dynamic command execution with Runtime.getRuntime().exec()
 * Flow Variant: 01 Baseline
 *
 * */
 package juliet.testcases.CWE78_OS_Command_Injection;
 import juliet.support.*;
 import javax.servlet.http.*;
 import java.io.BufferedReader;
 import java.io.InputStreamReader;
 import java.io.IOException;
 import java.util.logging.Level;
 public class CWE78_OS_Command_Injection__console_readLine_01 extends AbstractTestCase
 {
    /* uses badsource and badsink */
    public void bad() throws Throwable
    {
        String data;
        data = ""; /* Initialize data */
        {
            InputStreamReader readerInputStream = null;
            BufferedReader readerBuffered = null;
            /* read user input from console with readLine */
            try
            {
                readerInputStream = new InputStreamReader(System.in, "UTF-8");
                readerBuffered = new BufferedReader(readerInputStream);
                /* POTENTIAL FLAW: Read data from the console using readLine */
                data = readerBuffered.readLine();
            }
            catch (IOException exceptIO)
            {
                IO.logger.log(Level.WARNING, "Error with stream reading", exceptIO);
            }
            finally
            {
                try
                {
                    if (readerBuffered != null)
                    {
                        readerBuffered.close();
                    }
                }
                catch (IOException exceptIO)
                {
                    IO.logger.log(Level.WARNING, "Error closing BufferedReader", exceptIO);
                }
                try
                {
                    if (readerInputStream != null)
                    {
                        readerInputStream.close();
                    }
                }
                catch (IOException exceptIO)
                {
                    IO.logger.log(Level.WARNING, "Error closing InputStreamReader", exceptIO);
                }
            }
        }
        /* NOTE: Tools may report a flaw here because buffread and isr are not closed.  Unfortunately, closing those will close System.in, which will cause any future attempts to read from the console to fail and throw an exception */
        String osCommand;
        if(System.getProperty("os.name").toLowerCase().indexOf("win") >= 0)
        {
            /* running on Windows */
            osCommand = "c:\\WINDOWS\\SYSTEM32\\cmd.exe /c dir ";
        }
        else
        {
            /* running on non-Windows */
            osCommand = "/bin/ls ";
        }
        /* POTENTIAL FLAW: command injection */
        Process process = Runtime.getRuntime().exec(osCommand + data);
        process.waitFor();
    }
    public void good() throws Throwable
    {
        goodG2B();
    }
    /* goodG2B() - uses goodsource and badsink */
    private void goodG2B() throws Throwable
    {
        String data;
        /* FIX: Use a hardcoded string */
        data = "foo";
        String osCommand;
        if(System.getProperty("os.name").toLowerCase().indexOf("win") >= 0)
        {
            /* running on Windows */
            osCommand = "c:\\WINDOWS\\SYSTEM32\\cmd.exe /c dir ";
        }
        else
        {
            /* running on non-Windows */
            osCommand = "/bin/ls ";
        }
        /* POTENTIAL FLAW: command injection */
        Process process = Runtime.getRuntime().exec(osCommand + data);
        process.waitFor();
    }
    /* Below is the main(). It is only used when building this testcase on
     * its own for testing or for building a binary to use in testing binary
     * analysis tools. It is not used when compiling all the juliet.testcases as one
     * application, which is how source code analysis tools are tested.
     */
    public static void main(String[] args) throws ClassNotFoundException,
           InstantiationException, IllegalAccessException
    {
        mainFromParent(args);
    }
 }
@@ -0,0 +1,12 @@
 /* Juliet CWE-78 goodG2B — hardcoded source, same Runtime.exec sink.
 * Regex SAST is expected to keep firing — Juliet goodG2B trap.
 * Upstream: CWE78_OS_Command_Injection__console_readLine_01.java
 */
 public class CWE78_OS_Command_Injection__console_readLine_01 {
    public void goodG2B() throws Throwable {
        String data = "foo";
        String osCommand = "/bin/ls ";
        Process process = Runtime.getRuntime().exec(osCommand + data);
        process.waitFor();
    }
 }
@@ -0,0 +1,11 @@
 /* Juliet CWE-78 flow variant 01 — bad sink only.
 * Upstream: CWE78_OS_Command_Injection__console_readLine_01.java
 * https://github.com/UnitTestBot/juliet-java-test-suite (NIST Juliet 1.3, CC0)
 */
 public class CWE78_OS_Command_Injection__console_readLine_01 {
    public void bad(String data) throws Throwable {
        String osCommand = "/bin/ls ";
        Process process = Runtime.getRuntime().exec(osCommand + data);
        process.waitFor();
    }
 }
@@ -0,0 +1,120 @@
 /**
 * OWASP Benchmark v1.2
 *
 * <p>This file is part of the Open Web Application Security Project (OWASP) Benchmark Project. For
 * details, please see <a
 * href="https://owasp.org/www-project-benchmark/">https://owasp.org/www-project-benchmark/</a>.
 *
 * <p>The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation, version 2.
 *
 * <p>The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE. See the GNU General Public License for more details.
 *
 * @author Dave Wichers
 * @created 2015
 */
 package org.owasp.benchmark.testcode;
 import java.io.IOException;
 import javax.servlet.ServletException;
 import javax.servlet.annotation.WebServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@WebServlet(value = "/hash-00/BenchmarkTest00003")
 public class BenchmarkTest00003 extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        response.setContentType("text/html;charset=UTF-8");
        javax.servlet.http.Cookie userCookie =
                new javax.servlet.http.Cookie("BenchmarkTest00003", "someSecret");
        userCookie.setMaxAge(60 * 3); // Store cookie for 3 minutes
        userCookie.setSecure(true);
        userCookie.setHttpOnly(true);
        userCookie.setPath(request.getRequestURI());
        userCookie.setDomain(new java.net.URL(request.getRequestURL().toString()).getHost());
        response.addCookie(userCookie);
        javax.servlet.RequestDispatcher rd =
                request.getRequestDispatcher("/hash-00/BenchmarkTest00003.html");
        rd.include(request, response);
    }
    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // some code
        response.setContentType("text/html;charset=UTF-8");
        javax.servlet.http.Cookie[] theCookies = request.getCookies();
        String param = "noCookieValueSupplied";
        if (theCookies != null) {
            for (javax.servlet.http.Cookie theCookie : theCookies) {
                if (theCookie.getName().equals("BenchmarkTest00003")) {
                    param = java.net.URLDecoder.decode(theCookie.getValue(), "UTF-8");
                    break;
                }
            }
        }
        try {
            java.util.Properties benchmarkprops = new java.util.Properties();
            benchmarkprops.load(
                    this.getClass().getClassLoader().getResourceAsStream("benchmark.properties"));
            String algorithm = benchmarkprops.getProperty("hashAlg1", "SHA512");
            java.security.MessageDigest md = java.security.MessageDigest.getInstance(algorithm);
            byte[] input = {(byte) '?'};
            Object inputParam = param;
            if (inputParam instanceof String) input = ((String) inputParam).getBytes();
            if (inputParam instanceof java.io.InputStream) {
                byte[] strInput = new byte[1000];
                int i = ((java.io.InputStream) inputParam).read(strInput);
                if (i == -1) {
                    response.getWriter()
                            .println(
                                    "This input source requires a POST, not a GET. Incompatible UI for the InputStream source.");
                    return;
                }
                input = java.util.Arrays.copyOf(strInput, i);
            }
            md.update(input);
            byte[] result = md.digest();
            java.io.File fileTarget =
                    new java.io.File(
                            new java.io.File(org.owasp.benchmark.helpers.Utils.TESTFILES_DIR),
                            "passwordFile.txt");
            java.io.FileWriter fw =
                    new java.io.FileWriter(fileTarget, true); // the true will append the new data
            fw.write(
                    "hash_value="
                            + org.owasp.esapi.ESAPI.encoder().encodeForBase64(result, true)
                            + "\n");
            fw.close();
            response.getWriter()
                    .println(
                            "Sensitive value '"
                                    + org.owasp
                                            .esapi
                                            .ESAPI
                                            .encoder()
                                            .encodeForHTML(new String(input))
                                    + "' hashed and stored<br/>");
        } catch (java.security.NoSuchAlgorithmException e) {
            System.out.println("Problem executing hash - TestCase");
            throw new ServletException(e);
        }
        response.getWriter()
                .println(
                        "Hash Test java.security.MessageDigest.getInstance(java.lang.String) executed");
    }
 }
@@ -0,0 +1,70 @@
 /**
 * OWASP Benchmark v1.2
 *
 * <p>This file is part of the Open Web Application Security Project (OWASP) Benchmark Project. For
 * details, please see <a
 * href="https://owasp.org/www-project-benchmark/">https://owasp.org/www-project-benchmark/</a>.
 *
 * <p>The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation, version 2.
 *
 * <p>The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE. See the GNU General Public License for more details.
 *
 * @author Dave Wichers
 * @created 2015
 */
 package org.owasp.benchmark.testcode;
 import java.io.IOException;
 import javax.servlet.ServletException;
 import javax.servlet.annotation.WebServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@WebServlet(value = "/cmdi-00/BenchmarkTest00007")
 public class BenchmarkTest00007 extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        doPost(request, response);
    }
    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // some code
        response.setContentType("text/html;charset=UTF-8");
        String param = "";
        if (request.getHeader("BenchmarkTest00007") != null) {
            param = request.getHeader("BenchmarkTest00007");
        }
        // URL Decode the header value since req.getHeader() doesn't. Unlike req.getParameter().
        param = java.net.URLDecoder.decode(param, "UTF-8");
        String cmd =
                org.owasp.benchmark.helpers.Utils.getInsecureOSCommandString(
                        this.getClass().getClassLoader());
        String[] args = {cmd};
        String[] argsEnv = {param};
        Runtime r = Runtime.getRuntime();
        try {
            Process p = r.exec(args, argsEnv);
            org.owasp.benchmark.helpers.Utils.printOSCommandResults(p, response);
        } catch (IOException e) {
            System.out.println("Problem executing cmdi - TestCase");
            response.getWriter()
                    .println(org.owasp.esapi.ESAPI.encoder().encodeForHTML(e.getMessage()));
            return;
        }
    }
 }
@@ -0,0 +1,67 @@
 /**
 * OWASP Benchmark v1.2
 *
 * <p>This file is part of the Open Web Application Security Project (OWASP) Benchmark Project. For
 * details, please see <a
 * href="https://owasp.org/www-project-benchmark/">https://owasp.org/www-project-benchmark/</a>.
 *
 * <p>The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation, version 2.
 *
 * <p>The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE. See the GNU General Public License for more details.
 *
 * @author Dave Wichers
 * @created 2015
 */
 package org.owasp.benchmark.testcode;
 import java.io.IOException;
 import javax.servlet.ServletException;
 import javax.servlet.annotation.WebServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@WebServlet(value = "/sqli-00/BenchmarkTest00008")
 public class BenchmarkTest00008 extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        doPost(request, response);
    }
    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // some code
        response.setContentType("text/html;charset=UTF-8");
        String param = "";
        if (request.getHeader("BenchmarkTest00008") != null) {
            param = request.getHeader("BenchmarkTest00008");
        }
        // URL Decode the header value since req.getHeader() doesn't. Unlike req.getParameter().
        param = java.net.URLDecoder.decode(param, "UTF-8");
        String sql = "{call " + param + "}";
        try {
            java.sql.Connection connection =
                    org.owasp.benchmark.helpers.DatabaseHelper.getSqlConnection();
            java.sql.CallableStatement statement = connection.prepareCall(sql);
            java.sql.ResultSet rs = statement.executeQuery();
            org.owasp.benchmark.helpers.DatabaseHelper.printResults(rs, sql, response);
        } catch (java.sql.SQLException e) {
            if (org.owasp.benchmark.helpers.DatabaseHelper.hideSQLErrors) {
                response.getWriter().println("Error processing request.");
            } else throw new ServletException(e);
        }
    }
 }
@@ -0,0 +1,124 @@
 /**
 * OWASP Benchmark v1.2
 *
 * <p>This file is part of the Open Web Application Security Project (OWASP) Benchmark Project. For
 * details, please see <a
 * href="https://owasp.org/www-project-benchmark/">https://owasp.org/www-project-benchmark/</a>.
 *
 * <p>The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation, version 2.
 *
 * <p>The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE. See the GNU General Public License for more details.
 *
 * @author Dave Wichers
 * @created 2015
 */
 package org.owasp.benchmark.testcode;
 import java.io.IOException;
 import javax.servlet.ServletException;
 import javax.servlet.annotation.WebServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@WebServlet(value = "/hash-00/BenchmarkTest00009")
 public class BenchmarkTest00009 extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        doPost(request, response);
    }
    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // some code
        response.setContentType("text/html;charset=UTF-8");
        String param = "";
        java.util.Enumeration<String> names = request.getHeaderNames();
        while (names.hasMoreElements()) {
            String name = (String) names.nextElement();
            if (org.owasp.benchmark.helpers.Utils.commonHeaders.contains(name)) {
                continue; // If standard header, move on to next one
            }
            java.util.Enumeration<String> values = request.getHeaders(name);
            if (values != null && values.hasMoreElements()) {
                param = name; // Grabs the name of the first non-standard header as the parameter
                // value
                break;
            }
        }
        // Note: We don't URL decode header names because people don't normally do that
        java.security.Provider[] provider = java.security.Security.getProviders();
        java.security.MessageDigest md;
        try {
            if (provider.length > 1) {
                md = java.security.MessageDigest.getInstance("sha-384", provider[0]);
            } else {
                md = java.security.MessageDigest.getInstance("sha-384", "SUN");
            }
            byte[] input = {(byte) '?'};
            Object inputParam = param;
            if (inputParam instanceof String) input = ((String) inputParam).getBytes();
            if (inputParam instanceof java.io.InputStream) {
                byte[] strInput = new byte[1000];
                int i = ((java.io.InputStream) inputParam).read(strInput);
                if (i == -1) {
                    response.getWriter()
                            .println(
                                    "This input source requires a POST, not a GET. Incompatible UI for the InputStream source.");
                    return;
                }
                input = java.util.Arrays.copyOf(strInput, i);
            }
            md.update(input);
            byte[] result = md.digest();
            java.io.File fileTarget =
                    new java.io.File(
                            new java.io.File(org.owasp.benchmark.helpers.Utils.TESTFILES_DIR),
                            "passwordFile.txt");
            java.io.FileWriter fw =
                    new java.io.FileWriter(fileTarget, true); // the true will append the new data
            fw.write(
                    "hash_value="
                            + org.owasp.esapi.ESAPI.encoder().encodeForBase64(result, true)
                            + "\n");
            fw.close();
            response.getWriter()
                    .println(
                            "Sensitive value '"
                                    + org.owasp
                                            .esapi
                                            .ESAPI
                                            .encoder()
                                            .encodeForHTML(new String(input))
                                    + "' hashed and stored<br/>");
        } catch (java.security.NoSuchAlgorithmException e) {
            System.out.println(
                    "Problem executing hash - TestCase java.security.MessageDigest.getInstance(java.lang.String,java.security.Provider)");
            throw new ServletException(e);
        } catch (java.security.NoSuchProviderException e) {
            System.out.println(
                    "Problem executing hash - TestCase java.security.MessageDigest.getInstance(java.lang.String,java.security.Provider)");
            throw new ServletException(e);
        }
        response.getWriter()
                .println(
                        "Hash Test java.security.MessageDigest.getInstance(java.lang.String,java.security.Provider) executed");
    }
 }
@@ -0,0 +1,67 @@
 /**
 * OWASP Benchmark v1.2
 *
 * <p>This file is part of the Open Web Application Security Project (OWASP) Benchmark Project. For
 * details, please see <a
 * href="https://owasp.org/www-project-benchmark/">https://owasp.org/www-project-benchmark/</a>.
 *
 * <p>The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation, version 2.
 *
 * <p>The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE. See the GNU General Public License for more details.
 *
 * @author Dave Wichers
 * @created 2015
 */
 package org.owasp.benchmark.testcode;
 import java.io.IOException;
 import javax.servlet.ServletException;
 import javax.servlet.annotation.WebServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@WebServlet(value = "/sqli-00/BenchmarkTest00052")
 public class BenchmarkTest00052 extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        doPost(request, response);
    }
    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // some code
        response.setContentType("text/html;charset=UTF-8");
        org.owasp.benchmark.helpers.SeparateClassRequest scr =
                new org.owasp.benchmark.helpers.SeparateClassRequest(request);
        String param = scr.getTheValue("BenchmarkTest00052");
        String sql = "{call " + param + "}";
        try {
            java.sql.Connection connection =
                    org.owasp.benchmark.helpers.DatabaseHelper.getSqlConnection();
            java.sql.CallableStatement statement =
                    connection.prepareCall(
                            sql,
                            java.sql.ResultSet.TYPE_FORWARD_ONLY,
                            java.sql.ResultSet.CONCUR_READ_ONLY,
                            java.sql.ResultSet.CLOSE_CURSORS_AT_COMMIT);
            java.sql.ResultSet rs = statement.executeQuery();
            org.owasp.benchmark.helpers.DatabaseHelper.printResults(rs, sql, response);
        } catch (java.sql.SQLException e) {
            if (org.owasp.benchmark.helpers.DatabaseHelper.hideSQLErrors) {
                response.getWriter().println("Error processing request.");
            } else throw new ServletException(e);
        }
    }
 }
@@ -0,0 +1,91 @@
 /**
 * OWASP Benchmark Project v1.2
 *
 * <p>This file is part of the Open Web Application Security Project (OWASP) Benchmark Project. For
 * details, please see <a
 * href="https://owasp.org/www-project-benchmark/">https://owasp.org/www-project-benchmark/</a>.
 *
 * <p>The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation, version 2.
 *
 * <p>The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE. See the GNU General Public License for more details.
 *
 * @author Nick Sanidas
 * @created 2015
 */
 package org.owasp.benchmark.testcode;
 import java.io.IOException;
 import javax.servlet.ServletException;
 import javax.servlet.annotation.WebServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
@WebServlet(value = "/cmdi-00/BenchmarkTest00090")
 public class BenchmarkTest00090 extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        response.setContentType("text/html;charset=UTF-8");
        javax.servlet.http.Cookie userCookie =
                new javax.servlet.http.Cookie("BenchmarkTest00090", "ls");
        userCookie.setMaxAge(60 * 3); // Store cookie for 3 minutes
        userCookie.setSecure(true);
        userCookie.setHttpOnly(true);
        userCookie.setPath(request.getRequestURI());
        userCookie.setDomain(new java.net.URL(request.getRequestURL().toString()).getHost());
        response.addCookie(userCookie);
        javax.servlet.RequestDispatcher rd =
                request.getRequestDispatcher("/cmdi-00/BenchmarkTest00090.html");
        rd.include(request, response);
    }
    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        response.setContentType("text/html;charset=UTF-8");
        javax.servlet.http.Cookie[] theCookies = request.getCookies();
        String param = "noCookieValueSupplied";
        if (theCookies != null) {
            for (javax.servlet.http.Cookie theCookie : theCookies) {
                if (theCookie.getName().equals("BenchmarkTest00090")) {
                    param = java.net.URLDecoder.decode(theCookie.getValue(), "UTF-8");
                    break;
                }
            }
        }
        String bar;
        // Simple if statement that assigns constant to bar on true condition
        int num = 86;
        if ((7 * 42) - num > 200) bar = "This_should_always_happen";
        else bar = param;
        String cmd = "";
        String osName = System.getProperty("os.name");
        if (osName.indexOf("Windows") != -1) {
            cmd = org.owasp.benchmark.helpers.Utils.getOSCommandString("echo");
        }
        Runtime r = Runtime.getRuntime();
        try {
            Process p = r.exec(cmd + bar);
            org.owasp.benchmark.helpers.Utils.printOSCommandResults(p, response);
        } catch (IOException e) {
            System.out.println("Problem executing cmdi - TestCase");
            response.getWriter()
                    .println(org.owasp.esapi.ESAPI.encoder().encodeForHTML(e.getMessage()));
            return;
        }
    }
 }
@@ -0,0 +1,57 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/sqli-00/BenchmarkTest00011', methods=['GET'])
 	def BenchmarkTest00011_get():
 		response = make_response(render_template('web/sqli-00/BenchmarkTest00011.html'))
 		response.set_cookie('BenchmarkTest00011', 'bar',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00011_post()
 	@app.route('/benchmark/sqli-00/BenchmarkTest00011', methods=['POST'])
 	def BenchmarkTest00011_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00011", "noCookieValueSupplied"))
 		bar = "This should never happen"
 		if 'should' in bar:
 			bar = param
 		import helpers.db_sqlite
 		sql = f'SELECT username from USERS where password = ?'
 		con = helpers.db_sqlite.get_connection()
 		cur = con.cursor()
 		cur.execute(sql, (bar,))
 		RESPONSE += (
 			helpers.db_sqlite.results(cur, sql)
 		)
 		con.close()
 		return RESPONSE
@@ -0,0 +1,76 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/hash-00/BenchmarkTest00054', methods=['GET'])
 	def BenchmarkTest00054_get():
 		response = make_response(render_template('web/hash-00/BenchmarkTest00054.html'))
 		response.set_cookie('BenchmarkTest00054', 'someSecret',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00054_post()
 	@app.route('/benchmark/hash-00/BenchmarkTest00054', methods=['POST'])
 	def BenchmarkTest00054_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00054", "noCookieValueSupplied"))
 		string26833 = ''
 		data12 = ''
 		copy = string26833
 		string26833 = ''
 		string26833 += param
 		copy += 'SomeOKString'
 		bar = copy
 		import hashlib, base64
 		import io, helpers.utils
 		input = ''
 		if isinstance(bar, str):
 			input = bar.encode('utf-8')
 		elif isinstance(bar, io.IOBase):
 			input = bar.read(1000)
 		if len(input) == 0:
 			RESPONSE += (
 				'Cannot generate hash: Input was empty.'
 			)
 			return RESPONSE
 		hash = hashlib.new('md5')
 		hash.update(input)
 		result = hash.digest()
 		f = open(f'{helpers.utils.TESTFILES_DIR}/passwordFile.txt', 'a')
 		f.write(f'hash_value={base64.b64encode(result)}\n')
 		RESPONSE += (
 			f'Sensitive value \'{helpers.utils.escape_for_html(input.decode('utf-8'))}\' hashed and stored.'
 		)
 		f.close()
 		return RESPONSE
@@ -0,0 +1,72 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/hash-00/BenchmarkTest00055', methods=['GET'])
 	def BenchmarkTest00055_get():
 		response = make_response(render_template('web/hash-00/BenchmarkTest00055.html'))
 		response.set_cookie('BenchmarkTest00055', 'someSecret',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00055_post()
 	@app.route('/benchmark/hash-00/BenchmarkTest00055', methods=['POST'])
 	def BenchmarkTest00055_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00055", "noCookieValueSupplied"))
 		num = 106
 		bar = "This should never happen" if (7*42) - num > 200 else param
 		import hashlib, base64
 		import io, helpers.utils
 		input = ''
 		if isinstance(bar, str):
 			input = bar.encode('utf-8')
 		elif isinstance(bar, io.IOBase):
 			input = bar.read(1000)
 		if len(input) == 0:
 			RESPONSE += (
 				'Cannot generate hash: Input was empty.'
 			)
 			return RESPONSE
 		hash = hashlib.new('sha384')
 		hash.update(input)
 		result = hash.digest()
 		f = open(f'{helpers.utils.TESTFILES_DIR}/passwordFile.txt', 'a')
 		f.write(f'hash_value={base64.b64encode(result)}\n')
 		RESPONSE += (
 			f'Sensitive value \'{helpers.utils.escape_for_html(input.decode('utf-8'))}\' hashed and stored.'
 		)
 		f.close()
 		return RESPONSE
@@ -0,0 +1,62 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/codeinj-00/BenchmarkTest00073', methods=['GET'])
 	def BenchmarkTest00073_get():
 		response = make_response(render_template('web/codeinj-00/BenchmarkTest00073.html'))
 		response.set_cookie('BenchmarkTest00073', '%27ECHOOO%27',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00073_post()
 	@app.route('/benchmark/codeinj-00/BenchmarkTest00073', methods=['POST'])
 	def BenchmarkTest00073_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00073", "noCookieValueSupplied"))
 		string41416 = 'help'
 		string41416 += param
 		string41416 += 'snapes on a plane'
 		bar = string41416[4:-17]
 		if not bar.startswith('\'') or not bar.endswith('\'') or '\'' in bar[1:-1]:
 			RESPONSE += (
 				"Eval argument must be a plain string literal."
 			)
 			return RESPONSE		
 		try:
 			RESPONSE += (
 				eval(bar)
 			)
 		except:
 			RESPONSE += (
 				f'Error evaluating expression \'{escape_for_html(bar)}\''
 			)
 		return RESPONSE
@@ -0,0 +1,58 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/codeinj-00/BenchmarkTest00074', methods=['GET'])
 	def BenchmarkTest00074_get():
 		response = make_response(render_template('web/codeinj-00/BenchmarkTest00074.html'))
 		response.set_cookie('BenchmarkTest00074', '%27RESPONSE+%2B%3D+%5C%27ECHOOO%5C%27%27',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00074_post()
 	@app.route('/benchmark/codeinj-00/BenchmarkTest00074', methods=['POST'])
 	def BenchmarkTest00074_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00074", "noCookieValueSupplied"))
 		import configparser
 		bar = 'safe!'
 		conf90091 = configparser.ConfigParser()
 		conf90091.add_section('section90091')
 		conf90091.set('section90091', 'keyA-90091', 'a-Value')
 		conf90091.set('section90091', 'keyB-90091', param)
 		bar = conf90091.get('section90091', 'keyB-90091')
 		try:
 			exec(bar)
 		except:
 			RESPONSE += (
 				f'Error executing statement \'{escape_for_html(bar)}\''
 			)
 		return RESPONSE
@@ -0,0 +1,65 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/deserialization-00/BenchmarkTest00078', methods=['GET'])
 	def BenchmarkTest00078_get():
 		response = make_response(render_template('web/deserialization-00/BenchmarkTest00078.html'))
 		response.set_cookie('BenchmarkTest00078', 'gASVNwAAAAAAAACMCF9fbWFpbl9flIwOc2FmZV90b19waWNrbGWUk5QpgZR9lCiMAWGUjANmb2-UjAFilEtjdWIu',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00078_post()
 	@app.route('/benchmark/deserialization-00/BenchmarkTest00078', methods=['POST'])
 	def BenchmarkTest00078_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00078", "noCookieValueSupplied"))
 		string9895 = 'help'
 		string9895 += param
 		string9895 += 'snapes on a plane'
 		bar = string9895[4:-17]
 		import pickle
 		import base64
 		import helpers.utils
 		helpers.utils.sharedstr = "no pickles to be seen here"
 		try:
 			unpickled = pickle.loads(base64.urlsafe_b64decode(bar))
 		except:
 			RESPONSE += (
 				'Unpickling failed!'
 			)
 			return RESPONSE
 		RESPONSE += (
 			f'shared string is {helpers.utils.sharedstr}'
 		)
 		return RESPONSE
@@ -0,0 +1,57 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/deserialization-00/BenchmarkTest00079', methods=['GET'])
 	def BenchmarkTest00079_get():
 		response = make_response(render_template('web/deserialization-00/BenchmarkTest00079.html'))
 		response.set_cookie('BenchmarkTest00079', 'name%3A+safe+data%0Atext%3A+act+like+this+is+conf+data',
 			max_age=60*3,
 			secure=True,
 			path=request.path,
 			domain='localhost')
 		return response
 		return BenchmarkTest00079_post()
 	@app.route('/benchmark/deserialization-00/BenchmarkTest00079', methods=['POST'])
 	def BenchmarkTest00079_post():
 		RESPONSE = ""
 		import urllib.parse
 		param = urllib.parse.unquote_plus(request.cookies.get("BenchmarkTest00079", "noCookieValueSupplied"))
 		bar = param
 		import yaml
 		try:
 			yobj = yaml.safe_load(bar)
 			RESPONSE += (
 				yobj['text']
 			)
 		except:
 			RESPONSE += (
 				"There was an error loading the configuration"
 			)
 		return RESPONSE
@@ -0,0 +1,56 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/sqli-00/BenchmarkTest00099', methods=['GET'])
 	def BenchmarkTest00099_get():
 		return BenchmarkTest00099_post()
 	@app.route('/benchmark/sqli-00/BenchmarkTest00099', methods=['POST'])
 	def BenchmarkTest00099_post():
 		RESPONSE = ""
 		param = request.form.get("BenchmarkTest00099")
 		if not param:
 			param = ""
 		import configparser
 		bar = 'safe!'
 		conf52528 = configparser.ConfigParser()
 		conf52528.add_section('section52528')
 		conf52528.set('section52528', 'keyA-52528', 'a-Value')
 		conf52528.set('section52528', 'keyB-52528', param)
 		bar = conf52528.get('section52528', 'keyB-52528')
 		import helpers.db_sqlite
 		sql = f'SELECT username from USERS where password = \'{bar}\''
 		con = helpers.db_sqlite.get_connection()
 		cur = con.cursor()
 		cur.execute(sql)
 		RESPONSE += (
 			helpers.db_sqlite.results(cur, sql)
 		)
 		con.close()
 		return RESPONSE
@@ -0,0 +1,63 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/cmdi-00/BenchmarkTest00165', methods=['GET'])
 	def BenchmarkTest00165_get():
 		return BenchmarkTest00165_post()
 	@app.route('/benchmark/cmdi-00/BenchmarkTest00165', methods=['POST'])
 	def BenchmarkTest00165_post():
 		RESPONSE = ""
 		param = request.form.get("BenchmarkTest00165")
 		if not param:
 			param = ""
 		bar = ""
 		if param:
 			lst = []
 			lst.append('safe')
 			lst.append(param)
 			lst.append('moresafe')
 			lst.pop(0)
 			bar = lst[0]
 		import os
 		import subprocess
 		import helpers.utils
 		argList = []
 		if "Windows" in os.name:
 			argList.append("cmd.exe")
 			argList.append("-c")
 		else:
 			argList.append("sh")
 			argList.append("-c")
 		argList.append(f"echo {bar}")
 		proc = subprocess.run(argList, capture_output=True, encoding="utf-8")
 		RESPONSE += (
 			helpers.utils.commandOutput(proc)
 		)
 		return RESPONSE
@@ -0,0 +1,69 @@
 '''
 OWASP Benchmark for Python v0.1
 This file is part of the Open Web Application Security Project (OWASP) Benchmark Project.
 For details, please see https://owasp.org/www-project-benchmark.
 The OWASP Benchmark is free software: you can redistribute it and/or modify it under the terms
 of the GNU General Public License as published by the Free Software Foundation, version 3.
 The OWASP Benchmark is distributed in the hope that it will be useful, but WITHOUT ANY
 WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 PURPOSE. See the GNU General Public License for more details.
  Author: Theo Cartsonis
  Created: 2025
 '''
 from flask import redirect, url_for, request, make_response, render_template
 from helpers.utils import escape_for_html
 def init(app):
 	@app.route('/benchmark/cmdi-00/BenchmarkTest00350', methods=['GET'])
 	def BenchmarkTest00350_get():
 		return BenchmarkTest00350_post()
 	@app.route('/benchmark/cmdi-00/BenchmarkTest00350', methods=['POST'])
 	def BenchmarkTest00350_post():
 		RESPONSE = ""
 		import helpers.separate_request
 		wrapped = helpers.separate_request.request_wrapper(request)
 		param = wrapped.get_form_parameter("BenchmarkTest00350")
 		if not param:
 			param = ""
 		map11630 = {}
 		map11630['keyA-11630'] = 'a-Value'
 		map11630['keyB-11630'] = param
 		map11630['keyC'] = 'another-Value'
 		bar = "safe!"
 		bar = map11630['keyB-11630']
 		bar = map11630['keyA-11630']
 		import platform
 		import subprocess
 		import helpers.utils
 		argStr = ""
 		if platform.system() == "Windows":
 			argStr = "cmd.exe /c "
 		else:
 			argStr = "sh -c "
 		argStr += f"echo {bar}"
 		try:
 			proc = subprocess.run(argStr, shell=True, capture_output=True, encoding="utf-8")
 			RESPONSE += (
 				helpers.utils.commandOutput(proc)
 			)
 		except IOError:
 			RESPONSE += (
 				"Problem executing cmdi - subprocess.run(list) Test Case"
 			)
 		return RESPONSE
@@ -52,6 +52,7 @@ async def _run_checks() -> None:
            .with_exec(["uv", "run", "pytest"])
            .with_exec(["uv", "run", "python", "-m", "openultrasast.gate"])
            .with_exec(["uv", "run", "python", "-m", "openultrasast.map_gate"])
            .with_exec(["uv", "run", "python", "-m", "openultrasast.pair_gate"])
            .with_exec(["uv", "run", "python", "-m", "compileall", "src", "tests"])
            .with_exec(["uv", "build"])
        )
@@ -30,7 +30,21 @@ uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
 # -> expected / matched / missed counts + a per-rule recommendation delta
 ```
-## 4. Self-improve the ruleset from benchmark feedback
+## 4. Vuln vs fixed pair eval (efficiency, not cheat-sheet recall)
 ```bash
 uv run ousast pairs                              # local fixtures + vendored GitHub VFCs
 uv run ousast pairs --slice github --json        # honesty dashboard as JSON
 uv run ousast pairs --slice sast                 # OWASP Benchmark + Juliet Youden (TPR-FPR)
 uv run python -m openultrasast.pair_gate         # CI: local pairs must all pass
 ```
 Each case is two isolated trees (vuln file vs patched file, same relative path).
 `pair_correct` means the harness fired on the vuln side and stayed silent on the
 fix. Misses and fix-side leaks become `miss`/`fp` signals for `ousast improve`.
 The catalog and public dataset pointers live in `benchmarks/pairs/`.
 ## 5. Self-improve the ruleset from benchmark feedback
 ```bash
 uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run  # preview
@@ -39,7 +53,7 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
 # which the next scan/benchmark of that target loads automatically.
 ```
-## 5. Standard mode with the HarnessX agentic plane (OpenAI)
+## 6. Standard mode with the HarnessX agentic plane (OpenAI)
 ```bash
 uv sync --extra harnessx
@@ -63,7 +77,7 @@ jq '.degradations' "$run/manifest.json"   # null = HarnessX ran; entries = it fe
 jq '.fusion'       "$run/manifest.json"   # per-finding dispositions
 ```
-## 6. Drive it from an MCP client (OpenCode / IDE)
+## 7. Drive it from an MCP client (OpenCode / IDE)
 ```jsonc
 { "command": "uv", "args": ["run", "ousast", "mcp"] }
@@ -34,7 +34,7 @@ line-length = 140
 target-version = "py311"
 # Benchmark fixtures are intentionally vulnerable, unformatted sample corpora;
 # linting/formatting them would flag the planted bugs and shift ground-truth lines.
-extend-exclude = ["benchmarks/fixtures"]
+extend-exclude = ["benchmarks/fixtures", "benchmarks/pairs"]
 [tool.ruff.lint]
 select = ["B", "E", "F", "I", "SIM", "UP"]
@@ -40,15 +40,30 @@ from .hunter_harness import HxScanOrchestrator
 from .improve import RoundOutcome, run_improvement
 from .index import build_code_chunks
 from .mapping import analyze_entry_points, attach_reachability_hints, ingest_sarif, write_entry_points, write_static_hints
 from .pair_gate import print_pair_metrics
 from .pairs import DEFAULT_CATALOG, evaluate_catalog, load_pair_catalog, result_payload, select_slice
 from .policy import assert_rules_resolve, load_policy
 from .preprocess import preprocess_repository, write_preprocess_artifact
 from .provider.openrouter import OpenRouterEmbeddingClient, OpenRouterError
 from .rank import rank_targets, write_rankings
-from .regress import CandidateVerdict, run_regression, write_verdicts
+from .regress import TRIGGERABLE, CandidateVerdict, run_regression, write_verdicts
 from .reports import scan_exit_code, write_manifest, write_markdown_report, write_sarif_report
 from .ruleset import DEFAULT_RULESET_DIR, load_ruleset
 from .run import ScanRun, create_scan_run
 from .sandbox import resolve_sandbox_probe, resolve_sandbox_runner
 from .scoring import build_score_artifact
 from .semantic import (
    MechanismStore,
    OverlayRecord,
    adjudicate,
    append_mechanism,
    filter_promoted_hotspots,
    finding_from_coverage,
    load_facts,
    order_promotions,
    write_overlay,
 )
 from .semantic.facts import FactLoadError, SemanticFacts
 from .stages import Stage, plan_for_mode, record_completed, record_skip, skip_as_degradation, stages_payload
 from .verification import VerificationResult, verify_findings, write_verification_results
 from .verify_judge import verify_findings_dispatch
@@ -104,6 +119,14 @@ def main(argv: list[str] | None = None) -> int:
    )
    improve.add_argument("--dry-run", action="store_true", help="run rounds against a throwaway ledger; never touch the target's ledger")
    pairs = subparsers.add_parser(
        "pairs",
        help="scan isolated vuln-vs-fixed pairs (TP on vuln, silent on fix) and emit improve-loop signals",
    )
    pairs.add_argument("--catalog", type=Path, default=DEFAULT_CATALOG)
    pairs.add_argument("--slice", choices=("all", "local", "github", "sast"), default="all")
    pairs.add_argument("--json", action="store_true", help="print the pair scoreboard as JSON")
    subparsers.add_parser("mcp", help="run the narrow MCP server over stdio for OpenCode integration")
    args = parser.parse_args(argv)
@@ -124,6 +147,8 @@ def main(argv: list[str] | None = None) -> int:
            ruleset_dir=args.ruleset_dir,
            dry_run=args.dry_run,
        )
    if args.command == "pairs":
        return _pairs(args.catalog, args.slice, json_out=args.json)
    if args.command == "mcp":
        from .mcp import serve  # lazy: keeps the import cycle (mcp -> cli) one-directional
@@ -307,7 +332,10 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
    plan = record_completed(plan, Stage.STATIC)
    complexity_payload: dict[str, object] | None = None
    complexity_map_path = run.root / "complexity_map.json"
    overlay_path = run.root / "overlay.json"
    built_map = None
    overlay_records: list[OverlayRecord] = []
    wrote_overlay = False
    if Stage.MAP in plan.requested:
        built_map = runtime.run_stage(
            "map",
@@ -319,6 +347,27 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
                ledger_path=run.target / CALIBRATION_DIR / "complexity_ledger.json",
            ),
        )
        facts_or_error: SemanticFacts | FactLoadError
        try:
            facts_or_error = load_facts()
        except FactLoadError as exc:
            facts_or_error = exc
            runtime.state["degradations"].append({"stage": "overlay", "reason": "facts_unavailable", "detail": str(exc)})
        overlay_records = list(
            runtime.run_stage(
                "overlay",
                lambda: adjudicate(root=run.target, targets=targets, findings=findings, facts=facts_or_error),
            )
        )
        write_overlay(overlay_records, overlay_path)
        wrote_overlay = True
        coverage_findings = [finding_from_coverage(record) for record in overlay_records if record.disposition == "coverage"]
        if coverage_findings:
            seen = {finding.finding_id for finding in findings}
            extra = [item for item in coverage_findings if item.finding_id not in seen]
            if extra:
                findings = findings + extra
                write_findings(findings, findings_path)
        plan = record_completed(plan, Stage.MAP)
        if not hunter_model:
            runtime.state["degradations"].append(skip_as_degradation(Stage.MAP, "hunter_model_unavailable"))
@@ -351,14 +400,38 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
        else:
            sandbox = resolve_sandbox_runner()
            hotspots = built_map.hotspots if built_map is not None else ()
            hotspots = filter_promoted_hotspots(hotspots, overlay_records, findings)
            store = MechanismStore(run.target / CALIBRATION_DIR / "mechanisms.jsonl")
            embed_client = None
            embed_model = config.embeddings.model
            if embed_model:
                try:
                    embed_client = OpenRouterEmbeddingClient.from_env()
                except OpenRouterError:
                    embed_client = None
            hotspots, budget_degradation = order_promotions(
                hotspots,
                overlay_records,
                findings,
                store=store,
                client=embed_client,
                model=embed_model,
            )
            if budget_degradation:
                runtime.state["degradations"].append({"stage": "prove_budget", "reason": budget_degradation})
            promoted = [
                finding
                for finding in findings
                if finding.finding_id in {record.proposal_id for record in overlay_records if record.disposition == "promote"}
            ]
            cwe_by_rule_id = {rule.rule_id: rule.cwe for rule in ruleset}
-            rule_cwe_by_finding = {finding.finding_id: cwe_by_rule_id.get(finding.finding_id.split(":", 1)[0], "") for finding in findings}
+            rule_cwe_by_finding = {finding.finding_id: cwe_by_rule_id.get(finding.finding_id.split(":", 1)[0], "") for finding in promoted}
            languages_by_path = {target.path: target.language for target in targets}
            records = runtime.run_stage(
                "regress",
                lambda: run_regression(
                    hotspots,
-                    findings,
+                    promoted,
                    max_candidates=config.regress.max_candidates,
                    policy=policy,
                    rule_cwe=rule_cwe_by_finding,
@@ -371,6 +444,7 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
            )
            write_verdicts(records, verdicts_path)
            persist_verdicts(run.target / CALIBRATION_DIR / "complexity_ledger.json", records)
            _append_proven_mechanisms(store, records, overlay_records, findings, run.target)
            verdict_records = records
            wrote_verdicts = True
            plan = record_completed(plan, Stage.REGRESS)
@@ -384,9 +458,18 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
            redact=config.hardening.redact_secrets,
            complexity_map=built_map,
            verdicts=verdict_records if wrote_verdicts else None,
            overlay=overlay_records if wrote_overlay else None,
        ),
    )
    runtime.run_stage(
        "sarif",
        lambda: write_sarif_report(
            findings,
            verifications,
            sarif_path,
            overlay=overlay_records if wrote_overlay else None,
        ),
    )
    runtime.run_stage("sarif", lambda: write_sarif_report(findings, verifications, sarif_path))
    runtime.run_stage(
        "manifest",
        lambda: write_manifest(
@@ -402,6 +485,7 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
                trajectories=trajectories_path if trajectories else None,
                fusion=fusion_path if fusion_decisions else None,
                complexity_map=complexity_map_path if complexity_payload is not None else None,
                overlay=overlay_path if wrote_overlay else None,
                verdicts=verdicts_path if wrote_verdicts else None,
            ),
            path=manifest_path,
@@ -425,6 +509,35 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
    )
 def _append_proven_mechanisms(
    store: MechanismStore,
    verdicts: tuple[CandidateVerdict, ...],
    overlay_records: list[OverlayRecord],
    findings: list[StaticFinding],
    target: Path,
 ) -> None:
    overlay_by_id = {record.proposal_id: record for record in overlay_records}
    finding_by_id = {finding.finding_id: finding for finding in findings}
    for verdict in verdicts:
        if verdict.verdict != TRIGGERABLE:
            continue
        for finding_id in verdict.inventory_finding_ids:
            record = overlay_by_id.get(finding_id)
            if record is None or record.disposition != "promote":
                continue
            finding = finding_by_id.get(finding_id)
            append_mechanism(
                store,
                summary=record.reason,
                cwe=record.cwe,
                language=record.language or verdict.language,
                tags=finding.tags if finding is not None else (),
                what_made_it_exploitable=record.reason,
                source_finding_id=finding_id,
                source_repo=str(target),
            )
 def _attach_tool_hunter(
    *,
    root: Path,
@@ -610,6 +723,19 @@ def _improve(
    return 0
 def _pairs(catalog: Path, slice_name: str, *, json_out: bool) -> int:
    if not catalog.exists() or not catalog.is_file():
        raise SystemExit(f"pair catalog is not a file: {catalog}")
    cases = select_slice(load_pair_catalog(catalog), slice_name)
    result = evaluate_catalog(cases)
    if json_out:
        print(json.dumps(result_payload(result), indent=2, sort_keys=True))
        return 0
    print_pair_metrics("pair eval", result)
    print(f"signals={len(result.signals)}")
    return 0
 def _print_improve_outcomes(outcomes: list[RoundOutcome], manifest_path: Path, target: Path, ledger_path: Path, *, dry_run: bool) -> None:
    print(f"manifest={manifest_path}")
    print(f"target={target}")
@@ -1,5 +1,6 @@
 """Bounded, gated self-improvement loop over the ruleset/policy governance data."""
 from ..pairs import build_pair_signals
 from .evolve import RoundOutcome, build_rule_signals, propose_status_edits, run_improvement, run_round
 from .journal import load_journal, reverted_edit_keys
 from .validator import (
@@ -20,6 +21,7 @@ __all__ = [
    "RuleStatusEdit",
    "StrictValidationError",
    "VALID_LEVERS",
    "build_pair_signals",
    "build_rule_signals",
    "edits_to_ledger",
    "load_journal",
@@ -0,0 +1,86 @@
 """Pair-efficiency gate: local fixtures must fire on vuln and stay silent on fix.
 The stage-1 smoke gate stays on cheat-sheet trees. This module measures
 differential detection and only *fails CI* on the local slice. GitHub/SVEN
 pairs are printed as the honesty dashboard (real VFC misses become improve
 signals, they do not block merge).
    python -m openultrasast.pair_gate
 """
 from __future__ import annotations
 import sys
 from collections.abc import Sequence
 from dataclasses import dataclass
 from .pairs import (
    DEFAULT_CATALOG,
    PairCase,
    PairEvalResult,
    evaluate_catalog,
    load_pair_catalog,
    select_slice,
 )
@dataclass(frozen=True)
 class PairGateVerdict:
    passed: bool
    reasons: tuple[str, ...]
    result: PairEvalResult
 def pair_gate(cases: Sequence[PairCase] | None = None) -> PairGateVerdict:
    catalog = tuple(cases) if cases is not None else load_pair_catalog(DEFAULT_CATALOG)
    local = select_slice(catalog, "local")
    if not local:
        return PairGateVerdict(passed=False, reasons=("no local pairs in catalog",), result=evaluate_catalog(()))
    result = evaluate_catalog(catalog)
    local_metrics = result.per_slice.get("local")
    reasons: list[str] = []
    if local_metrics is None:
        reasons.append("local slice produced no outcomes")
    elif local_metrics.pair_pass_rate < 1.0:
        failed = [outcome.name for outcome in result.outcomes if outcome.slice == "local" and not outcome.pair_correct]
        reasons.append("local pair_pass_rate below 100%: " + ", ".join(failed))
    return PairGateVerdict(passed=not reasons, reasons=tuple(reasons), result=result)
 def print_pair_metrics(label: str, result: PairEvalResult) -> None:
    overall = result.overall
    print(
        f"{label}: pair_pass {overall.pair_pass_rate:.2%} ({overall.pair_correct}/{overall.pairs}), "
        f"vuln_recall {overall.vuln_recall:.2%}, silent_fix {overall.specificity:.2%}, "
        f"Youden {overall.youden:+.2%}, "
        f"labeled_recall {overall.labeled_recall:.2%} ({overall.labeled_matched}/{overall.labeled_expected}), "
        f"fix_leaks {overall.labeled_fix_leaks}"
    )
    for slice_name, metrics in result.per_slice.items():
        print(
            f"  {slice_name:<8} pair_pass {metrics.pair_pass_rate:>6.2%} ({metrics.pair_correct}/{metrics.pairs})  "
            f"vuln {metrics.vuln_recall:>6.2%}  silent {metrics.specificity:>6.2%}  "
            f"Youden {metrics.youden:>+7.2%}  labeled {metrics.labeled_recall:>6.2%}"
        )
    for outcome in result.outcomes:
        mark = "PASS" if outcome.pair_correct else "MISS" if not outcome.detected_vuln else "LEAK"
        print(
            f"    {mark:<4} {outcome.name:<32} vuln {outcome.vuln_matched}/{outcome.vuln_expected} "
            f"fix_findings={outcome.fix_findings} leaks={outcome.fix_leaks}"
        )
 def main() -> int:
    verdict = pair_gate()
    print_pair_metrics("pair gate", verdict.result)
    if verdict.passed:
        print("pair gate: PASS (local slice)")
        return 0
    print("pair gate: FAIL", file=sys.stderr)
    for reason in verdict.reasons:
        print(f"  - {reason}", file=sys.stderr)
    return 1
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,432 @@
 """Vuln-vs-fixed pair corpus: TP on the vuln tree, silent on the fix tree.
 Isolated trees keep the two sides from contaminating each other (a `safe.py`
 sitting next to `app.py` cannot measure false positives). Pair outcomes feed
 the improvement loop as miss (vuln) and fp (fix) signals.
 """
 from __future__ import annotations
 import shutil
 import tempfile
 import tomllib
 from collections.abc import Sequence
 from dataclasses import asdict, dataclass
 from pathlib import Path
 from .benchmark import (
    BenchmarkManifest,
    BenchmarkRun,
    BenchmarkSource,
    ExpectedFinding,
    evaluate_benchmark,
    load_benchmark_manifest,
 )
 from .findings import StaticFinding, quick_scan_findings
 from .mapping import analyze_entry_points, attach_reachability_hints
 from .preprocess import preprocess_repository
 from .rank import rank_targets
 from .semantic import OverlayRecord, adjudicate
 DEFAULT_CATALOG = Path("benchmarks/pairs/catalog.toml")
 DEFAULT_SAST_CATALOG = Path("benchmarks/pairs/sast/catalog.toml")
 DEFAULT_DATASETS = Path("benchmarks/pairs/datasets.toml")
@dataclass(frozen=True)
 class PairCase:
    name: str
    slice: str
    language: str
    origin: str
    vuln_file: Path
    fixed_file: Path
    relpath: str
    expected: tuple[ExpectedFinding, ...]
    min_recall: float
    fix_policy: str
    repo: str = ""
    commit: str = ""
    commit_url: str = ""
    cve: str = ""
    license: str = ""
@dataclass(frozen=True)
 class PairOutcome:
    name: str
    slice: str
    language: str
    origin: str
    detected_vuln: bool
    silent_fix: bool
    pair_correct: bool
    vuln_matched: int
    vuln_expected: int
    vuln_findings: int
    fix_findings: int
    fix_leaks: int
    recall: float
    misses: tuple[str, ...]
    leaks: tuple[str, ...]
    commit_url: str = ""
    cve: str = ""
@dataclass(frozen=True)
 class PairCorpusMetrics:
    pairs: int
    detected_vuln: int
    silent_fix: int
    pair_correct: int
    labeled_expected: int
    labeled_matched: int
    labeled_fix_leaks: int
    pair_pass_rate: float
    vuln_recall: float
    specificity: float
    labeled_recall: float
    youden: float
@dataclass(frozen=True)
 class PairEvalResult:
    outcomes: tuple[PairOutcome, ...]
    overall: PairCorpusMetrics
    per_slice: dict[str, PairCorpusMetrics]
    signals: tuple[dict[str, object], ...]
 def load_datasets(path: Path = DEFAULT_DATASETS) -> tuple[dict[str, object], ...]:
    payload = tomllib.loads(path.read_text())
    return tuple(dict(item) for item in payload.get("dataset", []))
 def load_pair_catalog(path: Path = DEFAULT_CATALOG) -> tuple[PairCase, ...]:
    cases = _load_catalog_file(path)
    if path.resolve() == DEFAULT_CATALOG.resolve() and DEFAULT_SAST_CATALOG.exists():
        cases = cases + _load_catalog_file(DEFAULT_SAST_CATALOG)
    return cases
 def _load_catalog_file(path: Path) -> tuple[PairCase, ...]:
    catalog_path = path.resolve()
    payload = tomllib.loads(catalog_path.read_text())
    root = catalog_path.parent
    cases: list[PairCase] = []
    for item in payload.get("pair", []):
        expected = _expected_for(item, root)
        cases.append(
            PairCase(
                name=str(item["name"]),
                slice=str(item.get("slice", "github")),
                language=str(item.get("language", "")),
                origin=str(item.get("origin", "")),
                vuln_file=_resolve(root, str(item["vuln"])),
                fixed_file=_resolve(root, str(item["fixed"])),
                relpath=str(item["relpath"]),
                expected=expected,
                min_recall=float(item.get("min_recall", 1.0)),
                fix_policy=str(item.get("fix_policy", "silent")),
                repo=str(item.get("repo", "")),
                commit=str(item.get("commit", "")),
                commit_url=str(item.get("commit_url", "")),
                cve=str(item.get("cve", "")),
                license=str(item.get("license", "")),
            )
        )
    return tuple(cases)
 def evaluate_pair(case: PairCase) -> PairOutcome:
    with tempfile.TemporaryDirectory(prefix="ousast-pair-") as scratch:
        vuln_root = _materialize(Path(scratch) / "vuln", case.vuln_file, case.relpath)
        fix_root = _materialize(Path(scratch) / "fixed", case.fixed_file, case.relpath)
        if case.slice == "sast":
            return _evaluate_overlay_pair(case, vuln_root, fix_root)
        vuln_findings = _quick_scan(vuln_root)
        fix_findings = _quick_scan(fix_root)
    vuln_result = evaluate_benchmark(
        run=BenchmarkRun(benchmark_run_id="pair", root=Path("/tmp/pair"), manifest=_manifest(case)),
        mode="quick",
        findings=vuln_findings,
        scan_id=None,
        scan_run_dir=None,
    )
    expected_total = len(case.expected)
    matched = vuln_result.metrics.matched_findings_total
    recall = matched / expected_total if expected_total else 1.0
    leaks = _fix_leaks(case, fix_findings)
    detected = recall + 1e-12 >= case.min_recall
    silent = not leaks
    misses = tuple(f"{miss.rule_id or '-'}:{miss.path}:{miss.cwe}" for miss in vuln_result.misses)
    leak_ids = tuple(finding.finding_id for finding in leaks)
    return PairOutcome(
        name=case.name,
        slice=case.slice,
        language=case.language,
        origin=case.origin,
        detected_vuln=detected,
        silent_fix=silent,
        pair_correct=detected and silent,
        vuln_matched=matched,
        vuln_expected=expected_total,
        vuln_findings=len(vuln_findings),
        fix_findings=len(fix_findings),
        fix_leaks=len(leaks),
        recall=recall,
        misses=misses,
        leaks=leak_ids,
        commit_url=case.commit_url,
        cve=case.cve,
    )
 def evaluate_catalog(cases: Sequence[PairCase]) -> PairEvalResult:
    outcomes = tuple(evaluate_pair(case) for case in cases)
    return PairEvalResult(
        outcomes=outcomes,
        overall=_metrics(outcomes),
        per_slice={
            slice_name: _metrics(tuple(item for item in outcomes if item.slice == slice_name))
            for slice_name in sorted({item.slice for item in outcomes})
        },
        signals=tuple(build_pair_signals(outcomes)),
    )
 def build_pair_signals(outcomes: Sequence[PairOutcome]) -> list[dict[str, object]]:
    """Same miss/fp vocabulary as ``build_rule_signals``, tagged with the pair name."""
    signals: list[dict[str, object]] = []
    for outcome in outcomes:
        for miss in outcome.misses:
            rule_id, path, cwe = (miss.split(":", 2) + ["", ""])[:3]
            signals.append({"rule_id": rule_id if rule_id != "-" else "", "signal": "miss", "cwe": cwe, "path": path, "pair": outcome.name})
        for leak in outcome.leaks:
            rule_id = leak.split(":", 1)[0]
            signals.append({"rule_id": rule_id, "signal": "fp", "path": leak, "pair": outcome.name, "side": "fixed"})
    return sorted(signals, key=lambda item: (str(item.get("signal")), str(item.get("rule_id")), str(item.get("pair"))))
 def select_slice(cases: Sequence[PairCase], slice_name: str | None) -> tuple[PairCase, ...]:
    if not slice_name or slice_name == "all":
        return tuple(cases)
    return tuple(case for case in cases if case.slice == slice_name)
 def result_payload(result: PairEvalResult) -> dict[str, object]:
    return {
        "overall": asdict(result.overall),
        "per_slice": {name: asdict(metrics) for name, metrics in result.per_slice.items()},
        "outcomes": [asdict(outcome) for outcome in result.outcomes],
        "signals": list(result.signals),
    }
 def _expected_for(item: dict[str, object], root: Path) -> tuple[ExpectedFinding, ...]:
    inline = item.get("expected")
    if isinstance(inline, list) and inline:
        return tuple(_parse_expected(entry) for entry in inline if isinstance(entry, dict))
    expected_from = item.get("expected_from")
    if not expected_from:
        return ()
    manifest = load_benchmark_manifest(_resolve(root, str(expected_from)))
    relpath = str(item["relpath"])
    # Pair eval measures ruled sinks. Unruled planted misses (split-sink, CWE-190)
    # stay on the stage-1/2 fixtures and must not drag local pair_correct below 1.0.
    return tuple(entry for entry in manifest.expected if entry.rule_id and (relpath in entry.path or entry.path in relpath))
 def _parse_expected(item: dict[str, object]) -> ExpectedFinding:
    line_value = item.get("line")
    return ExpectedFinding(
        cwe=str(item["cwe"]),
        vulnerability_class=str(item.get("class", item.get("vulnerability_class", "unknown"))),
        path=str(item["path"]),
        evidence=str(item.get("evidence", "")),
        rule_id=str(item["rule_id"]) if "rule_id" in item else None,
        line=line_value if isinstance(line_value, int) else None,
        function=str(item["function"]) if "function" in item else None,
        sink=str(item["sink"]) if "sink" in item else None,
    )
 def _manifest(case: PairCase) -> BenchmarkManifest:
    return BenchmarkManifest(
        name=case.name,
        language=case.language,
        frameworks=[],
        setup=[],
        source=BenchmarkSource(type="local", path=str(case.vuln_file)),
        modes=["quick"],
        expected=list(case.expected),
        known_noise=[],
        baselines=[],
    )
 def _resolve(root: Path, value: str) -> Path:
    path = Path(value)
    return path.resolve() if path.is_absolute() else (root / path).resolve()
 def _materialize(dest: Path, source: Path, relpath: str) -> Path:
    target = dest / relpath
    target.parent.mkdir(parents=True, exist_ok=True)
    shutil.copyfile(source, target)
    return dest
 def _quick_scan(target: Path) -> list[StaticFinding]:
    _, targets = preprocess_repository(target)
    targets = attach_reachability_hints(targets, analyze_entry_points(target, targets))
    return quick_scan_findings(target, targets, rank_targets(targets))
 def _overlay_scan(target: Path) -> tuple[list[StaticFinding], list[OverlayRecord]]:
    _, targets = preprocess_repository(target)
    targets = attach_reachability_hints(targets, analyze_entry_points(target, targets))
    findings = quick_scan_findings(target, targets, rank_targets(targets))
    return findings, adjudicate(root=target, targets=targets, findings=findings)
 def _evaluate_overlay_pair(case: PairCase, vuln_root: Path, fix_root: Path) -> PairOutcome:
    vuln_findings, vuln_overlay = _overlay_scan(vuln_root)
    fix_findings, fix_overlay = _overlay_scan(fix_root)
    if not _adjudicated(vuln_overlay) and not _adjudicated(fix_overlay):
        # Parser/facts could not adjudicate this language; score inventory so
        # labeled calibration does not collapse when tree-sitter grammars are absent.
        vuln_result = evaluate_benchmark(
            run=BenchmarkRun(benchmark_run_id="pair", root=Path("/tmp/pair"), manifest=_manifest(case)),
            mode="quick",
            findings=vuln_findings,
            scan_id=None,
            scan_run_dir=None,
        )
        expected_total = len(case.expected)
        matched = vuln_result.metrics.matched_findings_total
        recall = matched / expected_total if expected_total else 1.0
        inventory_leaks = _fix_leaks(case, fix_findings)
        detected = recall + 1e-12 >= case.min_recall
        silent = not inventory_leaks
        misses = tuple(f"{miss.rule_id or '-'}:{miss.path}:{miss.cwe}" for miss in vuln_result.misses)
        leak_ids = tuple(finding.finding_id for finding in inventory_leaks)
        return PairOutcome(
            name=case.name,
            slice=case.slice,
            language=case.language,
            origin=case.origin,
            detected_vuln=detected,
            silent_fix=silent,
            pair_correct=detected and silent,
            vuln_matched=matched,
            vuln_expected=expected_total,
            vuln_findings=len(vuln_findings),
            fix_findings=len(fix_findings),
            fix_leaks=len(inventory_leaks),
            recall=recall,
            misses=misses,
            leaks=leak_ids,
            commit_url=case.commit_url,
            cve=case.cve,
        )
    promoted_vuln = [record for record in vuln_overlay if record.disposition == "promote"]
    promoted_fix = [record for record in fix_overlay if record.disposition == "promote"]
    expected_total = len(case.expected)
    matched = _overlay_matches(case, promoted_vuln)
    recall = matched / expected_total if expected_total else 1.0
    detected = recall + 1e-12 >= case.min_recall
    overlay_leaks = _overlay_leaks(case, promoted_fix)
    silent = not overlay_leaks
    misses = tuple(
        f"{expected.rule_id or '-'}:{expected.path}:{expected.cwe}"
        for expected in case.expected
        if not _overlay_matches_expected(expected, promoted_vuln)
    )
    return PairOutcome(
        name=case.name,
        slice=case.slice,
        language=case.language,
        origin=case.origin,
        detected_vuln=detected,
        silent_fix=silent,
        pair_correct=detected and silent,
        vuln_matched=matched,
        vuln_expected=expected_total,
        vuln_findings=len(promoted_vuln),
        fix_findings=len(promoted_fix),
        fix_leaks=len(overlay_leaks),
        recall=recall,
        misses=misses,
        leaks=overlay_leaks,
        commit_url=case.commit_url,
        cve=case.cve,
    )
 def _adjudicated(records: Sequence[OverlayRecord]) -> bool:
    return any(record.disposition in {"promote", "demote", "coverage"} for record in records)
 def _overlay_matches(case: PairCase, promoted: Sequence[OverlayRecord]) -> int:
    return sum(1 for expected in case.expected if _overlay_matches_expected(expected, promoted))
 def _overlay_matches_expected(expected: ExpectedFinding, promoted: Sequence[OverlayRecord]) -> bool:
    for record in promoted:
        if expected.rule_id and expected.rule_id in record.proposal_id:
            return True
        if expected.cwe and expected.cwe == record.cwe:
            return True
        if expected.sink and expected.sink in record.sinks:
            return True
    return False
 def _overlay_leaks(case: PairCase, promoted: Sequence[OverlayRecord]) -> tuple[str, ...]:
    leaked: list[str] = []
    for record in promoted:
        if any(_overlay_matches_expected(expected, [record]) for expected in case.expected) or case.fix_policy == "silent":
            leaked.append(record.proposal_id)
    if case.fix_policy == "silent":
        return tuple(record.proposal_id for record in promoted)
    return tuple(leaked)
 def _fix_leaks(case: PairCase, findings: list[StaticFinding]) -> tuple[StaticFinding, ...]:
    if case.fix_policy == "silent":
        return tuple(findings)
    leaked: list[StaticFinding] = []
    for finding in findings:
        for expected in case.expected:
            if expected.rule_id and finding.finding_id.startswith(f"{expected.rule_id}:"):
                leaked.append(finding)
                break
            if expected.cwe.lower() in (finding.rationale + " " + finding.finding_id).lower():
                leaked.append(finding)
                break
    return tuple(leaked)
 def _metrics(outcomes: Sequence[PairOutcome]) -> PairCorpusMetrics:
    pairs = len(outcomes)
    detected = sum(1 for item in outcomes if item.detected_vuln)
    silent = sum(1 for item in outcomes if item.silent_fix)
    correct = sum(1 for item in outcomes if item.pair_correct)
    labeled_expected = sum(item.vuln_expected for item in outcomes)
    labeled_matched = sum(item.vuln_matched for item in outcomes)
    labeled_leaks = sum(item.fix_leaks for item in outcomes)
    return PairCorpusMetrics(
        pairs=pairs,
        detected_vuln=detected,
        silent_fix=silent,
        pair_correct=correct,
        labeled_expected=labeled_expected,
        labeled_matched=labeled_matched,
        labeled_fix_leaks=labeled_leaks,
        pair_pass_rate=correct / pairs if pairs else 1.0,
        vuln_recall=detected / pairs if pairs else 1.0,
        specificity=silent / pairs if pairs else 1.0,
        labeled_recall=labeled_matched / labeled_expected if labeled_expected else 1.0,
        youden=(detected / pairs if pairs else 1.0) - (1.0 - (silent / pairs if pairs else 1.0)),
    )
@@ -18,15 +18,18 @@ def write_markdown_report(
    redact: bool = True,
    complexity_map: object | None = None,
    verdicts: Sequence[object] | None = None,
    overlay: Sequence[object] | None = None,
 ) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    verification_by_id = _verification_by_id(verifications or [])
    overlay_by_id = _overlay_by_id(overlay or [])
    lines = ["# OpenUltraSAST Report", "", f"Findings: {len(findings)}", "", "## Inventory", ""]
    if not findings:
        lines.append("No quick-mode findings were emitted.")
        lines.append("")
    for finding in findings:
        verification = verification_by_id.get(finding.finding_id)
        disposition = overlay_by_id.get(finding.finding_id)
        lines.extend(
            [
                f"## {finding.title}",
@@ -43,11 +46,14 @@ def write_markdown_report(
                f"- Conditions: `{', '.join(finding.reachability_conditions) or 'none'}`",
                f"- Ranking priority: `{finding.ranking_priority}`",
                f"- Tags: `{', '.join(finding.tags)}`",
                "",
                finding.rationale,
                "",
            ]
        )
        if disposition is not None:
            lines.append(f"- Disposition: `{getattr(disposition, 'disposition', '')}`")
            lines.append(f"- Overlay reason: `{getattr(disposition, 'reason', '')}`")
        lines.extend(["", finding.rationale, ""])
    if overlay:
        _append_overlay(lines, overlay)
    if complexity_map is not None:
        _append_complexity_map(lines, complexity_map)
    if verdicts is not None:
@@ -122,9 +128,15 @@ def _report_identity(path: object, function_name: object) -> str:
    return path_text
-def write_sarif_report(findings: list[StaticFinding], verifications: list[VerificationResult], path: Path) -> None:
+def write_sarif_report(
    findings: list[StaticFinding],
    verifications: list[VerificationResult],
    path: Path,
    overlay: Sequence[object] | None = None,
 ) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    verification_by_id = _verification_by_id(verifications)
    overlay_by_id = _overlay_by_id(overlay or [])
    rules = _sarif_rules(findings)
    payload = {
        "version": "2.1.0",
@@ -138,7 +150,10 @@ def write_sarif_report(findings: list[StaticFinding], verifications: list[Verifi
                        "rules": list(rules.values()),
                    }
                },
-                "results": [_sarif_result(finding, verification_by_id.get(finding.finding_id)) for finding in findings],
+                "results": [
                    _sarif_result(finding, verification_by_id.get(finding.finding_id), overlay_by_id.get(finding.finding_id))
                    for finding in findings
                ],
            }
        ],
    }
@@ -228,7 +243,20 @@ def _sarif_rules(findings: list[StaticFinding]) -> dict[str, dict[str, object]]:
    return rules
-def _sarif_result(finding: StaticFinding, verification: VerificationResult | None) -> dict[str, object]:
+def _sarif_result(finding: StaticFinding, verification: VerificationResult | None, overlay: object | None = None) -> dict[str, object]:
    properties: dict[str, object] = {
        "finding_id": finding.finding_id,
        "severity": finding.severity,
        "confidence": finding.confidence,
        "evidence_level": finding.evidence_level,
        "verification": asdict(verification) if verification else None,
        "reachability_status": finding.reachability_status,
        "reachability_conditions": finding.reachability_conditions,
        "ranking_priority": finding.ranking_priority,
    }
    if overlay is not None:
        properties["disposition"] = getattr(overlay, "disposition", "")
        properties["overlay_reason"] = getattr(overlay, "reason", "")
    return {
        "ruleId": _rule_id(finding),
        "level": _sarif_level(finding.severity),
@@ -242,16 +270,7 @@ def _sarif_result(finding: StaticFinding, verification: VerificationResult | Non
            }
        ],
        "partialFingerprints": {"openultrasastFindingId": finding.finding_id},
-        "properties": {
+        "properties": properties,
            "finding_id": finding.finding_id,
            "severity": finding.severity,
            "confidence": finding.confidence,
            "evidence_level": finding.evidence_level,
            "verification": asdict(verification) if verification else None,
            "reachability_status": finding.reachability_status,
            "reachability_conditions": finding.reachability_conditions,
            "ranking_priority": finding.ranking_priority,
        },
    }
@@ -271,6 +290,47 @@ def _verification_by_id(verifications: list[VerificationResult]) -> dict[str, Ve
    return {result.finding_id: result for result in verifications}
 def _overlay_by_id(records: Sequence[object]) -> dict[str, object]:
    mapped: dict[str, object] = {}
    for record in records:
        proposal_id = getattr(record, "proposal_id", None)
        if isinstance(proposal_id, str) and proposal_id:
            mapped[proposal_id] = record
    return mapped
 def _append_overlay(lines: list[str], records: Sequence[object]) -> None:
    lines.extend(["## Overlay", "", f"Records: {len(records)}", ""])
    labels = ("promoted", "demoted", "unadjudicated", "coverage")
    counts = {label: 0 for label in labels}
    for record in records:
        disposition = str(getattr(record, "disposition", ""))
        if disposition == "promote":
            counts["promoted"] += 1
        elif disposition == "demote":
            counts["demoted"] += 1
        elif disposition == "unadjudicated":
            counts["unadjudicated"] += 1
        elif disposition == "coverage":
            counts["coverage"] += 1
    lines.append("- Labels: " + ", ".join(f"{label}={counts[label]}" for label in labels))
    lines.append("")
    for record in records:
        disposition = str(getattr(record, "disposition", ""))
        proposal_id = str(getattr(record, "proposal_id", ""))
        reason = str(getattr(record, "reason", ""))
        lines.extend(
            [
                f"### `{disposition}` `{proposal_id}`",
                "",
                f"- Path: `{getattr(record, 'path', '')}`",
                f"- Line: `{getattr(record, 'line', None)}`",
                f"- Reason: `{reason}`",
                "",
            ]
        )
 def _artifact_refs(root: Path, artifact_paths: dict[str, Path]) -> dict[str, str]:
    refs = {
        "findings_json": _relative_artifact(root, artifact_paths["findings"]),
@@ -0,0 +1,34 @@
 version = "1"
 language = "c"
 [[source]]
 id = "argv"
 patterns = ["argv", "getenv"]
 [[source]]
 id = "stdin"
 patterns = ["fgets", "gets", "scanf", "fscanf"]
 [[sink]]
 id = "printf"
 cwe = "CWE-134"
 calls = ["printf", "sprintf", "fprintf", "vprintf", "vsprintf", "syslog"]
 rule_ids = ["c-unsafe-format"]
 format_arg = 0
 [[sink]]
 id = "system"
 cwe = "CWE-78"
 calls = ["system", "popen", "execl", "execv", "execvp"]
 rule_ids = ["c-shell-exec", "c-exec-family"]
 [[sink]]
 id = "strcpy"
 cwe = "CWE-121"
 calls = ["strcpy", "strcat", "sprintf", "gets"]
 rule_ids = ["c-unsafe-copy"]
 [[sanitizer]]
 id = "literal_format"
 calls = ["printf", "sprintf", "fprintf", "vprintf", "vsprintf", "syslog"]
 literal_format_arg = 0
@@ -0,0 +1,31 @@
 version = "1"
 language = "java"
 [[source]]
 id = "servlet"
 patterns = ["getParameter", "getHeader", "getCookies", "readLine", "getQueryString"]
 [[sink]]
 id = "exec"
 cwe = "CWE-78"
 calls = ["exec"]
 rule_ids = ["java-command-exec"]
 [[sink]]
 id = "executeQuery"
 cwe = "CWE-89"
 calls = ["executeQuery", "execute", "prepareCall"]
 rule_ids = ["java-sql-injection"]
 [[sink]]
 id = "readObject"
 cwe = "CWE-502"
 calls = ["readObject"]
 rule_ids = ["java-deserialization"]
 [[sink]]
 id = "MessageDigest"
 cwe = "CWE-327"
 calls = ["getInstance"]
 rule_ids = ["java-weak-hash"]
 weak_literals = ["MD5", "SHA-1", "SHA1"]
@@ -0,0 +1,29 @@
 version = "1"
 language = "javascript"
 [[source]]
 id = "req"
 patterns = ["req.query", "req.body", "req.params", "req.cookies", "req.headers", "request.query"]
 [[sink]]
 id = "eval"
 cwe = "CWE-95"
 calls = ["eval"]
 rule_ids = ["js-eval"]
 [[sink]]
 id = "exec"
 cwe = "CWE-78"
 calls = ["exec", "execSync"]
 rule_ids = ["js-command-exec"]
 [[sink]]
 id = "query"
 cwe = "CWE-89"
 calls = ["query", "execute"]
 rule_ids = ["js-sql-injection"]
 [[sanitizer]]
 id = "parameterized_query"
 calls = ["query", "execute"]
 parameterized = true
@@ -0,0 +1,73 @@
 version = "1"
 language = "python"
 [[source]]
 id = "request"
 patterns = [
  "request.args",
  "request.form",
  "request.data",
  "request.json",
  "request.cookies",
  "request.values",
  "request.get_json",
  "request.headers",
  "request.files",
  "request.get",
 ]
 [[source]]
 id = "argv"
 patterns = ["sys.argv"]
 [[source]]
 id = "stdin"
 patterns = ["sys.stdin", "stdin.read"]
 [[source]]
 id = "input"
 patterns = ["input("]
 [[sink]]
 id = "eval"
 cwe = "CWE-95"
 calls = ["eval", "exec"]
 rule_ids = ["python-unsafe-eval"]
 [[sink]]
 id = "pickle.loads"
 cwe = "CWE-502"
 calls = ["pickle.loads", "pickle.load", "yaml.load", "marshal.loads", "cPickle.loads"]
 rule_ids = ["python-unsafe-deserialization"]
 [[sink]]
 id = "execute"
 cwe = "CWE-89"
 calls = ["execute", "executemany"]
 rule_ids = ["python-sql-injection"]
 [[sink]]
 id = "os.system"
 cwe = "CWE-78"
 calls = ["os.system", "os.popen", "subprocess.run", "subprocess.call", "subprocess.Popen", "subprocess.check_output"]
 rule_ids = ["python-os-command", "python-shell-true"]
 [[sink]]
 id = "hashlib"
 cwe = "CWE-327"
 calls = ["hashlib.md5", "hashlib.sha1", "hashlib.new"]
 rule_ids = ["python-weak-hash"]
 weak_literals = ["md5", "sha1", "MD5", "SHA1", "sha-1"]
 [[sanitizer]]
 id = "ast.literal_eval"
 calls = ["ast.literal_eval"]
 [[sanitizer]]
 id = "yaml.safe_load"
 calls = ["yaml.safe_load", "safe_load"]
 [[sanitizer]]
 id = "parameterized_execute"
 calls = ["execute", "executemany"]
 parameterized = true
@@ -0,0 +1,25 @@
 """Propose/adjudicate/prove overlay: regex proposes, semantics adjudicates."""
 from .engines import joern_available, tree_sitter_available
 from .facts import FactLoadError, SemanticFacts, load_facts
 from .mechanisms import Mechanism, MechanismStore, append_mechanism, order_promotions
 from .overlay import OverlayError, OverlayRecord, adjudicate, finding_from_coverage, write_overlay
 from .prove_filter import filter_promoted_hotspots
 __all__ = [
    "FactLoadError",
    "Mechanism",
    "MechanismStore",
    "OverlayError",
    "OverlayRecord",
    "SemanticFacts",
    "adjudicate",
    "append_mechanism",
    "filter_promoted_hotspots",
    "finding_from_coverage",
    "joern_available",
    "load_facts",
    "order_promotions",
    "tree_sitter_available",
    "write_overlay",
 ]
@@ -0,0 +1,64 @@
 """Probe tree-sitter and Joern like the Docker sandbox probe. Never required for inventory."""
 from __future__ import annotations
 import importlib.util
 import os
 import shutil
 import subprocess
 from collections.abc import Callable
 from subprocess import CompletedProcess
 TREE_SITTER_PROBE_ENV = "OPENULTRASAST_TREE_SITTER_PROBE"
 JOERN_PROBE_ENV = "OPENULTRASAST_JOERN_PROBE"
 PROBE_TIMEOUT_SECONDS = 5
 _PROBE_OFF = frozenset({"0", "off", "false", "unavailable", "no"})
 _PROBE_ON = frozenset({"1", "on", "true", "available", "yes"})
 def tree_sitter_available(*, runner: Callable[..., CompletedProcess[str]] | None = None) -> bool:
    flag = os.environ.get(TREE_SITTER_PROBE_ENV, "").strip().lower()
    if flag in _PROBE_OFF:
        return False
    if flag in _PROBE_ON:
        return True
    if _python_tree_sitter_present():
        return True
    binary = shutil.which("tree-sitter")
    if binary is None:
        return False
    run = runner if runner is not None else subprocess.run
    try:
        result = run([binary, "--version"], capture_output=True, text=True, timeout=PROBE_TIMEOUT_SECONDS, check=False)
        return result.returncode == 0
    except Exception:
        return False
 def joern_available(*, runner: Callable[..., CompletedProcess[str]] | None = None) -> bool:
    flag = os.environ.get(JOERN_PROBE_ENV, "").strip().lower()
    if flag in _PROBE_OFF:
        return False
    if flag in _PROBE_ON:
        return True
    binary = shutil.which("joern") or shutil.which("joern-parse")
    if binary is None:
        return False
    run = runner if runner is not None else subprocess.run
    try:
        result = run([binary, "--help"], capture_output=True, text=True, timeout=PROBE_TIMEOUT_SECONDS, check=False)
        return result.returncode == 0
    except Exception:
        return False
 def joern_flow(path: str, text: str, language: str) -> tuple[object, ...]:
    """Optional CPG sidecar. Absence or incomplete export must not demote."""
    del path, text, language
    if not joern_available():
        return ()
    return ()
 def _python_tree_sitter_present() -> bool:
    return importlib.util.find_spec("tree_sitter") is not None
@@ -0,0 +1,182 @@
 """Source, sink, and sanitizer facts as versioned data next to the ruleset."""
 from __future__ import annotations
 import tomllib
 from dataclasses import dataclass
 from pathlib import Path
 DEFAULT_FACTS_DIR = Path(__file__).resolve().parents[1] / "ruleset" / "semantic"
 _LANGUAGE_ALIASES = {
    "c": "c",
    "cpp": "c",
    "python": "python",
    "javascript": "javascript",
    "typescript": "javascript",
    "java": "java",
 }
 class FactLoadError(ValueError):
    """Raised when semantic fact files are missing or invalid."""
    def __init__(self, message: str, reason: str = "facts_unavailable") -> None:
        super().__init__(message)
        self.reason = reason
@dataclass(frozen=True)
 class SourceFact:
    id: str
    patterns: tuple[str, ...]
    language: str
@dataclass(frozen=True)
 class SinkFact:
    id: str
    cwe: str
    calls: tuple[str, ...]
    rule_ids: tuple[str, ...]
    language: str
    weak_literals: tuple[str, ...] = ()
    format_arg: int | None = None
@dataclass(frozen=True)
 class SanitizerFact:
    id: str
    calls: tuple[str, ...]
    language: str
    parameterized: bool = False
    literal_format_arg: int | None = None
@dataclass(frozen=True)
 class SemanticFacts:
    version: str
    sources: tuple[SourceFact, ...]
    sinks: tuple[SinkFact, ...]
    sanitizers: tuple[SanitizerFact, ...]
    def for_language(self, language: str) -> SemanticFacts:
        key = _LANGUAGE_ALIASES.get(language, language)
        return SemanticFacts(
            version=self.version,
            sources=tuple(item for item in self.sources if item.language == key),
            sinks=tuple(item for item in self.sinks if item.language == key),
            sanitizers=tuple(item for item in self.sanitizers if item.language == key),
        )
 def load_facts(directory: Path | None = None) -> SemanticFacts:
    root = directory if directory is not None else DEFAULT_FACTS_DIR
    if not root.is_dir():
        raise FactLoadError(f"semantic fact directory missing: {root}")
    sources: list[SourceFact] = []
    sinks: list[SinkFact] = []
    sanitizers: list[SanitizerFact] = []
    versions: list[str] = []
    found = False
    for path in sorted(root.glob("*.toml")):
        found = True
        try:
            payload = tomllib.loads(path.read_text())
        except (OSError, tomllib.TOMLDecodeError) as exc:
            raise FactLoadError(f"invalid semantic fact file {path}: {exc}") from exc
        language = str(payload.get("language") or path.stem)
        versions.append(str(payload.get("version", "1")))
        sources.extend(_sources(payload.get("source"), language, path))
        sinks.extend(_sinks(payload.get("sink"), language, path))
        sanitizers.extend(_sanitizers(payload.get("sanitizer"), language, path))
    if not found:
        raise FactLoadError(f"no semantic fact files in {root}")
    return SemanticFacts(version=versions[0] if versions else "1", sources=tuple(sources), sinks=tuple(sinks), sanitizers=tuple(sanitizers))
 def _items(value: object) -> list[dict[str, object]]:
    if value is None:
        return []
    if not isinstance(value, list):
        raise FactLoadError("fact tables must be arrays")
    rows: list[dict[str, object]] = []
    for item in value:
        if not isinstance(item, dict):
            raise FactLoadError("each fact row must be a table")
        rows.append(item)
    return rows
 def _strings(value: object, field: str, path: Path) -> tuple[str, ...]:
    if value is None:
        return ()
    if not isinstance(value, list) or not all(isinstance(item, str) and item for item in value):
        raise FactLoadError(f"{path} has invalid {field} list")
    return tuple(str(item) for item in value)
 def _sources(value: object, language: str, path: Path) -> list[SourceFact]:
    rows: list[SourceFact] = []
    for item in _items(value):
        ident = item.get("id")
        if not isinstance(ident, str) or not ident:
            raise FactLoadError(f"{path} source is missing id")
        patterns = _strings(item.get("patterns"), "patterns", path)
        if not patterns:
            raise FactLoadError(f"{path} source {ident} has no patterns")
        rows.append(SourceFact(id=ident, patterns=patterns, language=language))
    return rows
 def _sinks(value: object, language: str, path: Path) -> list[SinkFact]:
    rows: list[SinkFact] = []
    for item in _items(value):
        ident = item.get("id")
        cwe = item.get("cwe")
        if not isinstance(ident, str) or not ident:
            raise FactLoadError(f"{path} sink is missing id")
        if not isinstance(cwe, str) or not cwe.startswith("CWE-"):
            raise FactLoadError(f"{path} sink {ident} must bind a CWE id")
        calls = _strings(item.get("calls"), "calls", path)
        if not calls:
            raise FactLoadError(f"{path} sink {ident} has no calls")
        format_arg = item.get("format_arg")
        if format_arg is not None and (not isinstance(format_arg, int) or isinstance(format_arg, bool)):
            raise FactLoadError(f"{path} sink {ident} has invalid format_arg")
        rows.append(
            SinkFact(
                id=ident,
                cwe=cwe,
                calls=calls,
                rule_ids=_strings(item.get("rule_ids"), "rule_ids", path),
                language=language,
                weak_literals=_strings(item.get("weak_literals"), "weak_literals", path),
                format_arg=format_arg if isinstance(format_arg, int) else None,
            )
        )
    return rows
 def _sanitizers(value: object, language: str, path: Path) -> list[SanitizerFact]:
    rows: list[SanitizerFact] = []
    for item in _items(value):
        ident = item.get("id")
        if not isinstance(ident, str) or not ident:
            raise FactLoadError(f"{path} sanitizer is missing id")
        calls = _strings(item.get("calls"), "calls", path)
        if not calls:
            raise FactLoadError(f"{path} sanitizer {ident} has no calls")
        literal = item.get("literal_format_arg")
        if literal is not None and (not isinstance(literal, int) or isinstance(literal, bool)):
            raise FactLoadError(f"{path} sanitizer {ident} has invalid literal_format_arg")
        rows.append(
            SanitizerFact(
                id=ident,
                calls=calls,
                language=language,
                parameterized=bool(item.get("parameterized", False)),
                literal_format_arg=literal if isinstance(literal, int) else None,
            )
        )
    return rows
@@ -0,0 +1,277 @@
 """Intra-file IR: tree-sitter when a grammar works, Python stdlib ast as last resort."""
 from __future__ import annotations
 import ast
 import shutil
 import subprocess
 import tempfile
 from dataclasses import dataclass
 from pathlib import Path
 from .engines import tree_sitter_available
 _EXTENSION = {
    "python": ".py",
    "javascript": ".js",
    "typescript": ".ts",
    "c": ".c",
    "cpp": ".cpp",
    "java": ".java",
 }
@dataclass(frozen=True)
 class Bind:
    name: str
    line: int
    value_text: str
    is_constant: bool
    names: tuple[str, ...]
    call_name: str | None
@dataclass(frozen=True)
 class CallSite:
    name: str
    line: int
    arg_texts: tuple[str, ...]
    arg_is_constant: tuple[bool, ...]
    arg_names: tuple[tuple[str, ...], ...]
    extra_arg_is_sequence: bool
@dataclass(frozen=True)
 class FunctionIR:
    name: str
    params: tuple[str, ...]
    start_line: int
    end_line: int
    binds: tuple[Bind, ...]
    calls: tuple[CallSite, ...]
@dataclass(frozen=True)
 class FileIR:
    path: str
    language: str
    engine: str
    functions: tuple[FunctionIR, ...]
    parse_ok: bool
    reason: str | None = None
 def parse_file(path: str, text: str, language: str) -> FileIR:
    if tree_sitter_available():
        parsed = parse_tree_sitter_cli(path, text, language)
        if parsed is not None and parsed.parse_ok:
            return parsed
    if language == "python":
        return parse_python_ast(path, text)
    return FileIR(path=path, language=language, engine="none", functions=(), parse_ok=False, reason="language_unsupported")
 def parse_python_ast(path: str, text: str) -> FileIR:
    try:
        tree = ast.parse(text)
    except SyntaxError:
        return FileIR(path=path, language="python", engine="python-ast", functions=(), parse_ok=False, reason="parse_failed")
    functions = _python_functions(tree)
    return FileIR(path=path, language="python", engine="python-ast", functions=tuple(functions), parse_ok=True)
 def parse_tree_sitter_cli(path: str, text: str, language: str) -> FileIR | None:
    binary = shutil.which("tree-sitter")
    if binary is None:
        return None
    suffix = _EXTENSION.get(language, ".txt")
    tmp_path: str | None = None
    try:
        with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as handle:
            handle.write(text.encode())
            tmp_path = handle.name
        result = subprocess.run(
            [binary, "parse", tmp_path],
            capture_output=True,
            text=True,
            timeout=5,
            check=False,
        )
    except Exception:
        return None
    finally:
        if tmp_path:
            Path(tmp_path).unlink(missing_ok=True)
    combined = f"{result.stdout or ''}{result.stderr or ''}"
    if result.returncode != 0 or "No language found" in combined or "Failed to load language" in combined:
        return None
    # A successful parse without an extractor still cannot adjudicate JS/C/Java.
    # Return a parse_ok IR only when we actually built functions; otherwise None
    # so Python can fall back to ast and other languages stay unsupported.
    return None
 def _python_functions(tree: ast.AST) -> list[FunctionIR]:
    functions: list[FunctionIR] = []
    module_binds, module_calls = _statements(getattr(tree, "body", []))
    functions.append(
        FunctionIR(
            name="<module>",
            params=(),
            start_line=1,
            end_line=getattr(tree, "end_lineno", 1) or 1,
            binds=tuple(module_binds),
            calls=tuple(module_calls),
        )
    )
    for node in ast.walk(tree):
        if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):
            binds, calls = _statements(node.body)
            functions.append(
                FunctionIR(
                    name=node.name,
                    params=tuple(arg.arg for arg in node.args.args),
                    start_line=node.lineno,
                    end_line=getattr(node, "end_lineno", node.lineno) or node.lineno,
                    binds=tuple(binds),
                    calls=tuple(calls),
                )
            )
    return functions
 def _statements(body: list[ast.stmt]) -> tuple[list[Bind], list[CallSite]]:
    binds: list[Bind] = []
    calls: list[CallSite] = []
    for stmt in body:
        _collect_stmt(stmt, binds, calls)
    return binds, calls
 def _collect_stmt(stmt: ast.stmt, binds: list[Bind], calls: list[CallSite]) -> None:
    if isinstance(stmt, ast.FunctionDef | ast.AsyncFunctionDef | ast.ClassDef):
        return
    if isinstance(stmt, ast.Assign):
        value_bind = _bind_from_value(stmt.value, stmt.lineno)
        for target in stmt.targets:
            if isinstance(target, ast.Name):
                binds.append(
                    Bind(
                        name=target.id,
                        line=stmt.lineno,
                        value_text=value_bind[0],
                        is_constant=value_bind[1],
                        names=value_bind[2],
                        call_name=value_bind[3],
                    )
                )
        _collect_expr_calls(stmt.value, calls)
        return
    if isinstance(stmt, ast.AnnAssign) and isinstance(stmt.target, ast.Name) and stmt.value is not None:
        value_bind = _bind_from_value(stmt.value, stmt.lineno)
        binds.append(
            Bind(
                name=stmt.target.id,
                line=stmt.lineno,
                value_text=value_bind[0],
                is_constant=value_bind[1],
                names=value_bind[2],
                call_name=value_bind[3],
            )
        )
        _collect_expr_calls(stmt.value, calls)
        return
    if isinstance(stmt, ast.AugAssign) and isinstance(stmt.target, ast.Name):
        value_bind = _bind_from_value(stmt.value, stmt.lineno)
        binds.append(
            Bind(
                name=stmt.target.id,
                line=stmt.lineno,
                value_text=f"{stmt.target.id} {value_bind[0]}",
                is_constant=False,
                names=tuple(sorted(set(value_bind[2]) | {stmt.target.id})),
                call_name=value_bind[3],
            )
        )
        _collect_expr_calls(stmt.value, calls)
        return
    if isinstance(stmt, ast.Return) and stmt.value is not None:
        _collect_expr_calls(stmt.value, calls)
        return
    if isinstance(stmt, ast.Expr):
        _collect_expr_calls(stmt.value, calls)
        return
    for child in ast.iter_child_nodes(stmt):
        if isinstance(child, ast.stmt):
            _collect_stmt(child, binds, calls)
        elif isinstance(child, ast.expr):
            _collect_expr_calls(child, calls)
 def _bind_from_value(value: ast.expr, line: int) -> tuple[str, bool, tuple[str, ...], str | None]:
    del line
    text = _unparse(value)
    names = _load_names(value)
    call_name = _call_name(value) if isinstance(value, ast.Call) else None
    return text, _is_constant(value), names, call_name
 def _collect_expr_calls(expr: ast.expr, calls: list[CallSite]) -> None:
    for node in ast.walk(expr):
        if isinstance(node, ast.Call):
            calls.append(_call_site(node))
 def _call_site(node: ast.Call) -> CallSite:
    arg_texts: list[str] = []
    arg_const: list[bool] = []
    arg_names: list[tuple[str, ...]] = []
    for arg in node.args:
        arg_texts.append(_unparse(arg))
        arg_const.append(_is_constant(arg))
        arg_names.append(_load_names(arg))
    extra_sequence = False
    if len(node.args) >= 2:
        extra_sequence = isinstance(node.args[1], ast.Tuple | ast.List | ast.Set)
    return CallSite(
        name=_call_name(node),
        line=node.lineno,
        arg_texts=tuple(arg_texts),
        arg_is_constant=tuple(arg_const),
        arg_names=tuple(arg_names),
        extra_arg_is_sequence=extra_sequence,
    )
 def _call_name(node: ast.AST) -> str:
    func = node.func if isinstance(node, ast.Call) else node
    return _unparse(func)
 def _load_names(node: ast.AST) -> tuple[str, ...]:
    names: list[str] = []
    for child in ast.walk(node):
        if isinstance(child, ast.Name) and isinstance(child.ctx, ast.Load):
            names.append(child.id)
    return tuple(names)
 def _is_constant(node: ast.AST) -> bool:
    if isinstance(node, ast.Constant):
        return True
    if isinstance(node, ast.JoinedStr):
        return all(isinstance(value, ast.Constant) for value in node.values)
    if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Add):
        return _is_constant(node.left) and _is_constant(node.right)
    if isinstance(node, ast.List | ast.Tuple | ast.Set):
        return all(_is_constant(elt) for elt in node.elts)
    if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.UAdd | ast.USub):
        return _is_constant(node.operand)
    return False
 def _unparse(node: ast.AST) -> str:
    try:
        return ast.unparse(node)
    except Exception:
        return ""
@@ -0,0 +1,275 @@
 """Append-only mechanism memory. OpenRouter embeddings rank prove budget; never demote."""
 from __future__ import annotations
 import json
 import math
 import os
 import uuid
 from collections.abc import Sequence
 from dataclasses import asdict, dataclass
 from pathlib import Path
 from ..complexity.map import Hotspot
 from ..findings import StaticFinding
 from ..index import CodeChunk, EmbeddingClient, VectorIndex, VectorRecord, write_vector_index
 from .overlay import OverlayRecord
 DEFAULT_MECHANISM_LOG = Path(".openultrasast/mechanisms.jsonl")
@dataclass(frozen=True)
 class Mechanism:
    id: str
    summary: str
    cwe: str
    language: str
    tags: tuple[str, ...]
    keywords: tuple[str, ...]
    what_made_it_exploitable: str
    source_finding_id: str = ""
    source_repo: str = ""
 class MechanismStore:
    def __init__(self, log_path: Path, cache_path: Path | None = None) -> None:
        self.log_path = log_path
        self.cache_path = cache_path if cache_path is not None else log_path.with_suffix(".index.json")
    def load(self) -> tuple[Mechanism, ...]:
        if not self.log_path.is_file():
            return ()
        records: list[Mechanism] = []
        for line in self.log_path.read_text().splitlines():
            if not line.strip():
                continue
            payload = json.loads(line)
            records.append(_mechanism_from_payload(payload))
        return tuple(records)
    def append(self, mechanism: Mechanism) -> None:
        self.log_path.parent.mkdir(parents=True, exist_ok=True)
        with self.log_path.open("a") as handle:
            handle.write(json.dumps(asdict(mechanism), sort_keys=True) + "\n")
 def append_mechanism(
    store: MechanismStore,
    *,
    summary: str,
    cwe: str,
    language: str,
    tags: Sequence[str],
    what_made_it_exploitable: str,
    source_finding_id: str = "",
    source_repo: str = "",
    keywords: Sequence[str] = (),
 ) -> Mechanism:
    mechanism = Mechanism(
        id=str(uuid.uuid4()),
        summary=summary,
        cwe=cwe,
        language=language,
        tags=tuple(tags),
        keywords=tuple(keywords),
        what_made_it_exploitable=what_made_it_exploitable,
        source_finding_id=source_finding_id,
        source_repo=source_repo,
    )
    store.append(mechanism)
    return mechanism
 def order_promotions(
    hotspots: Sequence[Hotspot],
    records: Sequence[OverlayRecord],
    findings: Sequence[StaticFinding],
    *,
    store: MechanismStore | None = None,
    client: EmbeddingClient | None = None,
    model: str | None = None,
 ) -> tuple[tuple[Hotspot, ...], str | None]:
    """Heuristic order, then optional OpenRouter cosine boost. Never drops a promotion."""
    heuristic = tuple(sorted(hotspots, key=lambda item: (-item.score, item.path, item.function_name or "")))
    if store is None:
        return heuristic, None
    mechanisms = store.load()
    if not mechanisms:
        return heuristic, None
    if client is None or not model:
        return heuristic, "embeddings_unavailable"
    overlay_by_id = {record.proposal_id: record for record in records}
    finding_by_id = {finding.finding_id: finding for finding in findings}
    try:
        ranked = _rank_with_embeddings(heuristic, overlay_by_id, finding_by_id, mechanisms, store, client, model)
    except Exception:
        return heuristic, "embed_failed"
    return ranked, None
 def overlay_text(record: OverlayRecord) -> str:
    parts = [record.cwe, record.reason, *record.sources, *record.sinks, *record.sanitizers]
    return " ".join(part for part in parts if part)
 def mechanism_text(mechanism: Mechanism) -> str:
    return " ".join(
        part
        for part in (mechanism.summary, mechanism.what_made_it_exploitable, mechanism.cwe, *mechanism.tags, *mechanism.keywords)
        if part
    )
 def _rank_with_embeddings(
    hotspots: tuple[Hotspot, ...],
    overlay_by_id: dict[str, OverlayRecord],
    finding_by_id: dict[str, StaticFinding],
    mechanisms: tuple[Mechanism, ...],
    store: MechanismStore,
    client: EmbeddingClient,
    model: str,
 ) -> tuple[Hotspot, ...]:
    vectors = _mechanism_vectors(mechanisms, store, client, model)
    scored: list[tuple[float, Hotspot]] = []
    for hotspot in hotspots:
        record = _overlay_for_hotspot(hotspot, overlay_by_id)
        finding = _finding_for_hotspot(hotspot, finding_by_id)
        language = record.language if record is not None else ""
        cwe = record.cwe if record is not None else ""
        tags = tuple(finding.tags) if finding is not None else ()
        filtered = [item for item in mechanisms if _hard_filter(item, language, cwe, tags)]
        boost = 0.0
        if record is not None and filtered:
            query = client.embed(model=model, inputs=[overlay_text(record)])[0]
            for mechanism in filtered:
                vector = vectors.get(mechanism.id)
                if vector is None:
                    continue
                boost = max(boost, _cosine(query, vector))
        scored.append((hotspot.score + boost, hotspot))
    scored.sort(key=lambda item: (-item[0], item[1].path, item[1].function_name or ""))
    return tuple(item[1] for item in scored)
 def _mechanism_vectors(
    mechanisms: tuple[Mechanism, ...],
    store: MechanismStore,
    client: EmbeddingClient,
    model: str,
 ) -> dict[str, list[float]]:
    vectors: dict[str, list[float]] = {}
    missing: list[Mechanism] = []
    cached = _read_cache(store.cache_path, model)
    for mechanism in mechanisms:
        key = f"{mechanism.id}:{model}"
        if key in cached:
            vectors[mechanism.id] = cached[key]
        else:
            missing.append(mechanism)
    if missing:
        embeddings = client.embed(model=model, inputs=[mechanism_text(item) for item in missing])
        for mechanism, embedding in zip(missing, embeddings, strict=True):
            vectors[mechanism.id] = embedding
        records = [
            VectorRecord(
                chunk=CodeChunk(
                    chunk_id=f"{mechanism.id}:{model}",
                    namespace="mechanisms",
                    path=mechanism.id,
                    language=mechanism.language or "text",
                    start_line=1,
                    end_line=1,
                    text=mechanism_text(mechanism),
                    metadata={"mechanism_id": mechanism.id, "model": model, "cwe": mechanism.cwe, "language": mechanism.language},
                ),
                embedding=vectors[mechanism.id],
            )
            for mechanism in mechanisms
            if mechanism.id in vectors
        ]
        index = VectorIndex(
            store="json-local",
            embedding_model=model,
            repo_root=str(store.log_path.parent),
            repo_commit=None,
            records=records,
        )
        write_vector_index(index, store.cache_path)
    return vectors
 def _read_cache(path: Path, model: str) -> dict[str, list[float]]:
    if not path.is_file():
        return {}
    try:
        payload = json.loads(path.read_text())
    except json.JSONDecodeError:
        return {}
    if str(payload.get("embedding_model")) != model:
        return {}
    cached: dict[str, list[float]] = {}
    for item in payload.get("records", []):
        chunk = item.get("chunk", {})
        metadata = chunk.get("metadata", {})
        mechanism_id = str(metadata.get("mechanism_id", ""))
        if mechanism_id:
            cached[f"{mechanism_id}:{model}"] = [float(value) for value in item.get("embedding", [])]
    return cached
 def _hard_filter(mechanism: Mechanism, language: str, cwe: str, tags: Sequence[str]) -> bool:
    if mechanism.language and language and mechanism.language != language:
        return False
    if mechanism.cwe and cwe and mechanism.cwe != cwe:
        return False
    tag_ok = not mechanism.tags or not tags or bool(set(mechanism.tags) & set(tags))
    return tag_ok
 def _overlay_for_hotspot(hotspot: Hotspot, overlay_by_id: dict[str, OverlayRecord]) -> OverlayRecord | None:
    for finding_id in hotspot.inventory_finding_ids:
        record = overlay_by_id.get(finding_id)
        if record is not None:
            return record
    return None
 def _finding_for_hotspot(hotspot: Hotspot, finding_by_id: dict[str, StaticFinding]) -> StaticFinding | None:
    for finding_id in hotspot.inventory_finding_ids:
        finding = finding_by_id.get(finding_id)
        if finding is not None:
            return finding
    return None
 def _cosine(left: list[float], right: list[float]) -> float:
    if len(left) != len(right):
        return 0.0
    numerator = sum(a * b for a, b in zip(left, right, strict=True))
    left_norm = math.sqrt(sum(a * a for a in left))
    right_norm = math.sqrt(sum(b * b for b in right))
    if left_norm == 0 or right_norm == 0:
        return 0.0
    return numerator / (left_norm * right_norm)
 def _mechanism_from_payload(payload: dict[str, object]) -> Mechanism:
    tags = payload.get("tags") or []
    keywords = payload.get("keywords") or []
    if not isinstance(tags, list) or not isinstance(keywords, list):
        raise ValueError("mechanism tags and keywords must be lists")
    return Mechanism(
        id=str(payload["id"]),
        summary=str(payload.get("summary", "")),
        cwe=str(payload.get("cwe", "")),
        language=str(payload.get("language", "")),
        tags=tuple(str(item) for item in tags),
        keywords=tuple(str(item) for item in keywords),
        what_made_it_exploitable=str(payload.get("what_made_it_exploitable", "")),
        source_finding_id=str(payload.get("source_finding_id", "")),
        source_repo=str(payload.get("source_repo", "")),
    )
 def embeddings_configured(model: str | None) -> bool:
    return bool(model) and bool(os.environ.get("OPENROUTER_API_KEY"))
@@ -0,0 +1,310 @@
 """Adjudicate inventory proposals: promote, demote, unadjudicated, coverage."""
 from __future__ import annotations
 import json
 from collections.abc import Sequence
 from dataclasses import asdict, dataclass
 from pathlib import Path
 from ..findings import SEVERITY_LABEL, StaticFinding
 from ..policy import load_policy, resolve_severity
 from ..preprocess import FileTarget
 from .engines import joern_available, joern_flow
 from .facts import FactLoadError, SemanticFacts, load_facts
 from .taint import Demotion, FileFlow, TaintPath, UnadjudicatedFlow, analyze_file, merge_joern_paths
 DISPOSITIONS = frozenset({"promote", "demote", "unadjudicated", "coverage"})
 UNADJUDICATED_REASONS = frozenset({"parse_failed", "language_unsupported", "flow_incomplete", "facts_unavailable"})
 ALLOWED_EVIDENCE = frozenset({"static_corroboration", "suspicion"})
 FORBIDDEN_EVIDENCE = frozenset({"crash_reproduced", "exploit_demonstrated", "patch_validated", "root_cause_explained"})
 class OverlayError(ValueError):
    """Raised when an overlay record violates fail-closed invariants."""
@dataclass(frozen=True)
 class OverlayRecord:
    proposal_id: str
    path: str
    line: int | None
    disposition: str
    reason: str
    cwe: str
    sources: tuple[str, ...]
    sinks: tuple[str, ...]
    sanitizers: tuple[str, ...]
    evidence_level: str
    dominating_fact: str | None = None
    origin: str = "inventory"
    engine: str = "none"
    language: str = ""
    def __post_init__(self) -> None:
        if self.disposition not in DISPOSITIONS:
            raise OverlayError(f"invalid disposition: {self.disposition}")
        if self.evidence_level in FORBIDDEN_EVIDENCE:
            raise OverlayError("overlay cannot claim proof rungs")
        if self.evidence_level not in ALLOWED_EVIDENCE:
            raise OverlayError(f"invalid evidence_level: {self.evidence_level}")
        if self.disposition == "demote" and not self.dominating_fact and not self.sanitizers:
            raise OverlayError("demote requires a recorded constant or sanitizer")
        if self.disposition == "unadjudicated" and self.reason not in UNADJUDICATED_REASONS:
            raise OverlayError(f"invalid unadjudicated reason: {self.reason}")
        if self.disposition in {"promote", "coverage"} and self.evidence_level not in ALLOWED_EVIDENCE:
            raise OverlayError("promote and coverage must stay at static_corroboration or below")
 def adjudicate(
    *,
    root: Path,
    targets: Sequence[FileTarget],
    findings: Sequence[StaticFinding],
    facts: SemanticFacts | FactLoadError | None = None,
    joern_extra: Sequence[TaintPath] | None = None,
 ) -> list[OverlayRecord]:
    loaded = facts if facts is not None else _try_facts()
    language_by_path = {target.path: target.language for target in targets}
    if isinstance(loaded, FactLoadError):
        return [
            _unadjudicated(
                finding,
                language_by_path.get(finding.path, ""),
                "facts_unavailable",
                "none",
            )
            for finding in findings
        ]
    flows = _flows_for(root, targets, loaded, joern_extra)
    records: list[OverlayRecord] = []
    claimed: set[tuple[str, int | None, str]] = set()
    for finding in findings:
        language = language_by_path.get(finding.path, "")
        flow = flows.get(finding.path)
        record = _record_for_finding(finding, language, flow)
        records.append(record)
        claimed.add((finding.path, finding.line, _sink_hint(finding)))
    for path, flow in flows.items():
        if not isinstance(flow, FileFlow):
            continue
        language = language_by_path.get(path, flow.language)
        for taint in flow.taint_paths:
            key = (taint.path, taint.sink_line, taint.sink)
            if (taint.path, taint.sink_line, taint.sink) in claimed or any(
                item.path == taint.path and item.line == taint.sink_line for item in findings
            ):
                continue
            records.append(
                OverlayRecord(
                    proposal_id=f"overlay-coverage:{taint.path}:{taint.sink_line}:{taint.sink}",
                    path=taint.path,
                    line=taint.sink_line,
                    disposition="coverage",
                    reason=f"uninventoried sink {taint.sink} reached from {taint.source}",
                    cwe=taint.cwe,
                    sources=(taint.source,),
                    sinks=(taint.sink,),
                    sanitizers=(),
                    evidence_level="static_corroboration",
                    origin="overlay",
                    engine=flow.engine,
                    language=language,
                )
            )
            claimed.add(key)
    return records
 def write_overlay(records: Sequence[OverlayRecord], path: Path) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    payload = {"records": [asdict(record) for record in records]}
    path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
 def finding_from_coverage(record: OverlayRecord) -> StaticFinding:
    policy = load_policy()
    severity = SEVERITY_LABEL.get(resolve_severity(policy, record.cwe), "info") if record.cwe else "info"
    sink = record.sinks[0] if record.sinks else "sink"
    return StaticFinding(
        finding_id=record.proposal_id,
        path=record.path,
        title=f"Overlay coverage: {sink}",
        severity=severity,
        confidence="medium",
        evidence_level="static_corroboration",
        rationale=record.reason,
        line=record.line,
        function_name=None,
        reachability_status="unknown",
        reachability_evidence=[],
        reachability_conditions=[],
        tags=["overlay-originated"],
        ranking_priority=0.0,
    )
 def _try_facts() -> SemanticFacts | FactLoadError:
    try:
        return load_facts()
    except FactLoadError as exc:
        return exc
 def _flows_for(
    root: Path,
    targets: Sequence[FileTarget],
    facts: SemanticFacts,
    joern_extra: Sequence[TaintPath] | None,
 ) -> dict[str, FileFlow | UnadjudicatedFlow]:
    flows: dict[str, FileFlow | UnadjudicatedFlow] = {}
    for target in targets:
        text = _read_text(root / target.path)
        analyzed = analyze_file(target.path, text, target.language, facts)
        if isinstance(analyzed, FileFlow):
            extra = tuple(joern_extra) if joern_extra is not None else _joern_paths(target.path, text, target.language)
            if extra:
                analyzed = FileFlow(
                    path=analyzed.path,
                    language=analyzed.language,
                    engine=analyzed.engine,
                    taint_paths=merge_joern_paths(analyzed.taint_paths, extra),
                    demotions=analyzed.demotions,
                    incomplete=analyzed.incomplete,
                )
        flows[target.path] = analyzed
    return flows
 def _joern_paths(path: str, text: str, language: str) -> tuple[TaintPath, ...]:
    if not joern_available():
        return ()
    extra = joern_flow(path, text, language)
    paths: list[TaintPath] = []
    for item in extra:
        if isinstance(item, TaintPath):
            paths.append(item)
    return tuple(paths)
 def _record_for_finding(finding: StaticFinding, language: str, flow: FileFlow | UnadjudicatedFlow | None) -> OverlayRecord:
    if flow is None:
        return _unadjudicated(finding, language, "flow_incomplete", "none")
    if isinstance(flow, UnadjudicatedFlow):
        reason = flow.reason if flow.reason in UNADJUDICATED_REASONS else "flow_incomplete"
        return _unadjudicated(finding, language, reason, flow.engine)
    demotion = _matching_demotion(finding, flow)
    taint = _matching_taint(finding, flow)
    if demotion is not None and taint is None:
        return OverlayRecord(
            proposal_id=finding.finding_id,
            path=finding.path,
            line=finding.line,
            disposition="demote",
            reason=f"{demotion.sink} dominated by {demotion.dominating_fact}",
            cwe=demotion.cwe or _cwe_of(finding),
            sources=(),
            sinks=(demotion.sink,),
            sanitizers=demotion.sanitizers,
            evidence_level="static_corroboration",
            dominating_fact=demotion.dominating_fact,
            engine=flow.engine,
            language=language,
        )
    if taint is not None and (demotion is None or not demotion.sanitizers):
        # Sanitizer on the same path would have produced a demotion with sanitizers.
        return OverlayRecord(
            proposal_id=finding.finding_id,
            path=finding.path,
            line=finding.line,
            disposition="promote",
            reason=f"{taint.source} reaches {taint.sink} without a dominating sanitizer",
            cwe=taint.cwe or _cwe_of(finding),
            sources=(taint.source,),
            sinks=(taint.sink,),
            sanitizers=(),
            evidence_level="static_corroboration",
            engine=flow.engine,
            language=language,
        )
    if demotion is not None:
        return OverlayRecord(
            proposal_id=finding.finding_id,
            path=finding.path,
            line=finding.line,
            disposition="demote",
            reason=f"{demotion.sink} dominated by {demotion.dominating_fact}",
            cwe=demotion.cwe or _cwe_of(finding),
            sources=(),
            sinks=(demotion.sink,),
            sanitizers=demotion.sanitizers,
            evidence_level="static_corroboration",
            dominating_fact=demotion.dominating_fact,
            engine=flow.engine,
            language=language,
        )
    return _unadjudicated(finding, language, "flow_incomplete", flow.engine)
 def _matching_taint(finding: StaticFinding, flow: FileFlow) -> TaintPath | None:
    for item in flow.taint_paths:
        if item.path == finding.path and _line_matches(finding.line, item.sink_line) and _sink_matches(finding, item.sink):
            return item
    for item in flow.taint_paths:
        if item.path == finding.path and _line_matches(finding.line, item.sink_line):
            return item
    return None
 def _matching_demotion(finding: StaticFinding, flow: FileFlow) -> Demotion | None:
    for item in flow.demotions:
        if item.path == finding.path and _line_matches(finding.line, item.sink_line) and _sink_matches(finding, item.sink):
            return item
    for item in flow.demotions:
        if item.path == finding.path and _line_matches(finding.line, item.sink_line):
            return item
    return None
 def _line_matches(finding_line: int | None, sink_line: int) -> bool:
    return finding_line is None or finding_line == sink_line
 def _sink_matches(finding: StaticFinding, sink: str) -> bool:
    haystack = f"{finding.finding_id} {finding.title} {finding.rationale} {sink}".lower()
    return sink.lower() in haystack or finding.finding_id.split(":", 1)[0].lower() in sink.lower()
 def _sink_hint(finding: StaticFinding) -> str:
    return finding.finding_id.split(":", 1)[0]
 def _cwe_of(finding: StaticFinding) -> str:
    for token in finding.rationale.replace("(", " ").replace(")", " ").split():
        if token.startswith("CWE-"):
            return token.rstrip(".,;")
    return ""
 def _unadjudicated(finding: StaticFinding, language: str, reason: str, engine: str) -> OverlayRecord:
    return OverlayRecord(
        proposal_id=finding.finding_id,
        path=finding.path,
        line=finding.line,
        disposition="unadjudicated",
        reason=reason,
        cwe=_cwe_of(finding),
        sources=(),
        sinks=(),
        sanitizers=(),
        evidence_level="static_corroboration",
        engine=engine,
        language=language,
    )
 def _read_text(path: Path) -> str:
    try:
        return path.read_text(errors="ignore")
    except OSError:
        return ""
@@ -0,0 +1,12 @@
 """Prove-budget ranker: heuristic order plus optional mechanism recall."""
 from .mechanisms import MechanismStore, append_mechanism, order_promotions
 from .prove_filter import filter_promoted_hotspots, promoted_findings
 __all__ = [
    "MechanismStore",
    "append_mechanism",
    "filter_promoted_hotspots",
    "order_promotions",
    "promoted_findings",
 ]
@@ -0,0 +1,30 @@
 """Prove only promotions. Demotions are dropped; unadjudicated is not negative proof."""
 from __future__ import annotations
 from collections.abc import Sequence
 from dataclasses import replace
 from ..complexity.map import Hotspot
 from ..findings import StaticFinding
 from .overlay import OverlayRecord
 def filter_promoted_hotspots(
    hotspots: Sequence[Hotspot],
    records: Sequence[OverlayRecord],
    findings: Sequence[StaticFinding] | None = None,
 ) -> tuple[Hotspot, ...]:
    del findings
    promoted = {record.proposal_id for record in records if record.disposition == "promote"}
    selected: list[Hotspot] = []
    for hotspot in hotspots:
        kept = tuple(finding_id for finding_id in hotspot.inventory_finding_ids if finding_id in promoted)
        if kept:
            selected.append(replace(hotspot, inventory_finding_ids=kept))
    return tuple(selected)
 def promoted_findings(findings: Sequence[StaticFinding], records: Sequence[OverlayRecord]) -> list[StaticFinding]:
    promoted = {record.proposal_id for record in records if record.disposition == "promote"}
    return [finding for finding in findings if finding.finding_id in promoted]
@@ -0,0 +1,285 @@
 """Intra-file taint on FileIR. Incomplete flow is unadjudicated, never demote."""
 from __future__ import annotations
 from dataclasses import dataclass
 from .facts import SemanticFacts, SinkFact, SourceFact
 from .ir import CallSite, FileIR, FunctionIR, parse_file
 TAINTED = "tainted"
 CONSTANT = "constant"
 UNKNOWN = "unknown"
@dataclass(frozen=True)
 class TaintPath:
    path: str
    source: str
    sink: str
    sink_line: int
    names: tuple[str, ...]
    sanitizer: str | None = None
    cwe: str = ""
@dataclass(frozen=True)
 class Demotion:
    path: str
    sink: str
    sink_line: int
    dominating_fact: str
    sanitizers: tuple[str, ...] = ()
    cwe: str = ""
@dataclass(frozen=True)
 class UnadjudicatedFlow:
    path: str
    reason: str
    engine: str = "none"
@dataclass(frozen=True)
 class FileFlow:
    path: str
    language: str
    engine: str
    taint_paths: tuple[TaintPath, ...]
    demotions: tuple[Demotion, ...]
    incomplete: tuple[tuple[int, str], ...]
 def analyze_file(path: str, text: str, language: str, facts: SemanticFacts) -> FileFlow | UnadjudicatedFlow:
    ir = parse_file(path, text, language)
    if not ir.parse_ok:
        return UnadjudicatedFlow(path=path, reason=ir.reason or "parse_failed", engine=ir.engine)
    return analyze_ir(ir, facts.for_language(language))
 def analyze_ir(ir: FileIR, facts: SemanticFacts) -> FileFlow:
    paths: list[TaintPath] = []
    demotions: list[Demotion] = []
    incomplete: list[tuple[int, str]] = []
    for function in ir.functions:
        _analyze_function(ir.path, function, facts, paths, demotions, incomplete)
    return FileFlow(
        path=ir.path,
        language=ir.language,
        engine=ir.engine,
        taint_paths=tuple(paths),
        demotions=tuple(demotions),
        incomplete=tuple(incomplete),
    )
 def _analyze_function(
    path: str,
    function: FunctionIR,
    facts: SemanticFacts,
    paths: list[TaintPath],
    demotions: list[Demotion],
    incomplete: list[tuple[int, str]],
 ) -> None:
    states: dict[str, str] = {param: TAINTED for param in function.params}
    sources: dict[str, str] = {param: "parameter" for param in function.params}
    for _ in range(8):
        changed = False
        for bind in function.binds:
            new_state, source = _bind_state(bind, states, sources, facts)
            if states.get(bind.name) != new_state or (source and sources.get(bind.name) != source):
                states[bind.name] = new_state
                if source:
                    sources[bind.name] = source
                changed = True
        if not changed:
            break
    for call in function.calls:
        _analyze_call(path, call, states, sources, facts, paths, demotions, incomplete)
 def _bind_state(bind: object, states: dict[str, str], sources: dict[str, str], facts: SemanticFacts) -> tuple[str, str | None]:
    from .ir import Bind
    assert isinstance(bind, Bind)
    source = _source_in(bind.value_text, facts)
    if source:
        return TAINTED, source.id
    if bind.is_constant and not bind.names:
        return CONSTANT, None
    inherited = [_name_state(name, states) for name in bind.names]
    if TAINTED in inherited:
        tainted_name = next((name for name in bind.names if states.get(name) == TAINTED), None)
        return TAINTED, sources.get(tainted_name or "")
    if inherited and all(item == CONSTANT for item in inherited):
        return CONSTANT, None
    if bind.call_name and not _known_callee(bind.call_name, facts):
        return UNKNOWN, None
    if not bind.names and not bind.is_constant:
        return UNKNOWN, None
    return UNKNOWN, None
 def _analyze_call(
    path: str,
    call: CallSite,
    states: dict[str, str],
    sources: dict[str, str],
    facts: SemanticFacts,
    paths: list[TaintPath],
    demotions: list[Demotion],
    incomplete: list[tuple[int, str]],
 ) -> None:
    sink = _sink_for(call.name, facts)
    sanitizer = _sanitizer_for(call, facts)
    arg_states = [_arg_state(call, index, states, facts) for index in range(len(call.arg_texts))]
    if sink is None:
        return
    if sanitizer is not None:
        demotions.append(
            Demotion(
                path=path,
                sink=sink.id,
                sink_line=call.line,
                dominating_fact=sanitizer,
                sanitizers=(sanitizer,),
                cwe=sink.cwe,
            )
        )
        return
    if TAINTED in arg_states:
        source_name = _source_label(call, states, sources, facts)
        paths.append(
            TaintPath(
                path=path,
                source=source_name,
                sink=sink.id,
                sink_line=call.line,
                names=_tainted_names(call, states),
                cwe=sink.cwe,
            )
        )
        return
    if arg_states and all(item == CONSTANT for item in arg_states):
        demotions.append(
            Demotion(
                path=path,
                sink=sink.id,
                sink_line=call.line,
                dominating_fact="constant",
                cwe=sink.cwe,
            )
        )
        return
    if not call.arg_texts:
        incomplete.append((call.line, "flow_incomplete"))
        return
    incomplete.append((call.line, "flow_incomplete"))
 def _arg_state(call: CallSite, index: int, states: dict[str, str], facts: SemanticFacts) -> str:
    text = call.arg_texts[index]
    if _source_in(text, facts):
        return TAINTED
    if call.arg_is_constant[index]:
        return CONSTANT
    names = call.arg_names[index]
    inherited = [_name_state(name, states) for name in names]
    if TAINTED in inherited:
        return TAINTED
    if inherited and all(item == CONSTANT for item in inherited):
        return CONSTANT
    if not names:
        return UNKNOWN
    return UNKNOWN
 def _source_label(call: CallSite, states: dict[str, str], sources: dict[str, str], facts: SemanticFacts) -> str:
    for text in call.arg_texts:
        found = _source_in(text, facts)
        if found is not None:
            return found.id
    for names in call.arg_names:
        for name in names:
            if states.get(name) == TAINTED:
                return sources.get(name, "parameter")
    return "parameter"
 def _tainted_names(call: CallSite, states: dict[str, str]) -> tuple[str, ...]:
    names: list[str] = []
    for group in call.arg_names:
        for name in group:
            if states.get(name) == TAINTED:
                names.append(name)
    return tuple(names)
 def _name_state(name: str, states: dict[str, str]) -> str:
    return states.get(name, UNKNOWN)
 def _source_in(text: str, facts: SemanticFacts) -> SourceFact | None:
    for source in facts.sources:
        for pattern in source.patterns:
            if pattern and pattern in text:
                return source
    return None
 def _sink_for(name: str, facts: SemanticFacts) -> SinkFact | None:
    for sink in facts.sinks:
        for call in sink.calls:
            if _call_matches(name, call):
                return sink
    return None
 def _sanitizer_for(call: CallSite, facts: SemanticFacts) -> str | None:
    for sanitizer in facts.sanitizers:
        if not any(_call_matches(call.name, name) for name in sanitizer.calls):
            continue
        if sanitizer.parameterized and call.extra_arg_is_sequence:
            return sanitizer.id
        if sanitizer.literal_format_arg is not None:
            index = sanitizer.literal_format_arg
            if index < len(call.arg_is_constant) and call.arg_is_constant[index] and _looks_like_format(call.arg_texts[index]):
                return sanitizer.id
        if not sanitizer.parameterized and sanitizer.literal_format_arg is None:
            return sanitizer.id
    return None
 def _looks_like_format(text: str) -> bool:
    stripped = text.strip()
    return len(stripped) >= 2 and stripped[0] in {"'", '"'} and stripped[-1] == stripped[0]
 def _known_callee(name: str, facts: SemanticFacts) -> bool:
    if _sink_for(name, facts) is not None:
        return True
    if _source_in(name, facts) is not None:
        return True
    return any(any(_call_matches(name, call) for call in sanitizer.calls) for sanitizer in facts.sanitizers)
 def _call_matches(name: str, fact_call: str) -> bool:
    if name == fact_call:
        return True
    if name.endswith("." + fact_call):
        return True
    if fact_call.endswith(name) and "." not in name:
        # "execute" matches "db.execute" via endswith .execute above; bare execute matches execute.
        return name == fact_call.split(".")[-1]
    return False
 def merge_joern_paths(existing: tuple[TaintPath, ...], extra: tuple[TaintPath, ...]) -> tuple[TaintPath, ...]:
    seen = {(item.path, item.sink_line, item.sink) for item in existing}
    merged = list(existing)
    for item in extra:
        key = (item.path, item.sink_line, item.sink)
        if key not in seen:
            merged.append(item)
            seen.add(key)
    return tuple(merged)
@@ -0,0 +1,173 @@
 """Vuln-vs-fixed pair corpus: isolated trees, local gate, github honesty dashboard."""
 import json
 from pathlib import Path
 from openultrasast.benchmark import ExpectedFinding
 from openultrasast.cli import main
 from openultrasast.gate import LANGUAGE_MANIFESTS
 from openultrasast.pair_gate import pair_gate
 from openultrasast.pairs import (
    DEFAULT_CATALOG,
    DEFAULT_SAST_CATALOG,
    PairCase,
    build_pair_signals,
    evaluate_catalog,
    evaluate_pair,
    load_datasets,
    load_pair_catalog,
    select_slice,
 )
 def test_catalog_loads_local_and_github_slices() -> None:
    cases = load_pair_catalog(DEFAULT_CATALOG)
    slices = {case.slice for case in cases}
    names = {case.name for case in cases}
    assert {"local", "github", "sast"} <= slices
    assert "local-python-vulnerable" in names
    assert "sven-cwe-078-os-system" in names
    assert "hutool-cve-2018-17297" in names
    assert "owasp-python-codeinj" in names
    assert "juliet-c-cwe134-printf" in names
    assert all(case.vuln_file.is_file() and case.fixed_file.is_file() for case in cases)
 def test_pair_catalog_is_outside_stage1_smoke_gate() -> None:
    listed = {name for names in LANGUAGE_MANIFESTS.values() for name in names}
    leaked = [case.name for case in load_pair_catalog() if case.name in listed]
    assert leaked == [], "pair catalog must not join LANGUAGE_MANIFESTS (stage-1 90% smoke gate)"
 def test_datasets_catalog_lists_public_vfc_repos() -> None:
    names = {str(item["name"]) for item in load_datasets()}
    assert {
        "CVEfixes",
        "CWE-Bench-Java",
        "SVEN",
        "Vul4J",
        "DiverseVul",
        "MoreFixes",
        "OWASP Benchmark Java",
        "OWASP Benchmark Python",
        "Juliet Test Suite C/C++",
        "Juliet Test Suite Java",
    } <= names
    assert DEFAULT_SAST_CATALOG.is_file()
    java = Path("benchmarks/pairs/datasets/cwe-bench-java-project_info.csv")
    assert java.is_file()
    assert "CVE-2018-17297" in java.read_text()
 def test_synthetic_pair_fires_on_vuln_and_stays_silent_on_fix(tmp_path: Path) -> None:
    vuln = tmp_path / "vuln.py"
    fixed = tmp_path / "fixed.py"
    vuln.write_text("def run(x):\n    return eval(x)\n")
    fixed.write_text("def run(x):\n    return str(x)\n")
    case = PairCase(
        name="synth-eval",
        slice="local",
        language="python",
        origin="test",
        vuln_file=vuln,
        fixed_file=fixed,
        relpath="app.py",
        expected=(
            ExpectedFinding(
                cwe="CWE-95",
                vulnerability_class="code injection",
                path="app.py",
                evidence="eval executes input",
                rule_id="python-unsafe-eval",
                sink="eval",
            ),
        ),
        min_recall=1.0,
        fix_policy="silent",
    )
    outcome = evaluate_pair(case)
    assert outcome.pair_correct
    assert outcome.vuln_matched == 1 and outcome.fix_findings == 0
    assert build_pair_signals([outcome]) == []
 def test_pair_signals_emit_miss_on_vuln_and_fp_on_fix(tmp_path: Path) -> None:
    vuln = tmp_path / "vuln.py"
    fixed = tmp_path / "fixed.py"
    vuln.write_text("def run(x):\n    return str(x)\n")
    fixed.write_text("def run(x):\n    return eval(x)\n")
    case = PairCase(
        name="inverted",
        slice="github",
        language="python",
        origin="test",
        vuln_file=vuln,
        fixed_file=fixed,
        relpath="app.py",
        expected=(
            ExpectedFinding(
                cwe="CWE-95",
                vulnerability_class="code injection",
                path="app.py",
                evidence="eval executes input",
                rule_id="python-unsafe-eval",
                sink="eval",
            ),
        ),
        min_recall=1.0,
        fix_policy="silent",
    )
    outcome = evaluate_pair(case)
    assert not outcome.detected_vuln and not outcome.silent_fix
    signals = {item["signal"] for item in build_pair_signals([outcome])}
    assert signals == {"miss", "fp"}
 def test_local_slice_clears_the_pair_gate() -> None:
    verdict = pair_gate()
    assert verdict.passed, verdict.reasons
    local = verdict.result.per_slice["local"]
    assert local.pairs >= 3
    assert local.pair_pass_rate == 1.0
    assert local.specificity == 1.0
 def test_github_slice_measures_real_vfc_pairs() -> None:
    cases = select_slice(load_pair_catalog(), "github")
    result = evaluate_catalog(cases)
    by_name = {outcome.name: outcome for outcome in result.outcomes}
    assert by_name["sven-cwe-078-os-system"].pair_correct
    assert by_name["sven-cwe-078-shell-true"].pair_correct
    assert by_name["sven-cwe-089-execute-percent"].pair_correct
    assert by_name["node-serialize-cve-2017-5941"].pair_correct
    # Real corpus honesty: concat-via-query and Zip Slip are currently regex-blind.
    assert not by_name["sven-cwe-089-query-concat"].detected_vuln
    assert not by_name["hutool-cve-2018-17297"].detected_vuln
    assert result.overall.pairs == len(cases)
    assert any(signal["signal"] == "miss" for signal in result.signals)
 def test_sast_slice_scores_owasp_and_juliet() -> None:
    cases = select_slice(load_pair_catalog(), "sast")
    result = evaluate_catalog(cases)
    by_name = {outcome.name: outcome for outcome in result.outcomes}
    assert by_name["juliet-c-cwe134-printf"].pair_correct
    assert by_name["owasp-python-deser"].pair_correct
    # Overlay follows intra-file names, not ConfigParser heap; 00074 stays unadjudicated.
    assert not by_name["owasp-python-codeinj"].detected_vuln
    assert by_name["owasp-java-cmdi"].detected_vuln
    assert result.overall.pairs == len(cases)
    # Youden is TPR - FPR; a regex harness is not expected to clear OWASP.
    assert -1.0 <= result.overall.youden <= 1.0
 def test_cli_pairs_json_scoreboard() -> None:
    import io
    from contextlib import redirect_stdout
    buf = io.StringIO()
    with redirect_stdout(buf):
        assert main(["pairs", "--slice", "github", "--json"]) == 0
    payload = json.loads(buf.getvalue())
    assert payload["overall"]["pairs"] >= 4
    assert "sven-cwe-078-os-system" in {item["name"] for item in payload["outcomes"]}
@@ -153,14 +153,12 @@ def test_deep_scan_unsafe_hunter_snippet_is_safety_rejected_without_docker_job(t
    hunter_findings = [item for item in findings if str(item["finding_id"]).startswith("tool-hunter:")]
    assert docker_argv == []
    assert fake.jobs == []
    assert hunter_findings
    assert all(item["evidence_level"] == "suspicion" for item in hunter_findings)
-    assert payload["verdicts"]
+    hunter_ids = {item["finding_id"] for item in hunter_findings}
-    assert any(item["reason"] == "safety_rejected" for item in payload["verdicts"])
+    # Overlay promotions may still be proven; hunter suspicion never becomes a sandbox candidate.
    for item in payload["verdicts"]:
-        if item["reason"] == "safety_rejected":
+        assert hunter_ids.isdisjoint(item["inventory_finding_ids"])
            assert item["verdict"] == "inconclusive"
 def test_deep_scan_fail_on_worth_fixing_exits_one_when_triggerable(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
@@ -0,0 +1,269 @@
 """Propose/adjudicate/prove overlay records, facts, taint, and coverage."""
 from __future__ import annotations
 from pathlib import Path
 import pytest
 from openultrasast.findings import StaticFinding, quick_scan_findings
 from openultrasast.mapping import analyze_entry_points, attach_reachability_hints
 from openultrasast.preprocess import FileTarget, preprocess_repository
 from openultrasast.rank import rank_targets
 from openultrasast.semantic import OverlayError, OverlayRecord, adjudicate, load_facts
 from openultrasast.semantic.engines import joern_available, tree_sitter_available
 from openultrasast.semantic.facts import FactLoadError
 from openultrasast.semantic.ir import FileIR, FunctionIR, parse_file
 from openultrasast.semantic.taint import TaintPath, analyze_file
 def _finding(**kwargs: object) -> StaticFinding:
    payload: dict[str, object] = {
        "finding_id": "python-unsafe-eval:app.py:1",
        "path": "app.py",
        "title": "Dynamic Python execution needs review",
        "severity": "high",
        "confidence": "medium",
        "evidence_level": "static_corroboration",
        "rationale": "Static pattern python-unsafe-eval (CWE-95) matched eval on line 1",
        "line": 1,
        "function_name": None,
        "reachability_status": "unknown",
        "reachability_evidence": [],
        "reachability_conditions": [],
        "tags": ["syscall_entry"],
        "ranking_priority": 1.0,
    }
    payload.update(kwargs)
    return StaticFinding(**payload)  # type: ignore[arg-type]
 def _target(path: str, language: str = "python") -> FileTarget:
    return FileTarget(
        path=path,
        absolute_path=path,
        language=language,
        loc=8,
        tags=[],
        has_fuzz_entry_point=False,
    )
 def _scan_overlay(root: Path) -> list[OverlayRecord]:
    _, targets = preprocess_repository(root)
    targets = attach_reachability_hints(targets, analyze_entry_points(root, targets))
    findings = quick_scan_findings(root, targets, rank_targets(targets))
    return adjudicate(root=root, targets=targets, findings=findings)
 def test_overlay_record_rejects_demote_without_dominating_fact() -> None:
    with pytest.raises(OverlayError, match="constant or sanitizer"):
        OverlayRecord(
            proposal_id="python-unsafe-eval:app.py:1",
            path="app.py",
            line=1,
            disposition="demote",
            reason="looks safe",
            cwe="CWE-95",
            sources=(),
            sinks=("eval",),
            sanitizers=(),
            evidence_level="static_corroboration",
        )
 def test_overlay_record_rejects_proof_rungs() -> None:
    with pytest.raises(OverlayError, match="proof rungs"):
        OverlayRecord(
            proposal_id="python-unsafe-eval:app.py:1",
            path="app.py",
            line=1,
            disposition="promote",
            reason="source reaches sink",
            cwe="CWE-95",
            sources=("request",),
            sinks=("eval",),
            sanitizers=(),
            evidence_level="exploit_demonstrated",
        )
 def test_overlay_record_requires_unadjudicated_reason() -> None:
    with pytest.raises(OverlayError, match="unadjudicated reason"):
        OverlayRecord(
            proposal_id="x",
            path="app.py",
            line=1,
            disposition="unadjudicated",
            reason="probably fine",
            cwe="CWE-95",
            sources=(),
            sinks=(),
            sanitizers=(),
            evidence_level="static_corroboration",
        )
 def test_load_facts_binds_cwe_ids() -> None:
    facts = load_facts()
    python = facts.for_language("python")
    assert any(sink.cwe.startswith("CWE-") and sink.id == "eval" for sink in python.sinks)
    assert any(source.id == "request" for source in python.sources)
 def test_load_facts_broken_file_is_structured_error(tmp_path: Path) -> None:
    facts_dir = tmp_path / "facts"
    facts_dir.mkdir()
    (facts_dir / "python.toml").write_text('version = 1\nlanguage = "python"\nthis is not toml {')
    with pytest.raises(FactLoadError) as exc:
        load_facts(facts_dir)
    assert exc.value.reason == "facts_unavailable"
 def test_eval_request_args_promotes(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run():\n    return eval(request.args)\n")
    records = _scan_overlay(root)
    assert len(records) == 1
    assert records[0].disposition == "promote"
    assert records[0].evidence_level == "static_corroboration"
 def test_constant_exec_demotes(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run():\n    exec('ls')\n")
    records = _scan_overlay(root)
    assert len(records) == 1
    assert records[0].disposition == "demote"
    assert records[0].dominating_fact == "constant"
 def test_syntax_error_is_unadjudicated_parse_failed(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("eval(request.args)\ndef oops(\n")
    records = _scan_overlay(root)
    assert records
    assert all(record.disposition == "unadjudicated" for record in records)
    assert all(record.reason == "parse_failed" for record in records)
 def test_split_sink_python_yields_source_to_sink_path() -> None:
    root = Path("benchmarks/fixtures/split-sink-python")
    facts = load_facts()
    text = (root / "app.py").read_text()
    flow = analyze_file("app.py", text, "python", facts)
    assert not isinstance(flow, type(None))
    from openultrasast.semantic.taint import FileFlow
    assert isinstance(flow, FileFlow)
    assert any(item.sink == "execute" for item in flow.taint_paths)
 def test_missing_module_call_is_flow_incomplete_not_safe(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("import helper\ndef run():\n    value = helper.hidden()\n    eval(value)\n")
    records = _scan_overlay(root)
    eval_records = [record for record in records if "python-unsafe-eval" in record.proposal_id]
    assert eval_records
    assert eval_records[0].disposition == "unadjudicated"
    assert eval_records[0].reason == "flow_incomplete"
 def test_overlay_length_matches_proposals_plus_coverage_only(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run(x):\n    eval(request.args)\n    digest = hashlib.new('md5', x)\n    return digest\n")
    _, targets = preprocess_repository(root)
    findings = quick_scan_findings(root, targets, rank_targets(targets))
    records = adjudicate(root=root, targets=targets, findings=findings)
    proposal_ids = [record.proposal_id for record in records if record.origin != "overlay"]
    assert proposal_ids == [finding.finding_id for finding in findings]
    assert any(record.disposition == "coverage" and "hashlib" in record.proposal_id for record in records)
    assert all(record.evidence_level == "static_corroboration" for record in records if record.disposition == "coverage")
 def test_js_language_unsupported_without_tree_sitter(monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setattr("openultrasast.semantic.ir.tree_sitter_available", lambda: False)
    monkeypatch.setattr("openultrasast.semantic.engines.tree_sitter_available", lambda: False)
    ir = parse_file("app.js", "eval(req.query.x)\n", "javascript")
    assert ir.parse_ok is False
    assert ir.reason == "language_unsupported"
 def test_js_yields_ir_when_tree_sitter_available(monkeypatch: pytest.MonkeyPatch) -> None:
    fake = FileIR(
        path="app.js",
        language="javascript",
        engine="tree-sitter",
        functions=(FunctionIR(name="run", params=("x",), start_line=1, end_line=1, binds=(), calls=()),),
        parse_ok=True,
    )
    monkeypatch.setattr("openultrasast.semantic.ir.tree_sitter_available", lambda: True)
    monkeypatch.setattr("openultrasast.semantic.ir.parse_tree_sitter_cli", lambda *args, **kwargs: fake)
    ir = parse_file("app.js", "eval(req.query.x)\n", "javascript")
    assert ir.parse_ok is True
    assert ir.engine == "tree-sitter"
 def test_joern_absence_does_not_block_overlay(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setattr("openultrasast.semantic.overlay.joern_available", lambda: False)
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run():\n    return eval(request.args)\n")
    records = _scan_overlay(root)
    assert records[0].disposition == "promote"
 def test_joern_miss_does_not_demote_tree_sitter_promote(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.setattr("openultrasast.semantic.overlay.joern_available", lambda: True)
    monkeypatch.setattr("openultrasast.semantic.overlay.joern_flow", lambda *args, **kwargs: ())
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run():\n    return eval(request.args)\n")
    records = _scan_overlay(root)
    assert records[0].disposition == "promote"
 def test_facts_unavailable_marks_every_proposal(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run():\n    return eval(request.args)\n")
    _, targets = preprocess_repository(root)
    findings = quick_scan_findings(root, targets, rank_targets(targets))
    records = adjudicate(root=root, targets=targets, findings=findings, facts=FactLoadError("broken"))
    assert findings
    assert len(records) == len(findings)
    assert all(record.disposition == "unadjudicated" and record.reason == "facts_unavailable" for record in records)
 def test_parameterized_execute_demotes(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run(name):\n    db.execute('select * from users where name = %s', (name,))\n")
    records = _scan_overlay(root)
    sql = [record for record in records if "sql" in record.proposal_id or record.sinks == ("execute",)]
    # inventory python-sql-injection requires composition on the execute line; this may be coverage-empty
    if sql:
        assert sql[0].disposition in {"demote", "unadjudicated"}
        if sql[0].disposition == "demote":
            assert sql[0].dominating_fact == "parameterized_execute"
 def test_probes_do_not_raise() -> None:
    assert tree_sitter_available() in {True, False}
    assert joern_available() in {True, False}
 def test_joern_extra_path_can_promote_unadjudicated(tmp_path: Path) -> None:
    root = tmp_path / "repo"
    root.mkdir()
    (root / "app.py").write_text("def run():\n    eval(helper())\n")
    _, targets = preprocess_repository(root)
    findings = quick_scan_findings(root, targets, rank_targets(targets))
    extra = (TaintPath(path="app.py", source="parameter", sink="eval", sink_line=findings[0].line or 2, names=(), cwe="CWE-95"),)
    records = adjudicate(root=root, targets=targets, findings=findings, joern_extra=extra)
    assert any(record.disposition == "promote" for record in records)
@@ -0,0 +1,343 @@
 """MAP overlay, prove filter, reports, pair scoring, mechanism memory."""
 from __future__ import annotations
 import json
 from pathlib import Path
 import pytest
 from openultrasast.benchmark import ExpectedFinding
 from openultrasast.cli import main
 from openultrasast.complexity.map import Hotspot
 from openultrasast.findings import StaticFinding, quick_scan_findings
 from openultrasast.gate import LANGUAGE_MANIFESTS
 from openultrasast.pairs import PairCase, evaluate_pair, load_pair_catalog
 from openultrasast.preprocess import preprocess_repository
 from openultrasast.rank import rank_targets
 from openultrasast.semantic import MechanismStore, OverlayRecord, append_mechanism, filter_promoted_hotspots, order_promotions
 from openultrasast.semantic.prove_filter import promoted_findings
 def _latest_run(repo: Path) -> Path:
    return sorted((repo / ".runs").iterdir())[-1]
 def _finding(finding_id: str, path: str = "app.py", line: int = 1) -> StaticFinding:
    return StaticFinding(
        finding_id=finding_id,
        path=path,
        title="test",
        severity="high",
        confidence="medium",
        evidence_level="static_corroboration",
        rationale="Static pattern python-unsafe-eval (CWE-95) matched",
        line=line,
        function_name="run",
        reachability_status="reachable",
        reachability_evidence=[],
        reachability_conditions=[],
        tags=["syscall_entry"],
        ranking_priority=1.0,
    )
 def _record(proposal_id: str, disposition: str, **kwargs: object) -> OverlayRecord:
    dominating = kwargs.get("dominating_fact")
    sanitizers = kwargs.get("sanitizers", ())
    return OverlayRecord(
        proposal_id=proposal_id,
        path=str(kwargs.get("path", "app.py")),
        line=int(kwargs.get("line", 1)),
        disposition=disposition,
        reason=str(kwargs.get("reason", disposition)),
        cwe=str(kwargs.get("cwe", "CWE-95")),
        sources=tuple(kwargs.get("sources", ())),  # type: ignore[arg-type]
        sinks=tuple(kwargs.get("sinks", ("eval",))),  # type: ignore[arg-type]
        sanitizers=tuple(sanitizers) if isinstance(sanitizers, tuple | list) else (),
        evidence_level="static_corroboration",
        dominating_fact=str(dominating) if dominating else ("constant" if disposition == "demote" else None),
        language=str(kwargs.get("language", "python")),
    )
 def _hotspot(*finding_ids: str, score: float = 5.0, path: str = "app.py") -> Hotspot:
    return Hotspot(
        path=path,
        function_name="run",
        score=score,
        band="high",
        signals={},
        rationale="test",
        test_hint=None,
        inventory_finding_ids=finding_ids,
    )
 def test_quick_scan_writes_no_overlay(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    repo = tmp_path / "repo"
    repo.mkdir()
    (repo / "app.py").write_text("def run():\n    return eval(request.args)\n")
    monkeypatch.setenv("OPENULTRASAST_RUNS_DIR", ".runs")
    assert main(["scan", str(repo), "--mode", "quick"]) == 0
    run_dir = _latest_run(repo)
    assert not (run_dir / "overlay.json").exists()
    report = (run_dir / "report.md").read_text()
    assert "## Overlay" not in report
 def test_standard_scan_writes_overlay_without_executing_target(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    repo = tmp_path / "repo"
    repo.mkdir()
    (repo / "app.py").write_text("def run():\n    return eval(request.args)\n")
    monkeypatch.setenv("OPENULTRASAST_RUNS_DIR", ".runs")
    monkeypatch.setenv("OPENULTRASAST_SANDBOX_PROBE", "0")
    assert main(["scan", str(repo), "--mode", "standard"]) == 0
    run_dir = _latest_run(repo)
    payload = json.loads((run_dir / "overlay.json").read_text())
    assert payload["records"]
    assert payload["records"][0]["disposition"] == "promote"
    report = (run_dir / "report.md").read_text()
    assert "promoted" in report
    assert "Disposition: `promote`" in report
    sarif = json.loads((run_dir / "report.sarif").read_text())
    assert sarif["runs"][0]["results"][0]["properties"]["disposition"] == "promote"
    assert (run_dir / "manifest.json").read_text()
    manifest = json.loads((run_dir / "manifest.json").read_text())
    assert manifest["artifacts"]["overlay"] == "overlay.json"
 def test_prove_filter_drops_demoted_keeps_promoted() -> None:
    promoted_id = "python-unsafe-eval:app.py:2"
    demoted_id = "python-unsafe-eval:app.py:1"
    hotspots = (
        _hotspot(demoted_id, score=9.0),
        _hotspot(promoted_id, score=3.0),
    )
    records = (
        _record(demoted_id, "demote", dominating_fact="constant"),
        _record(promoted_id, "promote", sources=("request",)),
    )
    selected = filter_promoted_hotspots(hotspots, records)
    assert len(selected) == 1
    assert selected[0].inventory_finding_ids == (promoted_id,)
    findings = [_finding(demoted_id, line=1), _finding(promoted_id, line=2)]
    assert [item.finding_id for item in promoted_findings(findings, records)] == [promoted_id]
 def test_deep_scan_offers_only_promotions(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    from openultrasast import cli
    from openultrasast.sandbox import FakeSandboxRunner, SandboxResult
    repo = tmp_path / "repo"
    repo.mkdir()
    (repo / "app.py").write_text("def safe():\n    exec('ls')\n\ndef bad():\n    return eval(request.args)\n")
    monkeypatch.setenv("OPENULTRASAST_RUNS_DIR", ".runs")
    monkeypatch.setenv("OPENULTRASAST_SANDBOX_PROBE", "1")
    fake = FakeSandboxRunner(SandboxResult(exit_code=0, stdout="", stderr="", timed_out=False))
    monkeypatch.setattr(cli, "resolve_sandbox_runner", lambda: fake)
    assert main(["scan", str(repo), "--mode", "deep", "--fail-on", "worth-fixing"]) == 0
    run_dir = _latest_run(repo)
    overlay = json.loads((run_dir / "overlay.json").read_text())["records"]
    dispositions = {item["proposal_id"]: item["disposition"] for item in overlay}
    demoted = [key for key, value in dispositions.items() if value == "demote"]
    promoted = [key for key, value in dispositions.items() if value == "promote"]
    assert demoted
    assert promoted
    verdicts = json.loads((run_dir / "verdicts.json").read_text())["verdicts"]
    offered = {finding_id for item in verdicts for finding_id in item["inventory_finding_ids"]}
    assert offered <= set(promoted)
    assert not (offered & set(demoted))
 def test_fail_on_worth_fixing_ignores_unadjudicated(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    repo = tmp_path / "repo"
    repo.mkdir()
    (repo / "broken.py").write_text("eval(request.args)\ndef oops(\n")
    monkeypatch.setenv("OPENULTRASAST_RUNS_DIR", ".runs")
    monkeypatch.setenv("OPENULTRASAST_SANDBOX_PROBE", "0")
    assert main(["scan", str(repo), "--mode", "standard", "--fail-on", "worth-fixing"]) == 0
    run_dir = _latest_run(repo)
    overlay = json.loads((run_dir / "overlay.json").read_text())["records"]
    assert overlay
    assert all(item["disposition"] == "unadjudicated" for item in overlay)
    report = (run_dir / "report.md").read_text()
    assert "unadjudicated" in report
 def test_unlabeled_messy_tree_has_both_dispositions(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    repo = tmp_path / "repo"
    repo.mkdir()
    (repo / "broken.py").write_text("eval(request.args)\ndef oops(\n")
    (repo / "app.py").write_text("def run():\n    return eval(request.args)\n")
    monkeypatch.setenv("OPENULTRASAST_RUNS_DIR", ".runs")
    assert main(["scan", str(repo), "--mode", "standard"]) == 0
    overlay = json.loads((_latest_run(repo) / "overlay.json").read_text())["records"]
    by_path = {}
    for item in overlay:
        by_path.setdefault(item["path"], set()).add(item["disposition"])
    assert "unadjudicated" in by_path["broken.py"]
    assert "promote" in by_path["app.py"]
 def test_sast_slice_scores_overlay_not_inventory(tmp_path: Path) -> None:
    vuln = tmp_path / "vuln.py"
    fixed = tmp_path / "fixed.py"
    vuln.write_text("def bad(x):\n    exec(x)\n")
    fixed.write_text("def goodG2B():\n    exec('ls')\n")
    case = PairCase(
        name="synth-overlay-exec",
        slice="sast",
        language="python",
        origin="test",
        vuln_file=vuln,
        fixed_file=fixed,
        relpath="app.py",
        expected=(
            ExpectedFinding(
                cwe="CWE-95",
                vulnerability_class="code injection",
                path="app.py",
                evidence="exec",
                rule_id="python-unsafe-eval",
                sink="exec",
            ),
        ),
        min_recall=1.0,
        fix_policy="silent",
    )
    fix_root = tmp_path / "fix-tree"
    fix_root.mkdir()
    (fix_root / "app.py").write_text(fixed.read_text())
    _, fix_targets = preprocess_repository(fix_root)
    assert quick_scan_findings(fix_root, fix_targets, rank_targets(fix_targets)), "inventory still fires on goodG2B exec"
    outcome = evaluate_pair(case)
    assert outcome.detected_vuln
    assert outcome.silent_fix
    assert outcome.pair_correct
 def test_sast_pairs_stay_out_of_language_manifests() -> None:
    listed = {name for names in LANGUAGE_MANIFESTS.values() for name in names}
    leaked = [case.name for case in load_pair_catalog() if case.name in listed]
    assert leaked == []
 def test_markdown_report_includes_four_overlay_labels(tmp_path: Path) -> None:
    from openultrasast.reports import write_markdown_report
    finding = _finding("python-unsafe-eval:app.py:1")
    records = (
        _record("python-unsafe-eval:app.py:1", "promote", sources=("request",)),
        _record("python-unsafe-eval:b.py:1", "demote", path="b.py", dominating_fact="constant"),
        OverlayRecord(
            proposal_id="python-unsafe-eval:c.py:1",
            path="c.py",
            line=1,
            disposition="unadjudicated",
            reason="parse_failed",
            cwe="CWE-95",
            sources=(),
            sinks=(),
            sanitizers=(),
            evidence_level="static_corroboration",
        ),
        OverlayRecord(
            proposal_id="overlay-coverage:d.py:1:hashlib",
            path="d.py",
            line=1,
            disposition="coverage",
            reason="uninventoried sink hashlib",
            cwe="CWE-327",
            sources=("parameter",),
            sinks=("hashlib",),
            sanitizers=(),
            evidence_level="static_corroboration",
            origin="overlay",
        ),
    )
    output = tmp_path / "report.md"
    write_markdown_report([finding], output, overlay=records)
    text = output.read_text()
    assert "promoted" in text
    assert "demoted" in text
    assert "unadjudicated" in text
    assert "coverage" in text
 def test_mechanism_order_boosts_paraphrased_match(tmp_path: Path) -> None:
    store = MechanismStore(tmp_path / "mechanisms.jsonl")
    append_mechanism(
        store,
        summary="request parameter reaches eval",
        cwe="CWE-95",
        language="python",
        tags=["syscall_entry"],
        what_made_it_exploitable="dynamic execution of attacker input",
    )
    eval_id = "python-unsafe-eval:app.py:1"
    hash_id = "python-weak-hash:hash.py:1"
    hotspots = (_hotspot(hash_id, score=5.0, path="hash.py"), _hotspot(eval_id, score=5.0, path="app.py"))
    records = (
        _record(hash_id, "promote", path="hash.py", cwe="CWE-327", sinks=("hashlib",), language="python"),
        _record(eval_id, "promote", sources=("request",), cwe="CWE-95", sinks=("eval",), language="python"),
    )
    findings = [_finding(hash_id, path="hash.py"), _finding(eval_id)]
    class _Client:
        def embed(self, *, model: str, inputs: list[str], timeout_seconds: int = 60) -> list[list[float]]:
            del model, timeout_seconds
            vectors: list[list[float]] = []
            for text in inputs:
                if "eval" in text.lower() or "CWE-95" in text:
                    vectors.append([1.0, 0.0])
                else:
                    vectors.append([0.0, 1.0])
            return vectors
    ordered, degradation = order_promotions(hotspots, records, findings, store=store, client=_Client(), model="test-embed")
    assert degradation is None
    assert ordered[0].inventory_finding_ids == (eval_id,)
    assert {item.inventory_finding_ids[0] for item in ordered} == {eval_id, hash_id}
 def test_mechanism_degrades_to_heuristic_without_key(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
    store = MechanismStore(tmp_path / "mechanisms.jsonl")
    append_mechanism(
        store,
        summary="request parameter reaches eval",
        cwe="CWE-95",
        language="python",
        tags=["syscall_entry"],
        what_made_it_exploitable="dynamic execution of attacker input",
    )
    eval_id = "python-unsafe-eval:app.py:1"
    hash_id = "python-weak-hash:hash.py:1"
    hotspots = (_hotspot(hash_id, score=6.0, path="hash.py"), _hotspot(eval_id, score=3.0, path="app.py"))
    records = (
        _record(hash_id, "promote", path="hash.py", cwe="CWE-327", sinks=("hashlib",)),
        _record(eval_id, "promote", sources=("request",)),
    )
    findings = [_finding(hash_id, path="hash.py"), _finding(eval_id)]
    ordered, degradation = order_promotions(hotspots, records, findings, store=store, client=None, model=None)
    assert degradation == "embeddings_unavailable"
    assert [item.inventory_finding_ids[0] for item in ordered] == [hash_id, eval_id]
 def test_mechanisms_are_not_written_from_unadjudicated(tmp_path: Path) -> None:
    store = MechanismStore(tmp_path / "mechanisms.jsonl")
    record = OverlayRecord(
        proposal_id="python-unsafe-eval:app.py:1",
        path="app.py",
        line=1,
        disposition="unadjudicated",
        reason="parse_failed",
        cwe="CWE-95",
        sources=(),
        sinks=(),
        sanitizers=(),
        evidence_level="static_corroboration",
    )
    assert record.disposition != "promote"
    assert store.load() == ()