# Scanning This page shows the scan flows. - `ousast scan PATH --mode quick|standard|deep` scans a checkout. - `ousast pre-push` checks only what a push changes, or what an explicit `--base`/`--head` pair changes. The stage-by-stage pipeline, the evidence ladder and the score are in [Architecture](architecture.md). ## The three scan modes The stages each mode requests are listed in `MODE_STAGES` in `src/openultrasast/stages.py`: - `quick` runs STATIC. - `standard` adds MAP. - `deep` adds REGRESS. ```mermaid flowchart TD start["ousast scan PATH --mode M"] --> policy["policy_load, ruleset_load, policy_check"] policy --> prep["static_mapping, preprocess, entry_point_mapping, rank, calibrate"] prep --> which{"mode"} which -- "quick, deep" --> qf["quick_findings: language-scoped pattern rules"] which -- "standard" --> hp["hunter_pool: the same rules per ranked target"] qf --> verify["verify, record_calibration, score"] hp --> verify verify --> mapq{"MAP requested?"} mapq -- "quick: no" --> out["report.md, report.sarif, manifest.json"] mapq -- "standard, deep" --> map["MAP: complexity map, semantic overlay, obligations, Joern model layer"] map --> regq{"REGRESS requested?"} regq -- "standard: no" --> out regq -- "deep" --> reg["REGRESS: snippets in a Docker sandbox, verdicts.json"] reg --> out ``` | Mode | Adds | Needs | Without it | | --- | --- | --- | --- | | `quick` | pattern rules, entry-point hints, ranking, verification, score | nothing | - | | `standard` | MAP: complexity map, semantic overlay, authorization obligations, the Joern model layer; `fusion` (deterministic) | Joern; a provider key for the residual LLM question | `cpg_unavailable` degradation; the LLM's questions are skipped and only the graph's entailed findings are reported | | `deep` | REGRESS: promoted candidates loaded in a sandbox with no network, read-only source, non-root | a working `docker` | `sandbox_unavailable` degradation | Sources: `README.md` (scan modes), `src/openultrasast/cli.py` (`run_stage` calls), [Architecture](architecture.md#the-scan-pipeline). ## Quick rules per language Each rule runs only against files of its language. The bundled ruleset lives in `src/openultrasast/ruleset//rules.toml`. | Language | Classes (CWE) | | --- | --- | | C / C++ | stack buffer overflow (121), format string (134), command injection (78) | | Python | command injection (78), code injection (95), SSTI (94), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), insecure default (489) | | JavaScript / TS | command injection (78), code injection (95), XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) | | Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) | | PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules | The detection gate (`python -m openultrasast.gate`) enforces two limits per language on the bundled corpora in `benchmarks/manifests/`: at least 90% recall and under 10% false positives. - Those corpora are cheat-sheet fixtures. They catch regressions. They do not estimate recall on real code. - **PHP is not in the gate**. Its numbers are in-sample development numbers (`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). See `README.md` for the gate's per-language output. ## The Joern engine The model layer (`src/openultrasast/model/`, `src/openultrasast/cpg/`) is the core of `standard`. It builds one code property graph per scan with Joern. A code property graph is a graph of the program's syntax, control flow and data flow. The build runs `joern-parse`, then batched `joern --script` queries from `cpg/queries/*.sc`. The layer sends each question to the arbiter (the decider) of its type: - **taint**: a flow from source to sink with no discharging sanitizer; - **dominance**: does a guard govern the obligated operation; - **configuration**: constant abstraction over a value. Taint facts ship for C, Java, JavaScript, PHP and Python (`src/openultrasast/ruleset/semantic/*.toml`). ```mermaid flowchart LR src["Checkout"] --> parse["joern-parse: one CPG per repository"] parse --> q["Batched joern --script queries"] q --> arb{"Arbiter by question type"} arb --> taint["Taint: flow with no discharging sanitizer"] arb --> dom["Dominance: obligation with no guard"] arb --> conf["Configuration: permissive value"] taint --> rung["Rung: model_entailed or suspicion"] dom --> rung conf --> rung rung -- "residual question only" --> judge["LLM judge: confirm or contradict, never create a flow"] ``` The Docker image ships Joern and `php-cli`. The PHP frontend needs `php-cli`. Without Joern, the layer records `cpg_unavailable`; the rest of the scan is unchanged. - Engine installation and the Joern features used: [Engine and pre-push](ops/README.md). - How the graph is built, queried and refined, with the records behind each refinement: [Detection techniques](detection-techniques.md). ## Framework priors Some rule and taint-fact entries know a framework or library. They carry a `framework` or `library` tag. The tag names a row of `src/openultrasast/ruleset/frameworks.toml` (for example `wordpress`, `flask`, `django`, `express`, `spring`). The loaders take `priors=` with one of three values: - `"all"`: the default and today's behaviour; - `"off"`: language-level knowledge only; - a set of ids. So the contribution of framework knowledge can be measured and switched off. The decision engine's first slice ran with priors off ([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)). ## Pre-push: the delta check (experimental) `ousast pre-push` analyses the commits a `git push` would publish, or an explicit `--base`/`--head` pair. It compares head with base. Only new or worsened defects become candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`, [Engine and pre-push](ops/README.md#explicit-local-replay). ```mermaid flowchart TD input["git push refs on stdin, or --base / --head"] --> resolve["Resolve both commit identities"] resolve --> snap["Materialise tracked bytes in private scratch storage"] snap --> quick["Quick rules on the changed head lines (advisory, under 1 s)"] quick --> eng{"--engine"} eng -- "off" --> art eng -- "background" --> bg["Detached engine run; result shown on the next push"] bg --> art eng -- "inline, the default" --> engine["Engine on head: changed regions up to --max-regions, ranker, arbiter"] engine --> cmp["Targeted base comparison: new or worsened only"] cmp --> admit{"Admission policy: capability registry"} admit -- "registry empty today" --> diag["Advisory engine findings, not alerts"] admit -- "an enabled capability" --> alert["Normal alert"] diag --> art["Terminal lines + JSON artifact outside the repository"] alert --> art art --> exit{"--mode"} exit -- "advisory, the default" --> allow["Exit 0: allow"] exit -- "blocking, explicit opt-in" --> block["May block the push"] ``` - `--deadline` (30 s by default) bounds preparation, the quick rules, both revisions and reporting together. - The quick rules run first on the changed lines. Their matches are shown as advisory pattern matches, never as alerts, and never block. - The default capability registry is empty until independent qualification passes. So no normal alert is emitted today. An engine finding the push introduced is shown as an advisory engine finding; every other candidate stays diagnostic in the artifact. - A model is called only with an explicit `--model-config`. - The private cache (`--cache-dir`) and the manual install recipe are documented in [Engine and pre-push](ops/README.md#experimental-pre-push-integration). ### First run: install the hook on any repository The hook is advisory. It never rejects a push unless you set `OUSAST_PUSH_MODE=blocking`. A check that cannot run says so in one line and lets the push through: a missing `ousast`, a missing Docker image, an invalid setting, a crash. 1. Install the tool. The package is not on PyPI. Install it from the repository (or from a release wheel on the GitHub releases page): ```bash pip install "git+https://github.com/norandom/OpenUltraSAST" ``` 2. Install the hook in the repository you want checked. Git's effective hook directory is used, including `core.hooksPath`, and that setting is never edited: ```bash cd /path/to/your/repository ousast pre-push install # refuses if a pre-push hook already exists ousast pre-push install --force # moves an existing hook to pre-push.before-ousast and chains it ``` 3. Push as usual. Without the Joern engine, you get the quick rules and one line saying the engine did not run. For the engine, choose one: - a host install with `joern-parse` on the PATH (`pyinfra @local ops/joern.py`, see [Engine and pre-push](ops/README.md)); - Docker: `docker pull ghcr.io/norandom/openultrasast:2.0.1`, then `ousast pre-push install --docker`. The wrapper `ousast-docker` is installed beside the hook and runs the check in the container (repository read-only, no network, `--memory 3g`). The container runs the code of its image's version: the quick-rule tier, the plain skip lines and `--engine` need an image built from a version that has them (`docker build -t openultrasast:dev .`, then `OUSAST_IMAGE=openultrasast:dev`). 4. Remove it with `ousast pre-push uninstall`. Only a hook this tool wrote is removed, and a chained previous hook is restored. A push on the 2026-10-02 proof repository (serve-static, a new `debug.js` that evaluates `req.query.code`, no Joern on the host) prints: ```text Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine. - js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review - js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review No actionable defects reported; analysis is incomplete. Skipped: the changed code has no counterpart in the base revision to compare against Skipped: engine did not run: the code graph could not be built (is Joern installed? ...) Skipped: no base revision was analysed, so no engine finding can be called new or worsened Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details. Details: /home/you/.cache/openultrasast/push/push-12345.json ``` The hook reads these settings from the environment of `git push`: | Variable | Default | Effect | | --- | --- | --- | | `OUSAST_PUSH_DEADLINE` | `30` | seconds for the whole check | | `OUSAST_PUSH_MODE` | `advisory` | `blocking` is the only way a result can reject a push | | `OUSAST_INCOMPLETE_COVERAGE` | `allow` | with `blocking`: `block` also rejects incomplete coverage | | `OUSAST_PUSH_ENGINE` | `inline` | `background`: return after the quick rules, run the engine detached; `off`: quick rules only | | `OUSAST_COMPARISON_BASE` | unset | a local ref to compare a new branch against | | `OUSAST_ARTIFACT_DIR` | `~/.cache/openultrasast/push` | where the JSON results go; outside the repository | | `OUSAST_COMMAND` | `ousast` | the command the hook runs, for example the Docker wrapper | | `OUSAST_IMAGE` | `ghcr.io/norandom/openultrasast:2.0.1` | the Docker wrapper's image | | `OUSAST_DOCKER_MEMORY` | `3g` | the Docker wrapper's memory limit | **A new branch.** Git gives the hook no remote base for a branch the remote does not have yet. The check says so ("new branch: the remote has no base to compare against"), runs the quick rules over the files the branch's last commit changed instead, and says that too. Set `OUSAST_COMPARISON_BASE=main` to compare a new branch against your local `main`. **The engine and the deadline.** The engine does not fit a 30 s deadline today. The 2026-10-02 audit measured 260 s on a 26-file repository; graph reuse is the planned fix. When it cannot finish, one line says so. With `OUSAST_PUSH_ENGINE=background`, the hook returns after the quick rules and starts one detached engine run (`--background-deadline`, 900 s by default). The next push in the same repository prints `Engine result for from the previous push: ...` once. Only one background run happens at a time. The Docker wrapper cannot run the engine in the background, because the container ends with the hook. **The artifact.** Every run writes one JSON file, and the terminal prints its path after `Details:`. It records the following: - `snapshots[].bytes_read`: proof that the input was read; - `quick_tier`: the quick-rule matches, with the files and bytes read; - `admission.coverage_reasons`: the raw skip reasons; - `admission.dispositions`: every engine candidate and its vetoes; - `engine_background`: background runs started or shown; - `timings`. Files accumulate, one per push; delete old ones when you like. ### Skip reasons The terminal prints one `Skipped:` line per distinct reason in plain words. The artifact keeps the raw reason. Repository paths are hex-encoded there, and per-question reasons end in a hash. | Raw reason (artifact) | Terminal line says | What to do | | --- | --- | --- | | `cpg_build_failed`, `cpg_unavailable` | the engine did not run: the code graph could not be built | install Joern, or use the Docker wrapper | | `cpg_empty` | the engine read no code | check that the changed files are in a supported language | | `frontend_unsupported` | no engine frontend for a language in this push | none: that language gets the quick rules at most | | `language_not_covered:::` | language not covered: `` (n changed files, m in the repository) | none: neither the engine nor the quick rules cover it, so silence is not a clean result | | `deadline_exhausted` | the engine did not finish within the deadline | raise `OUSAST_PUSH_DEADLINE`, or set `OUSAST_PUSH_ENGINE=background` | | `change_context_deadline_exhausted` | the deadline ran out while relating the change to the code | as above | | `report_deadline_exhausted` | the deadline ran out while saving the details | as above | | `engine_off`, `engine_background` | the engine was switched off, or runs after the push | none | | `resolution:missing_base` | new branch: the remote has no base to compare against | set `OUSAST_COMPARISON_BASE` | | `resolution:unsupported_base`, `resolution:no_merge_base`, `resolution:ambiguous_base`, `resolution:missing_target`, `resolution:unsupported_target`, `resolution:resolution_error` | the pushed ref or its remote base could not be resolved locally | fetch the remote, then push again | | `base_comparison_unavailable`, `base_unavailable` | no base revision was analysed, so nothing can be called new | see the line before it for the cause | | `base_counterpart_unresolved` | the changed code has no counterpart in the base to compare against | none: expected for new files | | `:ambiguous_line_correspondence`, `:line_correspondence_unavailable`, `:ambiguous_path_rename_correspondence` | changed lines or a renamed file could not be matched to the base | none: findings there stay diagnostic | | `unmatched_added_deleted_path_correspondence` | files were both added and deleted, so a move cannot be told from a new file | push the move separately from other edits | | `declaration_paths_unavailable` | project declaration files could not be listed | rerun | | `capability_unavailable`, `capability_disabled`, `capability_unevaluated`, `capability_ambiguous` | engine findings are not qualified as alerts yet | none: they show as advisory engine findings | | `eligibility:` | the capability registry does not match this build | reinstall a consistent build | | `comparison_unknown`, `base_incomplete` and the other novelty reasons | whether this push introduced some engine findings could not be established | none: they stay diagnostic | | `dynamic_external_or_depth_context_unresolved:...` | n engine checks depend on dynamic or external code the engine cannot follow | none | | `query_failed`, `query_budget_reserved`, `query_too_expensive`, `regions_truncated`, `files_unparsed` | some engine questions were not answered | raise the deadline or `--max-regions` | | `cross_partition_semantics_unresolved` | flows between languages are not followed | none | | `dependency_unresolved`, `change_unsupported` | engine findings depend on unresolved code or have no supported link to the change | none | | `snapshot:` | some files could not be materialised for analysis | see the reason (symlinks, submodules, LFS) | | `_failed:`, `push_input_failed:` | the named step failed | rerun; report it if it repeats | A reason not in this table is still printed as its own words, followed by "see the details file". ### In CI The same check runs on a pushed range in CI. This GitHub Actions job is advisory: it never fails the build, and it uploads the artifact. ```yaml name: ousast pre-push (advisory) on: push jobs: pre-push: runs-on: ubuntu-latest steps: - uses: actions/checkout@v5 with: fetch-depth: 0 - uses: actions/setup-python@v5 with: python-version: "3.12" - run: pip install "git+https://github.com/norandom/OpenUltraSAST" - name: Check what this push changed if: github.event.before != '0000000000000000000000000000000000000000' run: | ousast pre-push . --base ${{ github.event.before }} --head ${{ github.sha }} \ --artifact "$RUNNER_TEMP/ousast-pre-push.json" --engine off --deadline 120 || true - uses: actions/upload-artifact@v4 if: always() with: name: ousast-pre-push path: ${{ runner.temp }}/ousast-pre-push.json ``` A branch's first push has no `github.event.before` to compare against, and the job skips it. For the engine in CI, run the job in the container image instead of `pip install`, and drop `--engine off`. ### `ousast pre-push --help` ```text usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL] [--comparison-base COMPARISON_BASE] [--model-config MODEL_CONFIG] [--prior-hook PRIOR_HOOK] --artifact ARTIFACT [--cache-dir CACHE_DIR] [--deadline DEADLINE] [--cancellation-allowance CANCELLATION_ALLOWANCE] [--max-regions MAX_REGIONS] [--mode {advisory,blocking}] [--incomplete-coverage {allow,block}] [--engine {inline,background,off}] [--background-deadline BACKGROUND_DEADLINE] [--experimental-declarations EXPERIMENTAL_DECLARATIONS] [--experimental-record-vetoes] path Check what a push changes: quick rules on the changed lines, then the Joern engine on head and base within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with 'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall [REPO]'. positional arguments: path the repository to check (the hook passes '.') options: -h, --help show this help message and exit --base BASE explicit replay: the base revision (with --head; instead of --remote) --head HEAD explicit replay: the head revision to check against --base --remote NAME URL Git hook mode: the remote name and URL Git passes to pre-push; the push input is read from stdin --comparison-base COMPARISON_BASE hook mode: a local ref to compare a NEW branch against (the remote has no base for it); the hook reads OUSAST_COMPARISON_BASE --model-config MODEL_CONFIG explicit optional witness selection endpoint configuration --prior-hook PRIOR_HOOK explicitly chain an existing hook with the same input and arguments --artifact ARTIFACT where to write the complete JSON result; must be outside the repository --cache-dir CACHE_DIR reuse compatible local artifacts in a private directory outside the repository --deadline DEADLINE seconds for the whole check: resolution, quick rules, engine and reporting (default 30; hook: OUSAST_PUSH_DEADLINE) --cancellation-allowance CANCELLATION_ALLOWANCE seconds after the deadline to stop work and save the result (default 2) --max-regions MAX_REGIONS most code regions the engine examines, highest-ranked first (default 500) --mode {advisory,blocking} advisory (default) never blocks; blocking is the explicit opt-in that rejects a push for an admitted alert (hook: OUSAST_PUSH_MODE) --incomplete-coverage {allow,block} with --mode blocking only: also block when coverage is incomplete (hook: OUSAST_INCOMPLETE_COVERAGE) --engine {inline,background,off} inline (default): run the Joern engine inside the deadline; background: return after the quick rules and run the engine detached, showing its result on the next run; off: quick rules only --background-deadline BACKGROUND_DEADLINE seconds the detached engine run of --engine background may take (default 900) --experimental-declarations EXPERIMENTAL_DECLARATIONS explicit --base/--head replay only: unreviewed capability declarations used to render the experimental evaluation explanation; never consulted by admission and cannot enable any capability --experimental-record-vetoes explicit --base/--head replay only: record each comparison and admission veto per raw finding instead of applying it; output is labeled experimental, changes no result and enables no hook capability ``` ```text usage: ousast pre-push install [-h] [--force] [--docker] [repository] Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory (core.hooksPath is honoured, never edited). positional arguments: repository the repository (default: current directory) options: -h, --help show this help message and exit --force install: replace our hook, or move a foreign hook to pre-push.before-ousast and chain it; uninstall: remove a hook this tool did not write --docker also install the ousast-docker wrapper beside the hook, so the check runs in the container image ``` ```text usage: ousast pre-push uninstall [-h] [--force] [repository] Remove the OpenUltraSAST pre-push hook and restore a chained previous hook. positional arguments: repository the repository (default: current directory) options: -h, --help show this help message and exit --force install: replace our hook, or move a foreign hook to pre-push.before-ousast and chain it; uninstall: remove a hook this tool did not write ```