marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

merge(pre-push): operational readiness increment 1 -- never block, never hide, quick tier, new files, background engine, install

Browse Source 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-03 07:29:17 +02:00
co-authored by Claude Opus 5.5
parent 49e11e0252 2ec8071c26
commit 432fd2947e
31 changed files with 3222 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/pre-push-safety-net/requirements.md
+37
View File
@@ -147,6 +147,43 @@ with Express-style routes and middleware, including vulnerable, fixed and benign
frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or
unsupported-property results separately from demonstrated vulnerability detection.
## Requirement 9: Operate as an advisory safety net on any repository
Added 2026-10-02, maintainer-approved, from the operational-readiness audit. The maintainer's
definition: a user installs the hook on ANY repository and gets honest, advisory-only results;
nothing ever blocks a push unless blocking was explicitly chosen.
9.1 (added 2026-10-02, maintainer-approved) When the installed hook cannot run the check, because the
`ousast` executable is missing or an argument or setting is invalid, the hook shall allow the push and
print one line saying the check did not run and how to fix it; only an explicit `--mode blocking`
opt-in may turn an analysis result into a rejected push.
9.2 (added 2026-10-02, maintainer-approved) When a check is skipped or incomplete, the terminal output
shall name each distinct reason in plain words, one line per reason, with repository paths decoded and
no opaque hash tokens; a language with no engine frontend and no quick rules shall be named with its
file count; a first push of a new branch shall say there was no base to compare against and what was
checked instead. The JSON artifact keeps the raw reason values.
9.3 (added 2026-10-02, maintainer-approved) Before the engine, within the same deadline, the Safety Net
shall run the shipped quick pattern rules over the head side of the changed lines and show their
findings as advisory pattern matches, labeled separately from admitted alerts. This amends 3.2 for this
tier only: quick-rule findings are shown, never admitted, and never block.
9.4 (added 2026-10-02, maintainer-approved) When a finding's operation lies in a file that is new in the
head revision, the comparison shall classify it as new rather than unknown; an engine finding whose
novelty is established but whose capability is not yet qualified shall be shown as an advisory engine
finding, not an admitted alert. Genuinely unknown comparisons keep their vetoes.
9.5 (added 2026-10-02, maintainer-approved) When the engine cannot finish within the deadline, one line
shall say so; on explicit request (`--engine background`) the Safety Net shall start the engine
detached after the push, record its result in the artifact directory, and show it on the next run as
the engine result for that head from the previous push.
9.6 (added 2026-10-02, maintainer-approved) The hook installer and remover shall ship in the wheel
(`ousast pre-push install|uninstall`), refuse to overwrite an existing hook unless forced, and a
hook-callable container wrapper shall pass the push input and artifact path through; the documentation
shall give a first-run guide, a glossary of skip reasons, a CI recipe and help text for every flag.
## Accepted experimental acceptance profile
These values are the initial implementation/evaluation profile accepted for task generation, not
.kiro/specs/pre-push-safety-net/tasks.md
+51 -2
View File
@@ -475,7 +475,7 @@ Preserve all completed task evidence below.
- _Depends: 13.1_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
- [x] 14. Ask every declared family whether it establishes anything (precondition for M4)
- Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
- [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
- Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
@@ -575,7 +575,7 @@ Preserve all completed task evidence below.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
- [x] 13.4 Let a question inside one partition complete, whatever else the repository holds
- **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
- The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
- **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
@@ -587,6 +587,7 @@ Preserve all completed task evidence below.
- _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
- _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
- _Checked 2026-10-02 per the operational-readiness audit:_ implemented and verified on PMPro (120 of 127, above). 13.3 stays open on its own blocker; this box was left unchecked only because it waits on it.
- _Depends: 13.3_
- _Requirements: 2.1, 2.2, 6.3, 8.1_
@@ -619,6 +620,54 @@ Preserve all completed task evidence below.
- _Requirements: 8.1, 8.2_
- [x] 17. Operational readiness (2026-10-02)
- **Named by the 2026-10-02 operational-readiness audit**, maintainer-approved as the first increment. The maintainer's definition: a user installs the hook on ANY repository and gets honest, advisory-only results; nothing ever blocks a push. The audit found the plumbing sound (snapshots, resolution, deadline accounting, advisory exit 0, the JSON artifact) and five things in the way: the hook can block a push, quick rules are not part of pre-push, the engine cannot finish a 26-file repository inside 30 s, the terminal hides every skip reason, and there is no supported install path without the source checkout. Before numbers for every item are the audit's live runs on records (Python), serve-static (JavaScript) and unrolled/secure (Go), each with one planted commit.
- This group is plumbing and honesty. It enables no capability, edits no rule for a planted sample, and leaves graph reuse (the engine speed work decided in 11.6) as a later task.
- _Done 2026-10-02:_ all seven subtasks done. The engine still does not fit the 30 s hook deadline (graph reuse, the 11.6 direction, stays a later task), and plane verdicts after the push were not in this increment.
- _Requirements: 9.1–9.6_
- [x] 17.1 Never block a push the user did not choose to block
- The installed hook exits 0 with one line when `ousast` is missing (the check did not run, and how to install). In hook (stdin) mode, an argument or usage error (an invalid `OUSAST_PUSH_DEADLINE`, a cache directory inside the repository) is advisory: one line and exit 0. `--mode blocking` stays the explicit opt-in that can reject a push, and keeps its usage errors. The prior-hook input-retention failure keeps exit 1, because it is the chained hook's verdict that cannot be obtained, not ours.
- Files: `ops/pre-push`, `src/openultrasast/cli.py`. Tests: `tests/test_push_hook.py` (a push with `ousast` absent from PATH succeeds and prints the line; a bad deadline in hook mode exits 0; blocking mode keeps exit 2).
- Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0.
- _Done 2026-10-02:_ the hook prints one line and exits 0 (chaining a prior hook if present) when `ousast` (or `OUSAST_COMMAND`) is missing; in hook mode without `--mode blocking`, argparse/usage errors and unexpected exceptions print one line and exit 0. Tests: `test_missing_ousast_never_blocks_the_push`, `test_hook_mode_usage_errors_are_advisory_unless_blocking` (blocking keeps exit 2), and `test_cli_refuses_the_flag_outside_explicit_replay` updated (the experimental flag is still refused in hook mode, now advisory). Gates: 1987 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.1, 5.3, 5.4_
- [x] 17.2 Name every skipped check in plain words
- One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: <lang> (<n> files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`).
- Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`.
- Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch.
- _Done 2026-10-02:_ new `push/explain.py` turns every raw coverage reason into one plain `Skipped:` line per distinct kind (hex paths decoded, sha256 tokens dropped and counted, unknown reasons still printed); the existing one-line `Notice:` keeps its next action and gains an engine-missing variant. `language_not_covered:<lang>:<changed>:<total>` is recorded for a changed language with no engine frontend and no quick rules; `ousast scan` prints `notice: language not covered: go (N files)` and its report says so. `reports.py` renders the eight previously dropped degradations and prints any reason it has no sentence for. The hook passes `OUSAST_COMPARISON_BASE`. The new-branch line names the base it lacks; what was checked instead lands with the quick tier (17.3). Tests: `tests/test_push_explain.py` (6), two in `tests/test_reports.py`; module-audit manifest regenerated for the new module. Gates: 1995 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.2, 5.2, 8.2_
- [x] 17.3 Run the quick rules as the fast first tier
- Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block.
- Files: `src/openultrasast/push/quick.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_quick.py` (new).
- Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
- _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.3_
- [x] 17.4 Classify a file new in head as new, not unknown
- `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
- Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
- Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
- _Done 2026-10-02:_ `ChangeContext.added_paths` records Git status-A paths (rename targets excluded). In `_compare_evidence`, after every existing context/head/deadline check, an operation in an added file is `new`/`file_added_in_head` when the claim is entailed, undischarged and has change evidence, and no base answer holds the same operation (else `operation_moved_within_change`); an unpaired add/delete still blocks as `change_context_unresolved`. Admission accepts `(new, file_added_in_head)`; with the empty registry such a candidate's only veto is `capability_unavailable`, and the terminal now shows it under "Engine: N finding(s) this push introduced. Advisory ... not an alert." Any comparison, evidence or location veto keeps a candidate out of that line. Tests: `tests/test_push_new_file.py` (9). Live engine evidence depends on host memory (17.7). Gates: 2007 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.4, 3.4_
- [x] 17.5 Say when the engine cannot finish, and offer it in the background
- One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
- Files: `src/openultrasast/push/background.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/cli.py`, `ops/pre-push`. Tests: `tests/test_push_background.py` (new).
- Evidence: a hook push with `OUSAST_PUSH_ENGINE=background` followed by a second push showing the previous result.
- _Done 2026-10-02:_ `--engine inline|background|off` (hook: `OUSAST_PUSH_ENGINE`) and `--background-deadline` (default 900 s). `off` and `background` run the quick tier and record `engine_off`/`engine_background` with a plain skip line; the deadline line names the deadline and the background option. New `push/background.py` starts one detached explicit replay (`start_new_session`, stdin closed, output to a log) per artifact directory and repository (lock file with a live-pid check: a second start reports busy), writes its artifact under `<artifact dir>/engine/<repo key>/<head>.json`, and the next run prints "Engine result for <head> from the previous push: ..." once, then marks it shown. Not usable through the Docker wrapper (the container exits with the hook); documented in 17.6. Tests: `tests/test_push_background.py` (4, one launching a real detached run). Gates: 2011 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.5, 4.1, 4.2_
- [x] 17.6 Install, Docker and documentation for a first-time user
- `ousast pre-push install|uninstall [--force] [--docker] [REPO]` ships the hook as package data, honours `core.hooksPath`, refuses an existing hook unless `--force` (which chains it as `pre-push.before-ousast`), and works from a pip install. `ops/ousast-docker` is a hook-callable script (not a shell function) that passes stdin and the artifact path through. The `openultrasast:dev` image is rebuilt only if `df -h /` shows at least 4 GB free; otherwise the rebuild command is documented and the record says it was not rebuilt. `docs/scanning.md` gains a first-run guide, a glossary of skip reasons, a GitHub Actions recipe and full `--help` text; README tiers 1 and 2 updated, pulling `ghcr.io/norandom/openultrasast:2.0.1` where the wrapper allows.
- Files: `src/openultrasast/push/hook/pre-push`, `src/openultrasast/push/install.py` (new), `src/openultrasast/cli.py`, `ops/ousast-docker`, `docs/scanning.md`, `README.md`, `ops/README.md`. Tests: `tests/test_push_install.py` (new), `mkdocs build --strict`.
- Evidence: install from the package into a scratch clone, push through it, uninstall restores the prior hook.
- _Done 2026-10-02:_ `ousast pre-push install|uninstall [REPO] [--force] [--docker]` (`push/install.py`) installs the hook shipped as package data (`openultrasast/push/hook/pre-push`, byte-identical to `ops/pre-push`, verified in the built wheel), honours `core.hooksPath` without editing it, refuses an existing hook unless `--force` (ours is replaced; a foreign hook moves to `pre-push.before-ousast` and is chained), and uninstall removes only a hook it wrote and restores the chained one. `ops/ousast-docker` (also package data; `--docker` installs it beside the hook, which then uses it) is a script that mounts the repository read-only and the artifact directory at their own absolute paths, passes stdin with `-i`, runs as the user with no network and `--memory 3g`, defaults to `ghcr.io/norandom/openultrasast:2.0.1`, never pulls inside a hook, and exits 0 unless `OUSAST_PUSH_MODE=blocking`. The hook passes `--engine`/`--comparison-base` only when set, so an older ousast or image still runs. Every pre-push flag has help text; `docs/scanning.md` has the first-run guide, settings table, new-branch and background notes, the artifact keys, a skip-reason glossary (a test keeps it in sync with `push/explain.py`), a GitHub Actions recipe and the full `--help` text; README tiers 1 and 2 and `ops/README.md` updated (pull and tag the GHCR image instead of building). **Image not rebuilt:** `df -h /` showed 1.1 GB free (< 4 GB); rebuild with `docker build -t openultrasast:dev .`. Tests: `tests/test_push_install.py` (11) and the glossary test. Gates: 2022 passed, 42 skipped; ruff, format, mypy clean; `mkdocs build --strict` exit 0.
- _Requirements: 9.6, 7.4_
- [x] 17.7 Prove it on the audit's three repositories
- Rerun records, serve-static and unrolled/secure (same planted commits) with the new build in quick mode and with the hook installed, plus one Docker engine run on serve-static if free memory >= 3 GB and free disk >= 1 GB. Prove each run read its input (files, bytes).
- Files: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json`. Evidence: before (the audit) and after: exit codes, wall time against the deadline, findings shown, every skip line printed. A planted sample still missed is a coverage gap, recorded, never a rule edit.
- _Done 2026-10-02:_ `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (before = the audit, after = this build installed from its wheel into a scratch venv). 10 hook pushes and 6 replays, all exit 0, 0.69-1.21 s against 30 s on the host; the quick tier read exactly the changed file and every head snapshot read the full tree (records 43,577 B, serve-static 61,525 B, secure 103,023 B). The planted JS `eval` in the new `debug.js` surfaces: quick tier on the host, and as two advisory engine findings classified `new`/`file_added_in_head` in one Docker engine run (190.9 s against a 300 s deadline; MemAvailable 3.28 GB, 1.16 GB disk). **The planted Python `%`-formatted SQL is still missed**: no quick rule matches it and the host has no engine. Recorded as a coverage gap, no rule edited. Go is named as not covered. The proof caught a wrong install hint (`pip install openultrasast`, not on PyPI), corrected in the hook in this commit.
- _Requirements: 9.1–9.6, 6.3_
## Implementation Notes
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
README.md
+25 -7
View File
@@ -58,6 +58,13 @@ uv run ousast scan . --mode quick --fail-on verified # CI: exit 1 on any evide
This runs the language-scoped pattern rules, entry-point reachability hints, ranking,
verification and scoring. It is deterministic and reproducible. It needs no network and no keys.
The same install gives you the advisory pre-push hook for any repository. It runs the quick rules
on the lines a push changes, names every check it could not run, and never blocks a push:
```bash
cd /path/to/your/repository && ousast pre-push install # remove: ousast pre-push uninstall
```
What it cannot do:
- There is no engine. Nothing follows a value across statements or files.
@@ -70,12 +77,20 @@ the overlay.
### Tier 2: standard and deep scans (Docker)
`standard` and `deep` need Joern. The Docker image `openultrasast:dev` ships three things in one
image: the tool, Joern v4.0.625 and `php-cli` (the PHP frontend needs it). You build it locally
from the `Dockerfile`; there is no published image yet. It is big: a JRE plus about 2 GB of Joern.
`standard` and `deep` need Joern. The Docker image ships three things in one image: the tool,
Joern v4.0.625 and `php-cli` (the PHP frontend needs it). Each release publishes it to GHCR. It is
big: a JRE plus about 2 GB of Joern. Pull it instead of building, and tag it with the name the
shell wrapper uses:
The shell wrapper builds the image on first use. Then it runs `ousast` in the container. Your
directory is mounted read-only and the container network is off.
```bash
docker pull ghcr.io/norandom/openultrasast:2.0.1
docker tag ghcr.io/norandom/openultrasast:2.0.1 openultrasast:dev # or build: docker compose build
```
The shell wrapper runs `ousast` in the container. It builds the image only if `openultrasast:dev`
is missing. Your directory is mounted read-only and the container network is off. For the pre-push
hook, use `ousast pre-push install --docker`: Git cannot call a shell function, so it installs the
script `ousast-docker` beside the hook.
```bash
source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1
@@ -158,8 +173,11 @@ loads settings. Every run writes these files under `<target>/.openultrasast/runs
and `trace/events.jsonl`.
`ousast pre-push` (experimental) analyses the commits a push would publish against their base.
Only new or worsened defects become candidates. It is advisory by default. Its capability registry
is empty, so it emits no normal alert today.
First, the quick rules run on the changed lines. Then the engine compares head with base, and only
new or worsened defects become candidates. It is advisory by default, and it prints one line per
check it could not run. Its capability registry is empty, so it emits no normal alert today. An
engine finding the push introduced is shown as advisory. Install, settings, skip reasons and a CI
recipe: [docs/scanning.md](docs/scanning.md#first-run-install-the-hook-on-any-repository).
Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md).
benchmarks/measurements/2026-09-08-module-audit.json
+50 -4
View File
@@ -1,7 +1,7 @@
{
"modules": 151,
"modules": 155,
"counts": {
"load_bearing": 146,
"load_bearing": 150,
"standalone_capability": 5
},
"orphaned": [],
@@ -229,6 +229,8 @@
"model.report",
"pairs",
"plane.tasks.alerts",
"push.quick",
"push.runner",
"rank",
"regress.candidate",
"reports",
@@ -560,6 +562,7 @@
"pairs",
"plane.tasks.alerts",
"preprocess",
"push.quick",
"push.runner"
]
},
@@ -579,6 +582,7 @@
"cli",
"cpg.backend",
"cpg.session",
"findings",
"learn.cli",
"learn.features",
"learn.labels",
@@ -602,6 +606,7 @@
"push.cache",
"push.contracts",
"push.policy",
"push.quick",
"push.report",
"push.reuse",
"push.runner",
@@ -649,6 +654,7 @@
"push.cache",
"push.contracts",
"push.policy",
"push.quick",
"push.report",
"push.reuse",
"push.runner",
@@ -716,7 +722,9 @@
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"model.scan"
"findings",
"model.scan",
"push.runner"
]
},
{
@@ -1093,6 +1101,7 @@
"improve.evolve",
"improve.validator",
"plane.tasks.loop",
"push.quick",
"scoring.project_score",
"semantic.overlay"
]
@@ -1132,6 +1141,7 @@
"plane.tasks.alerts",
"plane.tasks.features",
"plane.tasks.repo_facts",
"push.quick",
"push.runner",
"rank",
"regress.candidate",
@@ -1186,6 +1196,7 @@
"importers": [
"cli",
"push.assistance",
"push.background",
"push.eligibility",
"push.policy",
"push.report",
@@ -1203,6 +1214,14 @@
"push.runner"
]
},
{
"module": "push.background",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.runner"
]
},
{
"module": "push.cache",
"classification": "load_bearing",
@@ -1234,6 +1253,23 @@
"push.runner"
]
},
{
"module": "push.explain",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.report",
"push.runner"
]
},
{
"module": "push.install",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"cli"
]
},
{
"module": "push.policy",
"classification": "load_bearing",
@@ -1245,12 +1281,21 @@
"push.vetoes"
]
},
{
"module": "push.quick",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.runner"
]
},
{
"module": "push.report",
"classification": "load_bearing",
"reason": "pre-push-safety-net Req 5.1/5.2/5.5/7.3: compact output and complete artifacts; runner integration pending",
"importers": [
"push.assistance",
"push.background",
"push.runner"
]
},
@@ -1315,7 +1360,8 @@
"improve.evolve",
"map_gate",
"pairs",
"plane.tasks.alerts"
"plane.tasks.alerts",
"push.quick"
]
},
{
benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json
+792
View File
@@ -0,0 +1,792 @@
{
"id": "2026-10-02-pre-push-operational-1",
"task": "pre-push-safety-net 17.7",
"requirements": [
"9.1",
"9.2",
"9.3",
"9.4",
"9.5",
"9.6",
"6.3"
],
"question": "Does the hook installed on any repository give honest, advisory-only results, and never block a push?",
"subjects": "The audit's three shallow clones (kennethreitz/records, expressjs/serve-static, unrolled/secure), each with the same planted commit c2 on c1, copied fresh from the audit's scratch clones.",
"instrument_proof": "Every replay and hook push records snapshots[].bytes_read and quick_tier files/bytes_read. The head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head (records 43,577; serve-static 61,525; secure 103,023), and the quick tier reads exactly the changed source files.",
"host": "Joern is not on the host, so host runs cannot run the engine; that is the case a first-time pip user hits, and the output must say so. The Docker engine run uses the 2026-09-14 openultrasast:dev image with the worktree src mounted read-only, as the audit did; the image was not rebuilt (1.1 GB disk free, below the 4 GB threshold).",
"before": {
"source": "/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready/gap-report.md (audit at main 49e11e0)",
"replay_host": {
"records": {
"exit": 0,
"wall_seconds": 1.06,
"files_bytes_read": [
21,
43577
],
"findings_shown": 0,
"terminal": "generic: Some checks could not be completed. Review the comparison details...",
"skip_reasons_named_in_terminal": 0
},
"serve-static": {
"exit": 0,
"wall_seconds": 1.1,
"files_bytes_read": [
26,
61525
],
"findings_shown": 0,
"terminal": "same generic line",
"skip_reasons_named_in_terminal": 0
},
"secure": {
"exit": 0,
"wall_seconds": 1.04,
"files_bytes_read": [
19,
103023
],
"findings_shown": 0,
"terminal": "same generic line; Go not named",
"skip_reasons_named_in_terminal": 0
}
},
"docker_engine_serve_static": {
"deadline_30": {
"exit": 0,
"wall_seconds": 31.3,
"terminal": "Analysis did not finish. Rerun with a longer analysis deadline."
},
"deadline_300": {
"exit": 0,
"wall_seconds": 261.7,
"raw_engine_findings": 2,
"admitted": 0,
"dispositions": "injection and output_encoding at debug.js:2: comparison_unknown/base_incomplete + capability_unavailable",
"terminal": "No actionable defects reported; Details: /out/... (container path)"
}
},
"quick_scan": {
"serve-static": {
"exit": 0,
"wall_seconds": 0.46,
"findings": "js-eval, js-xss-response at debug.js:2"
},
"records": {
"exit": 0,
"wall_seconds": 0.49,
"findings": "2 pre-existing (setup.py os-command); planted %-formatted SQL missed"
},
"secure": {
"exit": 0,
"wall_seconds": 0.48,
"findings": 0,
"terminal": "No quick-mode findings were emitted (silent zero for Go)"
}
},
"hook": {
"ousast_not_on_path": {
"push_exit": "rejected",
"terminal": ".git/hooks/pre-push: 13: exec: ousast: not found"
},
"first_push_new_branch": {
"push_exit": 0,
"wall_seconds": 0.58,
"terminal": "No analysis result is available. (does not say new branch)"
},
"update": {
"push_exit": 0,
"wall_seconds": 1.36,
"terminal": "No actionable defects reported; analysis is incomplete."
},
"bad_OUSAST_PUSH_DEADLINE_or_cache_inside_repo": {
"exit": 2,
"effect": "would block the push"
},
"install": "ops/install-pre-push only (checkout); not in the wheel"
}
},
"verdict": {
"never_blocks": "PASS in this sample: 10 hook pushes across the three repositories all exited 0, including `ousast` absent from PATH (rejected before) and OUSAST_PUSH_DEADLINE=abc (exit 2 before). Blocking stays an explicit opt-in (unit-tested, not exercised here).",
"never_hides": "PASS in this sample: every incomplete run printed one plain `Skipped:` line per reason (2-5 per run), with no hex path or sha256 token. Go is named as not covered (`language not covered: go (1 changed file(s), 10 in the repository)`, and `notice: language not covered: go (10 files)` in `scan --mode quick`). The first push of a new branch says there is no base and what it checked instead.",
"quick_tier_within_deadline": "On the host, every replay and hook push finished in 0.69-1.21 s against the 30 s deadline. The quick tier read exactly the changed source file (records 1 file / 17,449 B; serve-static 1 / 87 B; secure 1 / 193 B).",
"planted_js_eval": "SURFACES. Quick tier: js-eval (CWE-95) and js-xss-response (CWE-79) at debug.js:2 on the host, 0.9 s, in replay and in the hook push. Engine (Docker, deadline 300): injection and output_encoding at debug.js:2, now novelty `new` / `file_added_in_head` (before: `unknown` / `base_incomplete`). The only veto is `capability_unavailable`, so it is shown as an advisory engine finding, not an alert.",
"planted_python_sql": "STILL MISSED. No quick rule matches the %-formatted db.query string in records.py, and the host has no Joern, so nothing in the pre-push path reports it. Recorded as a coverage gap, with no rule edit.",
"planted_go_command": "NOT DETECTED, NOW NAMED: Go has no engine frontend and no quick rules.",
"engine_deadline": "Not solved. The Docker engine took 190.9 s wall time (head 107.6 s, base 80.7 s) against a 300 s deadline. The audit measured 261.7 s on a different image/src combination. A 30 s hook deadline still cannot hold it. --engine background starts it after the push, and the next push showed the earlier result. On the host that result says 'the engine did not run (Joern unavailable)', which is honest."
},
"coverage_gaps": [
"records: the planted Python SQL injection (`db.query(\"... '%s'\" % name)`) is matched by no quick rule, before or after. Recorded as a quick-rule coverage gap; no rule was edited for a planted sample (that would be hand-tuning). On the host the engine cannot run, so nothing in the pre-push path reports it.",
"secure: Go has no engine frontend and no quick rules. It is now named in the terminal (language not covered: go), not detected.",
"The engine still cannot finish inside the 30 s hook deadline (the audit measured 260 s on 26 files); --engine background moves it after the push, graph reuse remains the later speed task."
],
"after": {
"replay_host": {
"records": {
"pre-push --engine inline": {
"exit": 0,
"wall_seconds": 1.206878771,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"changed lines could not be matched one-to-one to base lines in records.py",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
],
"snapshots_files_bytes": [
[
21,
43577
],
[
21,
43482
]
],
"quick_tier_files_bytes": [
[
1,
17449
]
],
"quick_tier_seconds": [
0.041
],
"coverage_reasons_raw": [
"7265636f7264732e7079:ambiguous_line_correspondence",
"base_counterpart_unresolved",
"cpg_build_failed",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.483
},
"pre-push --engine off": {
"exit": 0,
"wall_seconds": 0.954010589,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"changed lines could not be matched one-to-one to base lines in records.py",
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
],
"snapshots_files_bytes": [
[
21,
43577
],
[
21,
43482
]
],
"quick_tier_files_bytes": [
[
1,
17449
]
],
"quick_tier_seconds": [
0.042
],
"coverage_reasons_raw": [
"7265636f7264732e7079:ambiguous_line_correspondence",
"engine_off",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.498
},
"scan --mode quick": {
"exit": 0,
"wall_seconds": 0.407089313,
"terminal": [
"file_targets=9",
"findings=2"
]
}
},
"serve-static": {
"pre-push --engine inline": {
"exit": 0,
"wall_seconds": 0.892543629,
"deadline_seconds": 30,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [],
"skip_lines": [
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
],
"snapshots_files_bytes": [
[
26,
61525
],
[
25,
61438
]
],
"quick_tier_files_bytes": [
[
1,
87
]
],
"quick_tier_seconds": [
0.017
],
"coverage_reasons_raw": [
"base_counterpart_unresolved",
"cpg_build_failed",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.496
},
"pre-push --engine off": {
"exit": 0,
"wall_seconds": 0.878791036,
"deadline_seconds": 30,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [],
"skip_lines": [
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
],
"snapshots_files_bytes": [
[
26,
61525
],
[
25,
61438
]
],
"quick_tier_files_bytes": [
[
1,
87
]
],
"quick_tier_seconds": [
0.016
],
"coverage_reasons_raw": [
"engine_off",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.471
},
"scan --mode quick": {
"exit": 0,
"wall_seconds": 0.394269229,
"terminal": [
"file_targets=4",
"findings=2"
]
}
},
"secure": {
"pre-push --engine inline": {
"exit": 0,
"wall_seconds": 0.762186729,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"engine has no frontend for at least one language in this push",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
],
"snapshots_files_bytes": [
[
19,
103023
],
[
18,
102830
]
],
"quick_tier_files_bytes": [
[
1,
193
]
],
"quick_tier_seconds": [
0.015
],
"coverage_reasons_raw": [
"language_not_covered:go:1:10",
"base_counterpart_unresolved",
"cpg_build_failed",
"frontend_unsupported",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.352
},
"pre-push --engine off": {
"exit": 0,
"wall_seconds": 0.694374205,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
],
"snapshots_files_bytes": [
[
19,
103023
],
[
18,
102830
]
],
"quick_tier_files_bytes": [
[
1,
193
]
],
"quick_tier_seconds": [
0.016
],
"coverage_reasons_raw": [
"language_not_covered:go:1:10",
"engine_off",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.311
},
"scan --mode quick": {
"exit": 0,
"wall_seconds": 0.372512766,
"terminal": [
"file_targets=10",
"findings=0",
"notice: language not covered: go (10 files)"
]
}
}
},
"hook_host": {
"records": {
"new-branch": {
"push_exit": 0,
"wall_seconds": 0.873912782,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"New branch: instead, quick rules checked the files its last commit changed (compared with c925575).",
"Quick rules: no pattern match on changed code (2 file(s), 18416 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
" * [new branch] HEAD~1 -> main"
]
},
"update": {
"push_exit": 0,
"wall_seconds": 0.890046656,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"changed lines could not be matched one-to-one to base lines in records.py",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
" ea42736..ad3d7db HEAD -> main"
]
},
"install_uninstall": [
"install rc=0",
"uninstall rc=0"
]
},
"serve-static": {
"new-branch": {
"push_exit": 0,
"wall_seconds": 0.904302643,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"New branch: instead, quick rules checked the files its last commit changed (compared with 76312e1).",
"Quick rules: no pattern match on changed code (0 file(s), 0 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" * [new branch] HEAD~1 -> main"
]
},
"update": {
"push_exit": 0,
"wall_seconds": 0.92386555,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [],
"skip_lines": [
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" 74be78a..468213d HEAD -> main"
]
},
"no-ousast": {
"push_exit": 0,
"wall_seconds": 0.017937758,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [],
"other_lines": [
"ousast: pre-push check did not run: 'ousast' is not on PATH. Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" 468213d..aad3dfe HEAD -> main"
]
},
"bad-deadline": {
"push_exit": 0,
"wall_seconds": 0.300776,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [],
"other_lines": [
"usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]",
" [--comparison-base COMPARISON_BASE]",
" [--model-config MODEL_CONFIG] [--prior-hook PRIOR_HOOK]",
" --artifact ARTIFACT [--cache-dir CACHE_DIR]",
" [--deadline DEADLINE]",
" [--cancellation-allowance CANCELLATION_ALLOWANCE]",
" [--max-regions MAX_REGIONS]",
" [--mode {advisory,blocking}]",
" [--incomplete-coverage {allow,block}]",
" [--engine {inline,background,off}]",
" [--background-deadline BACKGROUND_DEADLINE]",
" [--experimental-declarations EXPERIMENTAL_DECLARATIONS]",
" [--experimental-record-vetoes]",
" path",
"ousast pre-push: error: argument --deadline: invalid float value: 'abc'",
"ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" aad3dfe..6fcfec7 HEAD -> main"
]
},
"background-1": {
"push_exit": 0,
"wall_seconds": 0.915950565,
"quick_findings_shown": [],
"engine_findings_shown": [
"Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result."
],
"skip_lines": [
"engine checks run in the background after this push; the next push shows their result",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 93 bytes read).",
"No analysis result is available.",
"Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" 6fcfec7..c5b6c4d HEAD -> main"
]
},
"background-2": {
"push_exit": 0,
"wall_seconds": 0.879868436,
"quick_findings_shown": [],
"engine_findings_shown": [
"Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable)."
],
"skip_lines": [
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable).",
"Quick rules: no pattern match on changed code (1 file(s), 99 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" c5b6c4d..3eee5ef HEAD -> main"
]
},
"install_uninstall": [
"install rc=0",
"uninstall rc=0"
]
},
"secure": {
"new-branch": {
"push_exit": 0,
"wall_seconds": 0.96295861,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"New branch: instead, quick rules checked the files its last commit changed (compared with ba42bc3).",
"Quick rules: no pattern match on changed code (1 file(s), 41228 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
" * [new branch] HEAD~1 -> main"
]
},
"update": {
"push_exit": 0,
"wall_seconds": 0.747049238,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"engine has no frontend for at least one language in this push",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
" 740abc8..64739c4 HEAD -> main"
]
},
"install_uninstall": [
"install rc=0",
"uninstall rc=0"
]
}
},
"docker_engine_serve_static": {
"log": [
"MemAvailable=3283MB disk_free=1159MiB",
"base=74be78a8ffad679edfe135c457ce141114d96fcc head=468213d8a737ad03a1f712c792c1db8476e154df",
"exit=0 elapsed=190.895648822s"
],
"exit": 0,
"wall_seconds": 190.895648822,
"deadline_seconds": 300,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [
"Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert."
],
"skip_lines": [
"engine findings are not qualified as alerts yet: no capability has passed independent evaluation",
"the capability registry does not match this build (eligibility provenance mismatch), so no capability can be used",
"20 engine check(s) depend on dynamic or external code the engine cannot follow"
],
"other_lines": [
"Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert.",
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No actionable defects reported; analysis is incomplete.",
"Notice: Some checks are not enabled for normal alerts. Review capability coverage in the details."
],
"snapshots_files_bytes": [
[
26,
61525
],
[
25,
61438
]
],
"quick_tier_files_bytes": [
[
1,
87
]
],
"quick_tier_seconds": [
0.038
],
"coverage_reasons_raw": [
"capability_unavailable",
"eligibility:eligibility_provenance_mismatch",
"dynamic_external_or_depth_context_unresolved:contributor-scan:7ba72698b15c888c0835cad443fc1f177f1d2e13ed2658b17e816cc0b81415de",
"dynamic_external_or_depth_context_unresolved:contributor-scan:392beff95d0e245cbaef2676813d7f69d35599fb32977c8f197e6df46738fc75",
"dynamic_external_or_depth_context_unresolved:contributor-scan:ec16a67e984a2f82c3ce9a04dbc44eae1e950393b1c1c4fae88485446f1e2f95",
"dynamic_external_or_depth_context_unresolved:contributor-scan:0a3bd3f145260792e9a5d4af2546020689cfd08d0b1e82264d7fe31dda7210e7",
"dynamic_external_or_depth_context_unresolved:contributor-scan:1551c000ba35ba4e5488013f51171b27ab10df8b80f29509610670192cff8a30",
"dynamic_external_or_depth_context_unresolved:contributor-scan:5b687fa7647cefd89d064b3e1b5e5a62842f8fd23104900e8af8d5b14fc800fd",
"dynamic_external_or_depth_context_unresolved:contributor-scan:f122da538137e0a3a3e598c24e6af57c5e0b90ff48dacb4fd1b9d82ac20c35ef",
"dynamic_external_or_depth_context_unresolved:contributor-scan:8ab9ddcb1f4431fa3a3c1cce4cc3d289bcb5d363d47aac1bcf35bc7db86ce866",
"dynamic_external_or_depth_context_unresolved:contributor-scan:19f1031c2c010d91fb41e23955cd5f18eaa5ede355e68fab9e18c8249c21c2ee",
"dynamic_external_or_depth_context_unresolved:contributor-scan:d31a861124274a09e3cb8f3398831ca299cc0cfad72ea385083c1fd88b7a2c45",
"dynamic_external_or_depth_context_unresolved:contributor-scan:c8a1c8042befd3ff6f296921666949850a1c54d63c77dd44296e7be96e621127",
"dynamic_external_or_depth_context_unresolved:contributor-scan:3bb50c50f77027f1e0347f490e35fdd0a75ae586fdf92775c753877833a9d099",
"dynamic_external_or_depth_context_unresolved:contributor-scan:c32cb93cad379fee7b814900e6673aa5ae470c27acd55d33f855987bccb1ab8e",
"dynamic_external_or_depth_context_unresolved:contributor-scan:300fba1acc39cb0ab32ebbd66a8674daecf496b3d8c675d27f7f71cf9f788c9e",
"dynamic_external_or_depth_context_unresolved:contributor-scan:40c99dfe6d923602dc3aa6353f3b5bc017c0d1a67595c19ddf8206ef431db23d",
"dynamic_external_or_depth_context_unresolved:contributor-scan:0e99666c9be7c4704f6c6f858844d721ed8f0b4e3978d53073bc2c1b64d3f075",
"dynamic_external_or_depth_context_unresolved:contributor-scan:da903214d018e0b1bd7a3a3b635b3e0fb2a23fd44527f41a9483c0bf3913f2df",
"dynamic_external_or_depth_context_unresolved:contributor-scan:79d89b41b86aadcc0423baa786e0518ccff2a5b7853ff05986682ecbd636af52",
"dynamic_external_or_depth_context_unresolved:contributor-scan:7d3331ca3c06d1013c758d3648bfebb1223385955e77df977266fc621f62e733",
"dynamic_external_or_depth_context_unresolved:contributor-scan:54fd93450c8db7e6daef06fe795babb26726402211ca8f6d0a8679a375786041",
"analysis_incomplete"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 189.327,
"dispositions": [
{
"family": "injection",
"site": "debug.js:2:debug",
"novelty": "new",
"reason": "file_added_in_head",
"vetoes": [
"capability_unavailable"
]
},
{
"family": "output_encoding",
"site": "debug.js:2:debug",
"novelty": "new",
"reason": "file_added_in_head",
"vetoes": [
"capability_unavailable"
]
}
],
"timings": {
"deadline_seconds": 300.0,
"cancellation_allowance_seconds": 2.0,
"resolution_seconds": 0.0,
"provenance_seconds": 0.3,
"snapshot_seconds": 0.3,
"discovery_seconds": 0.3,
"preparation_seconds": 0.6,
"quick_seconds": 0.0,
"head_seconds": 107.6,
"base_seconds": 80.7,
"admission_seconds": 0.0,
"cleanup_seconds": 0.0,
"total_seconds": 189.3
}
}
},
"build": {
"worktree_head": "591818d + hook install-hint fix (this commit)",
"wheel_sha256": "5b725adb052239c2afaec237fecf582f69fc5ea61799dc67f0c07e09e10908c9",
"installed_with": "uv pip install <wheel> into a scratch venv"
},
"inputs": {
"records": {
"language": "python",
"tracked_files": 21,
"tracked_bytes": 43577,
"planted": "records.py: db.query(\"... '%s'\" % name)"
},
"serve-static": {
"language": "javascript",
"tracked_files": 26,
"tracked_bytes": 61525,
"planted": "new debug.js: res.end(String(eval(req.query.code)))"
},
"secure": {
"language": "go",
"tracked_files": 19,
"tracked_bytes": 103023,
"planted": "new run.go: exec.Command(\"sh\", \"-c\", r.URL.Query().Get(\"cmd\"))"
}
},
"notes": [
"The Docker engine run used the worktree src at 591818d mounted over the stale 2026-09-14 openultrasast:dev image (not rebuilt: 1.1 GB disk free, below the 4 GB threshold). `eligibility_provenance_mismatch` still appears; the audit already suspected src/image skew, and it is not investigated here.",
"The run shows host paths after `Details:` because ops/ousast-docker mounts the artifact directory at its own path. Before, it showed the container path /out/...",
"The hook's 'not on PATH' hint first said 'pip install openultrasast', which is not on PyPI. The first proof run caught it. It was corrected to the git URL, and the replay and hook proofs were rerun on the rebuilt wheel recorded under build. The Docker engine run predates the hint fix and does not involve the hook script.",
"Host runs read their input. Each head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head, and the quick tier lists the files it read."
]
}
docs/scanning.md
+274 -9
View File
@@ -131,23 +131,288 @@ candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
flowchart TD
input["git push refs on stdin, or --base / --head"] --> resolve["Resolve both commit identities"]
resolve --> snap["Materialise tracked bytes in private scratch storage"]
snap --> regions["Discover changed regions, up to --max-regions"]
regions --> engine["Evidence ranker and arbiter on head"]
snap --> quick["Quick rules on the changed head lines (advisory, under 1 s)"]
quick --> eng{"--engine"}
eng -- "off" --> art
eng -- "background" --> bg["Detached engine run; result shown on the next push"]
bg --> art
eng -- "inline, the default" --> engine["Engine on head: changed regions up to --max-regions, ranker, arbiter"]
engine --> cmp["Targeted base comparison: new or worsened only"]
cmp --> admit{"Admission policy: capability registry"}
admit -- "registry empty today" --> diag["Diagnostic candidates in the artifact"]
admit -- "registry empty today" --> diag["Advisory engine findings, not alerts"]
admit -- "an enabled capability" --> alert["Normal alert"]
diag --> art["JSON artifact outside the repository"]
diag --> art["Terminal lines + JSON artifact outside the repository"]
alert --> art
art --> exit{"--mode"}
exit -- "advisory, the default" --> allow["Exit 0: allow"]
exit -- "blocking, explicit opt-in" --> block["May block the push"]
```
- `--deadline` (30 s by default) bounds preparation, both revisions and reporting together.
- `--incomplete-coverage block` opts into failing when coverage is incomplete.
- `--deadline` (30 s by default) bounds preparation, the quick rules, both revisions and reporting together.
- The quick rules run first on the changed lines. Their matches are shown as advisory pattern
matches, never as alerts, and never block.
- The default capability registry is empty until independent qualification passes. So no normal
alert is emitted today. Every candidate stays diagnostic.
alert is emitted today. An engine finding the push introduced is shown as an advisory engine
finding; every other candidate stays diagnostic in the artifact.
- A model is called only with an explicit `--model-config`.
- Hook installation (`ops/install-pre-push`) and the private cache (`--cache-dir`) are documented
in [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
- The private cache (`--cache-dir`) and the manual install recipe are documented in
[Engine and pre-push](ops/README.md#experimental-pre-push-integration).
### First run: install the hook on any repository
The hook is advisory. It never rejects a push unless you set `OUSAST_PUSH_MODE=blocking`. A check
that cannot run says so in one line and lets the push through: a missing `ousast`, a missing
Docker image, an invalid setting, a crash.
1. Install the tool. The package is not on PyPI. Install it from the repository (or from a
release wheel on the GitHub releases page):
```bash
pip install "git+https://github.com/norandom/OpenUltraSAST"
```
2. Install the hook in the repository you want checked. Git's effective hook directory is used,
including `core.hooksPath`, and that setting is never edited:
```bash
cd /path/to/your/repository
ousast pre-push install # refuses if a pre-push hook already exists
ousast pre-push install --force # moves an existing hook to pre-push.before-ousast and chains it
```
3. Push as usual. Without the Joern engine, you get the quick rules and one line saying the
engine did not run. For the engine, choose one:
- a host install with `joern-parse` on the PATH (`pyinfra @local ops/joern.py`, see
[Engine and pre-push](ops/README.md));
- Docker: `docker pull ghcr.io/norandom/openultrasast:2.0.1`, then
`ousast pre-push install --docker`. The wrapper `ousast-docker` is installed beside the
hook and runs the check in the container (repository read-only, no network,
`--memory 3g`). The container runs the code of its image's version: the quick-rule
tier, the plain skip lines and `--engine` need an image built from a version that has them
(`docker build -t openultrasast:dev .`, then `OUSAST_IMAGE=openultrasast:dev`).
4. Remove it with `ousast pre-push uninstall`. Only a hook this tool wrote is removed, and a
chained previous hook is restored.
A push on the 2026-10-02 proof repository (serve-static, a new `debug.js` that evaluates
`req.query.code`, no Joern on the host) prints:
```text
Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.
- js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review
- js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review
No actionable defects reported; analysis is incomplete.
Skipped: the changed code has no counterpart in the base revision to compare against
Skipped: engine did not run: the code graph could not be built (is Joern installed? ...)
Skipped: no base revision was analysed, so no engine finding can be called new or worsened
Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.
Details: /home/you/.cache/openultrasast/push/push-12345.json
```
The hook reads these settings from the environment of `git push`:
| Variable | Default | Effect |
| --- | --- | --- |
| `OUSAST_PUSH_DEADLINE` | `30` | seconds for the whole check |
| `OUSAST_PUSH_MODE` | `advisory` | `blocking` is the only way a result can reject a push |
| `OUSAST_INCOMPLETE_COVERAGE` | `allow` | with `blocking`: `block` also rejects incomplete coverage |
| `OUSAST_PUSH_ENGINE` | `inline` | `background`: return after the quick rules, run the engine detached; `off`: quick rules only |
| `OUSAST_COMPARISON_BASE` | unset | a local ref to compare a new branch against |
| `OUSAST_ARTIFACT_DIR` | `~/.cache/openultrasast/push` | where the JSON results go; outside the repository |
| `OUSAST_COMMAND` | `ousast` | the command the hook runs, for example the Docker wrapper |
| `OUSAST_IMAGE` | `ghcr.io/norandom/openultrasast:2.0.1` | the Docker wrapper's image |
| `OUSAST_DOCKER_MEMORY` | `3g` | the Docker wrapper's memory limit |
**A new branch.** Git gives the hook no remote base for a branch the remote does not have yet.
The check says so ("new branch: the remote has no base to compare against"), runs the quick rules
over the files the branch's last commit changed instead, and says that too. Set
`OUSAST_COMPARISON_BASE=main` to compare a new branch against your local `main`.
**The engine and the deadline.** The engine does not fit a 30 s deadline today. The 2026-10-02
audit measured 260 s on a 26-file repository; graph reuse is the planned fix. When it cannot
finish, one line says so. With `OUSAST_PUSH_ENGINE=background`, the hook returns after the quick
rules and starts one detached engine run (`--background-deadline`, 900 s by default). The next push
in the same repository prints `Engine result for <head> from the previous push: ...` once. Only
one background run happens at a time. The Docker wrapper cannot run the engine in the background,
because the container ends with the hook.
**The artifact.** Every run writes one JSON file, and the terminal prints its path after
`Details:`. It records the following:
- `snapshots[].bytes_read`: proof that the input was read;
- `quick_tier`: the quick-rule matches, with the files and bytes read;
- `admission.coverage_reasons`: the raw skip reasons;
- `admission.dispositions`: every engine candidate and its vetoes;
- `engine_background`: background runs started or shown;
- `timings`.
Files accumulate, one per push; delete old ones when you like.
### Skip reasons
The terminal prints one `Skipped:` line per distinct reason in plain words. The artifact keeps the
raw reason. Repository paths are hex-encoded there, and per-question reasons end in a hash.
| Raw reason (artifact) | Terminal line says | What to do |
| --- | --- | --- |
| `cpg_build_failed`, `cpg_unavailable` | the engine did not run: the code graph could not be built | install Joern, or use the Docker wrapper |
| `cpg_empty` | the engine read no code | check that the changed files are in a supported language |
| `frontend_unsupported` | no engine frontend for a language in this push | none: that language gets the quick rules at most |
| `language_not_covered:<lang>:<changed>:<total>` | language not covered: `<lang>` (n changed files, m in the repository) | none: neither the engine nor the quick rules cover it, so silence is not a clean result |
| `deadline_exhausted` | the engine did not finish within the deadline | raise `OUSAST_PUSH_DEADLINE`, or set `OUSAST_PUSH_ENGINE=background` |
| `change_context_deadline_exhausted` | the deadline ran out while relating the change to the code | as above |
| `report_deadline_exhausted` | the deadline ran out while saving the details | as above |
| `engine_off`, `engine_background` | the engine was switched off, or runs after the push | none |
| `resolution:missing_base` | new branch: the remote has no base to compare against | set `OUSAST_COMPARISON_BASE` |
| `resolution:unsupported_base`, `resolution:no_merge_base`, `resolution:ambiguous_base`, `resolution:missing_target`, `resolution:unsupported_target`, `resolution:resolution_error` | the pushed ref or its remote base could not be resolved locally | fetch the remote, then push again |
| `base_comparison_unavailable`, `base_unavailable` | no base revision was analysed, so nothing can be called new | see the line before it for the cause |
| `base_counterpart_unresolved` | the changed code has no counterpart in the base to compare against | none: expected for new files |
| `<path>:ambiguous_line_correspondence`, `<path>:line_correspondence_unavailable`, `<path>:ambiguous_path_rename_correspondence` | changed lines or a renamed file could not be matched to the base | none: findings there stay diagnostic |
| `unmatched_added_deleted_path_correspondence` | files were both added and deleted, so a move cannot be told from a new file | push the move separately from other edits |
| `declaration_paths_unavailable` | project declaration files could not be listed | rerun |
| `capability_unavailable`, `capability_disabled`, `capability_unevaluated`, `capability_ambiguous` | engine findings are not qualified as alerts yet | none: they show as advisory engine findings |
| `eligibility:<reason>` | the capability registry does not match this build | reinstall a consistent build |
| `comparison_unknown`, `base_incomplete` and the other novelty reasons | whether this push introduced some engine findings could not be established | none: they stay diagnostic |
| `dynamic_external_or_depth_context_unresolved:...` | n engine checks depend on dynamic or external code the engine cannot follow | none |
| `query_failed`, `query_budget_reserved`, `query_too_expensive`, `regions_truncated`, `files_unparsed` | some engine questions were not answered | raise the deadline or `--max-regions` |
| `cross_partition_semantics_unresolved` | flows between languages are not followed | none |
| `dependency_unresolved`, `change_unsupported` | engine findings depend on unresolved code or have no supported link to the change | none |
| `snapshot:<reason>` | some files could not be materialised for analysis | see the reason (symlinks, submodules, LFS) |
| `<stage>_failed:<Error>`, `push_input_failed:<Error>` | the named step failed | rerun; report it if it repeats |
A reason not in this table is still printed as its own words, followed by "see the details file".
### In CI
The same check runs on a pushed range in CI. This GitHub Actions job is advisory: it never fails
the build, and it uploads the artifact.
```yaml
name: ousast pre-push (advisory)
on: push
jobs:
pre-push:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install "git+https://github.com/norandom/OpenUltraSAST"
- name: Check what this push changed
if: github.event.before != '0000000000000000000000000000000000000000'
run: |
ousast pre-push . --base ${{ github.event.before }} --head ${{ github.sha }} \
--artifact "$RUNNER_TEMP/ousast-pre-push.json" --engine off --deadline 120 || true
- uses: actions/upload-artifact@v4
if: always()
with:
name: ousast-pre-push
path: ${{ runner.temp }}/ousast-pre-push.json
```
A branch's first push has no `github.event.before` to compare against, and the job skips it. For
the engine in CI, run the job in the container image instead of `pip install`, and drop
`--engine off`.
### `ousast pre-push --help`
```text
usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]
[--comparison-base COMPARISON_BASE] [--model-config MODEL_CONFIG]
[--prior-hook PRIOR_HOOK] --artifact ARTIFACT [--cache-dir CACHE_DIR]
[--deadline DEADLINE] [--cancellation-allowance CANCELLATION_ALLOWANCE]
[--max-regions MAX_REGIONS] [--mode {advisory,blocking}]
[--incomplete-coverage {allow,block}] [--engine {inline,background,off}]
[--background-deadline BACKGROUND_DEADLINE]
[--experimental-declarations EXPERIMENTAL_DECLARATIONS]
[--experimental-record-vetoes]
path
Check what a push changes: quick rules on the changed lines, then the Joern engine on head and
base within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with
'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall
[REPO]'.
positional arguments:
path the repository to check (the hook passes '.')
options:
-h, --help show this help message and exit
--base BASE explicit replay: the base revision (with --head; instead of --remote)
--head HEAD explicit replay: the head revision to check against --base
--remote NAME URL Git hook mode: the remote name and URL Git passes to pre-push; the push
input is read from stdin
--comparison-base COMPARISON_BASE
hook mode: a local ref to compare a NEW branch against (the remote has no
base for it); the hook reads OUSAST_COMPARISON_BASE
--model-config MODEL_CONFIG
explicit optional witness selection endpoint configuration
--prior-hook PRIOR_HOOK
explicitly chain an existing hook with the same input and arguments
--artifact ARTIFACT where to write the complete JSON result; must be outside the repository
--cache-dir CACHE_DIR
reuse compatible local artifacts in a private directory outside the
repository
--deadline DEADLINE seconds for the whole check: resolution, quick rules, engine and reporting
(default 30; hook: OUSAST_PUSH_DEADLINE)
--cancellation-allowance CANCELLATION_ALLOWANCE
seconds after the deadline to stop work and save the result (default 2)
--max-regions MAX_REGIONS
most code regions the engine examines, highest-ranked first (default 500)
--mode {advisory,blocking}
advisory (default) never blocks; blocking is the explicit opt-in that
rejects a push for an admitted alert (hook: OUSAST_PUSH_MODE)
--incomplete-coverage {allow,block}
with --mode blocking only: also block when coverage is incomplete (hook:
OUSAST_INCOMPLETE_COVERAGE)
--engine {inline,background,off}
inline (default): run the Joern engine inside the deadline; background:
return after the quick rules and run the engine detached, showing its
result on the next run; off: quick rules only
--background-deadline BACKGROUND_DEADLINE
seconds the detached engine run of --engine background may take (default
900)
--experimental-declarations EXPERIMENTAL_DECLARATIONS
explicit --base/--head replay only: unreviewed capability declarations
used to render the experimental evaluation explanation; never consulted by
admission and cannot enable any capability
--experimental-record-vetoes
explicit --base/--head replay only: record each comparison and admission
veto per raw finding instead of applying it; output is labeled
experimental, changes no result and enables no hook capability
```
```text
usage: ousast pre-push install [-h] [--force] [--docker] [repository]
Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory
(core.hooksPath is honoured, never edited).
positional arguments:
repository the repository (default: current directory)
options:
-h, --help show this help message and exit
--force install: replace our hook, or move a foreign hook to pre-push.before-ousast and
chain it; uninstall: remove a hook this tool did not write
--docker also install the ousast-docker wrapper beside the hook, so the check runs in the
container image
```
```text
usage: ousast pre-push uninstall [-h] [--force] [repository]
Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.
positional arguments:
repository the repository (default: current directory)
options:
-h, --help show this help message and exit
--force install: replace our hook, or move a foreign hook to pre-push.before-ousast and
chain it; uninstall: remove a hook this tool did not write
```
ops/README.md
+9 -2
View File
@@ -291,13 +291,20 @@ The default capability registry is empty pending independent qualification. Inst
this interface does not make the project rollout-ready. Advisory is the default;
blocking and strict incomplete-coverage handling are separate explicit choices.
Put the installed `ousast` executable on the Git process's PATH. From this tool's
checkout, explicitly install into the desired repository:
The supported install is `ousast pre-push install [REPO] [--force] [--docker]`, shipped in
the wheel; `ousast pre-push uninstall [REPO]` removes it. The first-run guide, settings, skip
reasons and a CI recipe are in [Scanning](../scanning.md#first-run-install-the-hook-on-any-repository).
If `ousast` is not on the Git process's PATH, the hook prints one line and lets the push through.
From this tool's checkout, the script installer remains available:
```sh
ops/install-pre-push /absolute/path/to/repository
```
`ops/ousast-docker` is the hook-callable Docker wrapper (`OUSAST_COMMAND=/path/to/ops/ousast-docker`,
or installed beside the hook by `--docker`). It is a script, not a shell function, so Git can run
it. It passes the push input through and mounts the artifact directory at its own path.
The installer resolves Git's effective hook directory (including `core.hooksPath`),
leaves that configuration untouched, and refuses to overwrite any existing hook.
For an existing hook, inspect it and explicitly move it aside **before** installation:
ops/ousast-docker
Executable
+61
View File
@@ -0,0 +1,61 @@
#!/bin/sh
# OpenUltraSAST in Docker, callable from a Git hook. A script, not a shell function: Git runs
# hooks non-interactively and never sees functions from ~/.bashrc.
#
# OUSAST_COMMAND=/path/to/ousast-docker git push # or: ousast pre-push install --docker
#
# The repository is mounted READ-ONLY at its own absolute path and the artifact directory
# read-write at its own path, so every path in the arguments and in the output ("Details: ...")
# means the same thing inside and outside the container. The push input on stdin is passed
# through (-i). The container runs as your user, with no network and --memory 3g.
#
# Advisory like the hook: when Docker or the image is missing, it says the check did not run
# and exits 0. Pull the image once first; a hook never downloads ~3 GB on its own:
#
# docker pull ghcr.io/norandom/openultrasast:2.0.1
#
# --engine background does not work through this wrapper: the container ends with the hook.
image=${OUSAST_IMAGE:-ghcr.io/norandom/openultrasast:2.0.1}
if ! command -v docker >/dev/null 2>&1; then
echo "ousast-docker: check did not run: docker is not installed. Push continues."
exit 0
fi
if ! docker image inspect "$image" >/dev/null 2>&1; then
echo "ousast-docker: check did not run: image $image is not present." \
"Run 'docker pull $image' once. Push continues."
exit 0
fi
repo=$(pwd -P)
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
previous=
for argument in "$@"; do
if [ "$previous" = "--artifact" ]; then
artifact_dir=$(dirname -- "$argument")
fi
previous=$argument
done
mkdir -p -- "$artifact_dir" || exit 0
artifact_dir=$(cd -- "$artifact_dir" && pwd -P)
# A linked worktree keeps its objects in the main repository's git directory: mount that too.
common=$(git rev-parse --path-format=absolute --git-common-dir 2>/dev/null || echo "$repo/.git")
case "$common" in
"$repo"/*) extra_mount= ;;
*) extra_mount="-v $common:$common:ro" ;;
esac
# shellcheck disable=SC2086 # OUSAST_DOCKER_ARGS and extra_mount are deliberately word-split
docker run --rm -i --network none --memory "${OUSAST_DOCKER_MEMORY:-3g}" \
--user "$(id -u):$(id -g)" -e HOME=/tmp -e TMPDIR=/work --tmpfs /work:size=4g \
-v "$repo:$repo:ro" -w "$repo" -v "$artifact_dir:$artifact_dir" $extra_mount \
${OUSAST_DOCKER_ARGS:-} "$image" "$@"
status=$?
if [ "$status" -ge 125 ]; then
echo "ousast-docker: check did not run: the container could not start (docker exit $status). Push continues."
exit 0
fi
# Only an explicit blocking opt-in may turn the container's exit into a rejected push; an older
# image that rejects a newer argument must not block either.
if [ "$status" -ne 0 ] && [ "${OUSAST_PUSH_MODE:-advisory}" != blocking ]; then
echo "ousast-docker: the check exited with status $status; push continues (advisory)."
exit 0
fi
exit "$status"
ops/pre-push
+32 -11
View File
@@ -1,17 +1,38 @@
#!/bin/sh
# Explicitly copy using the non-overwriting recipe in ops/README.md.
# A prior hook is chained only after you explicitly move it to this sibling path.
# OpenUltraSAST pre-push safety net. Advisory: nothing here rejects a push unless
# OUSAST_PUSH_MODE=blocking is set explicitly. Install with `ousast pre-push install`
# (or ops/install-pre-push from a checkout); remove with `ousast pre-push uninstall`.
# A prior hook is chained only after it was moved to the sibling path "$0.before-ousast".
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
prior="$0.before-ousast"
if [ -e "$prior" ] || [ -L "$prior" ]; then
exec ousast pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
--prior-hook "$prior" --remote "$1" "$2"
# The Docker wrapper installed beside this hook (`ousast pre-push install --docker`) is used
# unless OUSAST_COMMAND names another command; otherwise `ousast` from PATH.
if [ -z "${OUSAST_COMMAND:-}" ] && [ -x "$(dirname -- "$0")/ousast-docker" ]; then
OUSAST_COMMAND="$(dirname -- "$0")/ousast-docker"
fi
exec ousast pre-push . --artifact "$artifact_dir/push-$$.json" \
command=${OUSAST_COMMAND:-ousast}
if ! command -v "$command" >/dev/null 2>&1; then
echo "ousast: pre-push check did not run: '$command' is not on PATH." \
"Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues."
if [ -e "$prior" ] || [ -L "$prior" ]; then
exec "$prior" "$@"
fi
exit 0
fi
remote_name=$1
remote_url=$2
set -- pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
--remote "$1" "$2"
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}"
# Optional settings are passed only when set, so an older ousast (or image) still runs.
if [ -n "${OUSAST_PUSH_ENGINE:-}" ]; then
set -- "$@" --engine "$OUSAST_PUSH_ENGINE"
fi
if [ -n "${OUSAST_COMPARISON_BASE:-}" ]; then
set -- "$@" --comparison-base "$OUSAST_COMPARISON_BASE"
fi
if [ -e "$prior" ] || [ -L "$prior" ]; then
set -- "$@" --prior-hook "$prior"
fi
exec "$command" "$@" --remote "$remote_name" "$remote_url"
src/openultrasast/cli.py
+243 -85
View File
@@ -37,7 +37,7 @@ from .calibration import (
from .complexity import map as complexity_map
from .complexity.ledger import persist_verdicts
from .config import ModelLayerConfig, ObligationsConfig, ResolvedConfig, RetiredConfigError, load_config, load_dotenv
from .findings import StaticFinding, quick_scan_findings, write_findings
from .findings import StaticFinding, quick_scan_findings, uncovered_languages, write_findings
from .fusion import FusionDecision, fuse_findings
from .gate import FALSE_POSITIVE_CEILING, RECALL_FLOOR
from .harness import HarnessRuntime, HarnessTraceWriter, write_harness_config
@@ -103,6 +103,7 @@ class ScanOutcome:
finding_count: int
calibrations_applied: int
exit_code: int
notices: tuple[str, ...] = ()
if TYPE_CHECKING:
@@ -131,28 +132,84 @@ def main(argv: list[str] | None = None) -> int:
def _main(argv: list[str] | None) -> int:
if (argv if argv is not None else sys.argv[1:])[:1] != ["pre-push"]:
given = list(argv if argv is not None else sys.argv[1:])
if given[:1] == ["pre-push"] and given[1:2] in (["install"], ["uninstall"]):
return _hook_install(given[1], given[2:])
if given[:1] != ["pre-push"]:
load_dotenv()
parser = argparse.ArgumentParser(prog="ousast")
subparsers = parser.add_subparsers(dest="command", required=True)
replay_parser = subparsers.add_parser("pre-push", help="check Git push input or replay local revisions (experimental)")
replay_parser.add_argument("path", type=Path)
replay_parser.add_argument("--base")
replay_parser.add_argument("--head")
replay_parser.add_argument("--remote", nargs=2, metavar=("NAME", "URL"))
replay_parser.add_argument("--comparison-base")
replay_parser = subparsers.add_parser(
"pre-push",
help="check Git push input or replay local revisions (experimental, advisory)",
description="Check what a push changes: quick rules on the changed lines, then the Joern engine on head and base "
"within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with "
"'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall [REPO]'.",
)
replay_parser.add_argument("path", type=Path, help="the repository to check (the hook passes '.')")
replay_parser.add_argument("--base", help="explicit replay: the base revision (with --head; instead of --remote)")
replay_parser.add_argument("--head", help="explicit replay: the head revision to check against --base")
replay_parser.add_argument(
"--remote",
nargs=2,
metavar=("NAME", "URL"),
help="Git hook mode: the remote name and URL Git passes to pre-push; the push input is read from stdin",
)
replay_parser.add_argument(
"--comparison-base",
help="hook mode: a local ref to compare a NEW branch against (the remote has no base for it); "
"the hook reads OUSAST_COMPARISON_BASE",
)
replay_parser.add_argument("--model-config", type=Path, help="explicit optional witness selection endpoint configuration")
replay_parser.add_argument("--prior-hook", type=Path, help="explicitly chain an existing hook with the same input and arguments")
replay_parser.add_argument("--artifact", type=Path, required=True)
replay_parser.add_argument(
"--artifact", type=Path, required=True, help="where to write the complete JSON result; must be outside the repository"
)
replay_parser.add_argument(
"--cache-dir", type=Path, help="reuse compatible local artifacts in a private directory outside the repository"
)
replay_parser.add_argument("--deadline", type=float, default=30.0)
replay_parser.add_argument("--cancellation-allowance", type=float, default=2.0)
replay_parser.add_argument("--max-regions", type=int, default=500)
replay_parser.add_argument("--mode", choices=("advisory", "blocking"), default="advisory")
replay_parser.add_argument("--incomplete-coverage", choices=("allow", "block"), default="allow")
replay_parser.add_argument(
"--deadline",
type=float,
default=30.0,
help="seconds for the whole check: resolution, quick rules, engine and reporting (default 30; hook: OUSAST_PUSH_DEADLINE)",
)
replay_parser.add_argument(
"--cancellation-allowance",
type=float,
default=2.0,
help="seconds after the deadline to stop work and save the result (default 2)",
)
replay_parser.add_argument(
"--max-regions", type=int, default=500, help="most code regions the engine examines, highest-ranked first (default 500)"
)
replay_parser.add_argument(
"--mode",
choices=("advisory", "blocking"),
default="advisory",
help="advisory (default) never blocks; blocking is the explicit opt-in that rejects a push for an admitted "
"alert (hook: OUSAST_PUSH_MODE)",
)
replay_parser.add_argument(
"--incomplete-coverage",
choices=("allow", "block"),
default="allow",
help="with --mode blocking only: also block when coverage is incomplete (hook: OUSAST_INCOMPLETE_COVERAGE)",
)
replay_parser.add_argument(
"--engine",
choices=("inline", "background", "off"),
default="inline",
help="inline (default): run the Joern engine inside the deadline; background: return after the quick rules and "
"run the engine detached, showing its result on the next run; off: quick rules only",
)
replay_parser.add_argument(
"--background-deadline",
type=float,
default=900.0,
help="seconds the detached engine run of --engine background may take (default 900)",
)
replay_parser.add_argument(
"--experimental-declarations",
type=Path,
@@ -339,6 +396,20 @@ def _main(argv: list[str] | None) -> int:
"--qualify-population", action="append", default=[], metavar="NAME", help="with --loop: a population whose rows never propose"
)
raw = list(argv if argv is not None else sys.argv[1:])
if _advisory_hook(raw):
# Git hook mode (stdin) without an explicit blocking opt-in: a check that cannot run must
# never reject the push. Say so in one line and allow it (Requirement 9.1).
try:
return _hook_main(parser, argv)
except SystemExit as stop:
if stop.code in (0, None):
raise
print("ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.")
return 0
except Exception as error: # noqa: BLE001 -- an advisory hook reports a crash, it never blocks on one
print(f"ousast: pre-push check did not run: unexpected {type(error).__name__}. Push continues.")
return 0
args = parser.parse_args(argv)
# Diagnosis needs the stage costs, and nothing configures logging, so the default root level of
# WARNING silently dropped every informational line. A run that reports nothing is
@@ -347,77 +418,7 @@ def _main(argv: list[str] | None) -> int:
_configure_logging()
start_profiling() # only when OUSAST_SAMPLE_PROFILE names a file
if args.command == "pre-push":
from .config import PushConfig
from .push.runner import push, replay
try:
if bool(args.base) != bool(args.head) or bool(args.base) == bool(args.remote):
parser.error("provide --base and --head together, or --remote NAME URL for Git stdin")
settings = PushConfig(
comparison_base=args.comparison_base,
deadline_seconds=args.deadline,
cancellation_allowance_seconds=args.cancellation_allowance,
mode=args.mode,
incomplete_coverage_policy=args.incomplete_coverage,
)
options: dict[str, Any] = dict(
artifact=args.artifact,
config=settings,
max_regions=args.max_regions,
cache_dir=args.cache_dir,
model_config=args.model_config,
)
if args.base and args.prior_hook:
parser.error("--prior-hook requires Git stdin mode")
if args.experimental_record_vetoes and not args.base:
parser.error("--experimental-record-vetoes requires explicit --base/--head replay; it is never a hook capability")
if args.experimental_declarations and not args.experimental_record_vetoes:
parser.error("--experimental-declarations requires --experimental-record-vetoes")
prior_data = None
if args.base:
delivery = replay(
args.path,
base=args.base,
head=args.head,
record_vetoes=args.experimental_record_vetoes,
declarations=args.experimental_declarations,
**options,
)
else:
# multiprocessing closes sys.stdin in its child; duplicate the Git pipe first.
with os.fdopen(os.dup(0), "rb") as stream:
if args.prior_hook:
from .model.contracts import ExecutionBudget
from .push.runner import _prepare
shared = ExecutionBudget(time.monotonic() + settings.deadline_seconds, settings.cancellation_allowance_seconds)
try:
prior_data = _prepare(lambda: stream.read(1024 * 1024 + 1), shared)
if len(prior_data) > 1024 * 1024:
raise ValueError("input exceeds hook integration limit")
except (ValueError, RuntimeError, TimeoutError):
print("Hook input could not be retained for all consumers. Retry with complete input and sufficient time.")
return 1
options["execution_budget"] = shared
def updates() -> str:
return (prior_data if prior_data is not None else stream.read(1024 * 1024 + 1)).decode("utf-8")
delivery = push(args.path, updates=updates, remote_name=args.remote[0], remote_url=args.remote[1], **options)
except ValueError as error:
parser.error(str(error))
print(delivery.text, end="", flush=True)
if args.prior_hook and prior_data is not None:
# This is the explicitly chained hook's own behavior, outside analysis.
# Never replace its rejection with our advisory success or impose our timeout on it.
try:
prior = subprocess.run([str(args.prior_hook.resolve()), *args.remote], input=prior_data, cwd=args.path)
except OSError:
print("Existing hook could not be started. Restore its executable path before retrying.")
return 1
if prior.returncode:
return prior.returncode if prior.returncode > 0 else 128 - prior.returncode
return delivery.exit_code
return _pre_push(args, parser)
if args.command == "scan":
return _scan(args.path, args.config, args.mode, args.fail_on)
if args.command == "benchmark":
@@ -463,6 +464,145 @@ def _main(argv: list[str] | None) -> int:
return 2
def _hook_install(action: str, argv: list[str]) -> int:
"""`ousast pre-push install|uninstall [REPO]` from the hook shipped in the package (task 17.6)."""
from .push.install import install, uninstall
parser = argparse.ArgumentParser(
prog=f"ousast pre-push {action}",
description="Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory "
"(core.hooksPath is honoured, never edited)."
if action == "install"
else "Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.",
)
parser.add_argument("repository", nargs="?", type=Path, default=Path("."), help="the repository (default: current directory)")
parser.add_argument(
"--force",
action="store_true",
help="install: replace our hook, or move a foreign hook to pre-push.before-ousast and chain it; "
"uninstall: remove a hook this tool did not write",
)
if action == "install":
parser.add_argument(
"--docker",
action="store_true",
help="also install the ousast-docker wrapper beside the hook, so the check runs in the container image",
)
args = parser.parse_args(argv)
try:
code, lines = (
install(args.repository, force=args.force, docker=args.docker)
if action == "install"
else uninstall(args.repository, force=args.force)
)
except (OSError, ValueError, subprocess.SubprocessError) as error:
print(f"ousast pre-push {action}: {error}", file=sys.stderr)
return 1
for line in lines:
print(line, file=sys.stderr if code else sys.stdout)
return code
def _advisory_hook(argv: list[str]) -> bool:
"""A Git hook invocation (`--remote NAME URL`, stdin input) that did not opt into blocking."""
if argv[:1] != ["pre-push"] or "--remote" not in argv:
return False
blocking = any(a == "--mode=blocking" for a in argv) or any(
a == "--mode" and i + 1 < len(argv) and argv[i + 1] == "blocking" for i, a in enumerate(argv)
)
return not blocking
def _hook_main(parser: argparse.ArgumentParser, argv: list[str] | None) -> int:
args = parser.parse_args(argv)
_configure_logging()
return _pre_push(args, parser)
def _pre_push(args: argparse.Namespace, parser: argparse.ArgumentParser) -> int:
from .config import PushConfig
from .push.runner import push, replay
try:
if bool(args.base) != bool(args.head) or bool(args.base) == bool(args.remote):
parser.error("provide --base and --head together, or --remote NAME URL for Git stdin")
settings = PushConfig(
comparison_base=args.comparison_base,
deadline_seconds=args.deadline,
cancellation_allowance_seconds=args.cancellation_allowance,
mode=args.mode,
incomplete_coverage_policy=args.incomplete_coverage,
)
options: dict[str, Any] = dict(
artifact=args.artifact,
config=settings,
max_regions=args.max_regions,
cache_dir=args.cache_dir,
model_config=args.model_config,
)
if args.base and args.prior_hook:
parser.error("--prior-hook requires Git stdin mode")
if args.experimental_record_vetoes and not args.base:
parser.error("--experimental-record-vetoes requires explicit --base/--head replay; it is never a hook capability")
if args.experimental_declarations and not args.experimental_record_vetoes:
parser.error("--experimental-declarations requires --experimental-record-vetoes")
prior_data = None
if args.base:
delivery = replay(
args.path,
base=args.base,
head=args.head,
record_vetoes=args.experimental_record_vetoes,
declarations=args.experimental_declarations,
engine=args.engine,
background_deadline=args.background_deadline,
**options,
)
else:
# multiprocessing closes sys.stdin in its child; duplicate the Git pipe first.
with os.fdopen(os.dup(0), "rb") as stream:
if args.prior_hook:
from .model.contracts import ExecutionBudget
from .push.runner import _prepare
shared = ExecutionBudget(time.monotonic() + settings.deadline_seconds, settings.cancellation_allowance_seconds)
try:
prior_data = _prepare(lambda: stream.read(1024 * 1024 + 1), shared)
if len(prior_data) > 1024 * 1024:
raise ValueError("input exceeds hook integration limit")
except (ValueError, RuntimeError, TimeoutError):
print("Hook input could not be retained for all consumers. Retry with complete input and sufficient time.")
return 1
options["execution_budget"] = shared
def updates() -> str:
return (prior_data if prior_data is not None else stream.read(1024 * 1024 + 1)).decode("utf-8")
delivery = push(
args.path,
updates=updates,
remote_name=args.remote[0],
remote_url=args.remote[1],
engine=args.engine,
background_deadline=args.background_deadline,
**options,
)
except ValueError as error:
parser.error(str(error))
print(delivery.text, end="", flush=True)
if args.prior_hook and prior_data is not None:
# This is the explicitly chained hook's own behavior, outside analysis.
# Never replace its rejection with our advisory success or impose our timeout on it.
try:
prior = subprocess.run([str(args.prior_hook.resolve()), *args.remote], input=prior_data, cwd=args.path)
except OSError:
print("Existing hook could not be started. Restore its executable path before retrying.")
return 1
if prior.returncode:
return prior.returncode if prior.returncode > 0 else 128 - prior.returncode
return delivery.exit_code
def _scan(path: Path, config_path: Path, mode: str, fail_on: str) -> int:
outcome = _run_scan(path, config_path, mode, fail_on)
_print_scan_outcome(outcome)
@@ -495,6 +635,17 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
"preprocess",
lambda: preprocess_repository(run.target, run.root / "preprocess" / "file_targets.json", static_hints),
)
# A language nothing here covers is named, never a silent zero (pre-push 17.2, Requirement 9.2).
for language, count in uncovered_languages(targets, ruleset, engine=mode != "quick"):
runtime.state["degradations"].append(
{
"stage": "quick_findings",
"reason": "language_not_covered",
"language": language,
"count": count,
"engine": "" if mode == "quick" else " and no engine frontend",
}
)
entry_points = runtime.run_stage("entry_point_mapping", lambda: analyze_entry_points(run.target, targets))
write_entry_points(entry_points, run.root / "mapping" / "entry_points.json")
targets = attach_reachability_hints(targets, entry_points)
@@ -793,6 +944,11 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
finding_count=len(findings),
calibrations_applied=len(applied_calibrations),
exit_code=scan_exit_code(findings, verifications, fail_on, worth_fixing_verdicts=verdict_records),
notices=tuple(
f"language not covered: {item['language']} ({item['count']} files)"
for item in runtime.state["degradations"]
if item.get("reason") == "language_not_covered"
),
)
@@ -867,6 +1023,8 @@ def _print_scan_outcome(outcome: ScanOutcome) -> None:
print(f"ranked_targets={outcome.ranked_target_count}")
print(f"findings={outcome.finding_count}")
print(f"calibrations_applied={outcome.calibrations_applied}")
for notice in outcome.notices:
print(f"notice: {notice}")
def _load_static_hints(sarif_paths: tuple[str, ...]) -> list[object]:
src/openultrasast/findings.py
+25
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import json
import re
from collections.abc import Sequence
from dataclasses import asdict, dataclass
from pathlib import Path
@@ -74,6 +75,30 @@ def quick_scan_findings(
return sorted(findings, key=lambda item: (_severity_sort(item.severity), -item.ranking_priority, item.path))
def uncovered_languages(
targets: Sequence[FileTarget], ruleset: Sequence[PatternRule] | None = None, *, engine: bool = False
) -> list[tuple[str, int]]:
"""Languages present in `targets` that no enabled quick rule (and, with `engine`, no engine frontend) covers.
A repository in such a language produces zero findings whatever it contains; callers name it so that
zero is never read as a clean result.
"""
from .model.partitions import FRONTENDS
rules = PATTERN_RULES if ruleset is None else ruleset
enabled = [rule for rule in rules if rule.status != "disabled"]
if any(not rule.languages for rule in enabled): # a rule without a language list applies to every file
return []
covered = {language for rule in enabled for language in rule.languages}
if engine:
covered |= set(FRONTENDS)
counts: dict[str, int] = {}
for target in targets:
if target.language not in covered and target.language != "unknown":
counts[target.language] = counts.get(target.language, 0) + 1
return sorted(counts.items())
def build_quick_hunter_prompt(target: FileTarget, source_excerpt: str) -> str:
return (
"Review this file for security-relevant issues. Return structured findings only when evidence exists.\n"
src/openultrasast/model/contracts.py
+4
View File
@@ -111,6 +111,9 @@ class ChangeContext(Contract):
# are not encoded or decoded by this field.
path_encoding: Literal["text", "filesystem-bytes-hex"] = "text"
line_correspondences: tuple[LineCorrespondence, ...] = ()
# Paths Git reports as added (status A) in head, in this context's path encoding. A rename
# target is not added. Only a lexical fact: novelty still needs complete head evidence.
added_paths: tuple[str, ...] = ()
def decode_path(self, path: str) -> str:
"""Return the filesystem spelling for matching ordinary question paths.
@@ -130,6 +133,7 @@ class ChangeContext(Contract):
*(s.path for s in self.spans),
*(p for r in self.renames for p in (r.base_path, r.head_path)),
*(p for m in self.line_correspondences for p in (m.base_path, m.head_path)),
*self.added_paths,
]
for path in paths:
try:
src/openultrasast/push/background.py
+167
View File
@@ -0,0 +1,167 @@
"""Run the engine after the push, and show its result on the next one (task 17.5, Requirement 9.5).
The engine cannot finish a small repository inside a 30 s hook deadline (the 2026-10-02 audit
measured 260 s on 26 files). Until graph reuse makes it fast, `--engine background` lets the hook
return after the quick tier and starts one detached explicit replay of the same base and head. That
replay writes an ordinary artifact under the artifact directory; the next run in the same repository
prints "engine result for <head> from the previous push" once, then marks it shown.
One background engine at a time per artifact directory: the host this was built on has 7 GB, and
two Joern JVMs is what this project has seen killed. No speed work happens here.
"""
from __future__ import annotations
import contextlib
import hashlib
import json
import os
import subprocess
import sys
from collections.abc import Mapping
from pathlib import Path
from openultrasast.push.report import CAPABILITY_VETOES
_LOCK = "running.json"
def results_dir(artifact: Path, repository: Path) -> Path:
"""Per-repository directory beside the artifact; the repository path is hashed, never stored raw."""
key = hashlib.sha256(str(repository.resolve()).encode()).hexdigest()[:16]
return artifact.parent / "engine" / key
def _alive(pid: object) -> bool:
if not isinstance(pid, int) or pid <= 0:
return False
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def start(
repository: Path,
*,
base: str,
head: str,
artifact: Path,
deadline_seconds: float,
max_regions: int = 500,
cache_dir: Path | None = None,
) -> dict[str, object]:
"""Start one detached engine replay of base..head; never waits for it and never raises."""
directory = results_dir(artifact, repository)
try:
directory.mkdir(parents=True, exist_ok=True)
lock = directory / _LOCK
try:
running = json.loads(lock.read_text())
except (OSError, ValueError):
running = {}
if isinstance(running, dict) and _alive(running.get("pid")):
return {"status": "busy", "head": str(running.get("head", "")), "directory": str(directory)}
target = directory / f"{head}.json"
log = directory / f"{head}.log"
command = [
sys.executable,
"-m",
"openultrasast.cli",
"pre-push",
str(repository.resolve()),
"--base",
base,
"--head",
head,
"--artifact",
str(target),
"--deadline",
f"{deadline_seconds:g}",
"--max-regions",
str(max_regions),
"--engine",
"inline",
]
if cache_dir is not None:
command.extend(("--cache-dir", str(cache_dir)))
with open(log, "wb") as output:
process = subprocess.Popen( # noqa: S603 -- our own CLI with immutable commit ids, no shell
command,
stdin=subprocess.DEVNULL,
stdout=output,
stderr=subprocess.STDOUT,
cwd=str(repository.resolve()),
start_new_session=True,
close_fds=True,
)
lock.write_text(json.dumps({"pid": process.pid, "head": head, "base": base}))
return {
"status": "started",
"head": head,
"deadline_seconds": deadline_seconds,
"artifact": str(target),
"log": str(log),
}
except Exception as error: # noqa: BLE001 -- a background start that fails is a stated gap, never a block
return {"status": "failed", "reason": type(error).__name__}
def _summary(data: Mapping[str, object]) -> dict[str, object]:
admission = data.get("admission")
result = data.get("result")
admission = admission if isinstance(admission, Mapping) else {}
result = result if isinstance(result, Mapping) else {}
defects = admission.get("defects")
reasons = admission.get("coverage_reasons")
reasons = [str(r) for r in reasons] if isinstance(reasons, list) else []
advisory = []
dispositions = admission.get("dispositions")
for item in dispositions if isinstance(dispositions, list) else []:
try:
delta = item["candidate"]["delta"]
op = delta["head_operation"] or {}
if (
not item["admitted"]
and delta["novelty"] in ("new", "worsened")
and set(item["reasons"]) <= CAPABILITY_VETOES
and op
and delta["witness"]
):
advisory.append({"family": delta["family"], "path": op["path"], "line": op["line"], "novelty": delta["novelty"]})
except (KeyError, TypeError):
continue
unique = list({(a["family"], a["path"], a["line"]): a for a in advisory}.values())
return {
"alerts": len(defects) if isinstance(defects, list) else 0,
"advisory": unique,
"coverage": result.get("coverage_status", "unknown"),
"finished": "deadline_exhausted" not in reasons,
"engine_ran": not any(r in ("cpg_build_failed", "cpg_unavailable") for r in reasons),
}
def previous(artifact: Path, repository: Path) -> list[dict[str, object]]:
"""Finished background results not yet shown for this repository; each is shown exactly once."""
directory = results_dir(artifact, repository)
shown: list[dict[str, object]] = []
try:
candidates = sorted(directory.glob("*.json"), key=lambda p: p.stat().st_mtime)
except OSError:
return shown
for path in candidates:
if path.name == _LOCK or path.with_suffix(".shown").exists():
continue
try:
data = json.loads(path.read_text())
except (OSError, ValueError):
continue # still being written, or unreadable: leave it for the next run
if not isinstance(data, dict):
continue
shown.append({"head": path.stem, "artifact": str(path), **_summary(data)})
with contextlib.suppress(OSError):
path.with_suffix(".shown").touch()
return shown
src/openultrasast/push/explain.py
+178
View File
@@ -0,0 +1,178 @@
"""Plain-language lines for pre-push coverage reasons (Requirement 9.2).
The artifact keeps every raw reason exactly as recorded. The terminal gets one line per distinct
kind of skip, in words a first-time user can act on: hex-encoded repository paths are decoded,
per-question sha256 tokens are dropped and counted, and a reason this table does not know is
still printed rather than hidden.
"""
from __future__ import annotations
import os
import re
from collections.abc import Sequence
_HEX_PATH = re.compile(r"^((?:[0-9a-f]{2})+):(.+)$")
_TOKEN = re.compile(r"^(.+?):(?:[a-z][a-z0-9-]*:)?[0-9a-f]{64}$")
# Reasons that summarise others; printing them would only repeat a line already shown.
_SUMMARY = frozenset({"analysis_incomplete"})
_ENGINE_MISSING = (
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, "
"or run the hook through ops/ousast-docker)"
)
_NOVELTY = "whether this push introduced some engine findings could not be established ({detail})"
GLOSSARY: dict[str, str] = {
"cpg_build_failed": _ENGINE_MISSING,
"cpg_unavailable": _ENGINE_MISSING,
"cpg_empty": "engine read no code: the code graph came back empty",
"frontend_unsupported": "engine has no frontend for at least one language in this push",
"deadline_exhausted": (
"engine did not finish within the {deadline} s deadline; rerun with a longer --deadline, "
"or set OUSAST_PUSH_ENGINE=background to finish it after the push"
),
"change_context_deadline_exhausted": "the deadline ran out while relating the change to the code",
"report_deadline_exhausted": "the deadline ran out while saving the details",
"base_comparison_unavailable": "no base revision was analysed, so no engine finding can be called new or worsened",
"base_counterpart_unresolved": "the changed code has no counterpart in the base revision to compare against",
"base_unavailable": "the base revision is unavailable",
"declaration_paths_unavailable": "project declaration files could not be listed",
"ambiguous_line_correspondence": "changed lines could not be matched one-to-one to base lines in {paths}",
"line_correspondence_unavailable": "changed lines could not be mapped to the base in {paths}",
"ambiguous_path_rename_correspondence": "a renamed file could not be matched to one base file: {paths}",
"unmatched_added_deleted_path_correspondence": (
"files were both added and deleted, so a moved file cannot be told apart from a new one"
),
"capability_unavailable": ("engine findings are not qualified as alerts yet: no capability has passed independent evaluation"),
"capability_disabled": "an engine capability is declared but disabled",
"capability_unevaluated": "an engine capability has not passed its evaluation",
"capability_ambiguous": "more than one capability declaration matches a finding",
"eligibility": "the capability registry does not match this build ({detail}), so no capability can be used",
"comparison_unknown": _NOVELTY,
"dynamic_external_or_depth_context_unresolved": ("{n} engine check(s) depend on dynamic or external code the engine cannot follow"),
"query_failed": "an engine query failed, so some regions were not decided",
"query_budget_reserved": "some engine questions were not asked, to keep time for the ones that were",
"query_too_expensive": "some engine questions were too expensive to answer",
"regions_truncated": "only the highest-ranked regions were examined (--max-regions)",
"files_unparsed": "some files could not be parsed by the engine",
"engine_off": "engine checks were switched off for this run (--engine off)",
"engine_background": "engine checks run in the background after this push; the next push shows their result",
"cross_partition_semantics_unresolved": "flows between languages in this repository are not followed",
"resolution": "the pushed ref could not be resolved ({detail})",
"push_input_failed": "the push input could not be read ({detail})",
"snapshot": "some files could not be materialised for analysis ({detail})",
"dependency_unresolved": "{n} engine finding(s) depend on code the comparison could not resolve",
"change_unsupported": "{n} engine finding(s) have no supported relationship to the change",
}
# Comparison outcomes recorded per finding; each means novelty was not established.
_NOVELTY_REASONS = frozenset(
{
"base_incomplete",
"head_context_incomplete",
"witness_identity_unresolved",
"semantics_mismatch",
"change_context_unresolved",
"operation_correspondence_unresolved",
"base_reachability_bounded",
"base_operation_inventory_unavailable",
"base_enumeration_scope_reachability_derived",
"operation_present_in_base_without_traced_flow",
"operation_moved_within_change",
"source_correspondence_unresolved",
}
)
_RESOLUTION = {
"missing_base": (
"new branch: the remote has no base to compare against, so the engine comparison did not run "
"(OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)"
),
"unsupported_base": "the remote's current commit is not available locally, so there is no base to compare against",
"no_merge_base": "the pushed commit shares no history with the remote branch",
"ambiguous_base": "more than one base could apply to the pushed ref",
"missing_target": "the pushed commit could not be found locally",
"unsupported_target": "the pushed object is not a commit",
"resolution_error": "the pushed ref could not be resolved",
}
def _words(value: str) -> str:
return value.replace("_", " ").replace(":", ": ")
def _decode(path_hex: str) -> str | None:
try:
text = os.fsdecode(bytes.fromhex(path_hex))
except ValueError:
return None
return text if text and text.isprintable() else None
def _split(reason: str) -> tuple[str, str, str | None]:
"""(key, detail, decoded path) for one raw reason."""
path = None
match = _HEX_PATH.match(reason)
if match and (decoded := _decode(match.group(1))) is not None:
path, reason = decoded, match.group(2)
token = _TOKEN.match(reason)
if token:
reason = token.group(1)
if reason in GLOSSARY or reason in _NOVELTY_REASONS or reason in _SUMMARY:
return reason, "", path
head, _, rest = reason.partition(":")
if head not in GLOSSARY and head.endswith("_failed") and rest:
return "stage_failed", f"{head[: -len('_failed')]}:{rest}", path
return head, rest, path
def language_reason(language: str, changed: int, total: int) -> str:
"""The raw reason recorded when a language has no engine frontend and no quick rules."""
return f"language_not_covered:{language}:{changed}:{total}"
def explain(reasons: Sequence[str], *, deadline: float | None = None) -> list[str]:
"""One plain line per distinct kind of skipped or incomplete check, in first-seen order."""
groups: dict[tuple[str, str], list[str | None]] = {}
for reason in reasons:
key, detail, path = _split(str(reason))
if key in _SUMMARY:
continue
if key in _NOVELTY_REASONS:
key, detail = "comparison_unknown", _words(key)
groups.setdefault((key, detail), []).append(path)
novelty_details = [detail for (key, detail) in groups if key == "comparison_unknown" and detail]
lines: list[str] = []
for (key, detail), paths in groups.items():
if key == "comparison_unknown":
if detail or not novelty_details:
lines.append(_NOVELTY.format(detail=detail or "see the details file"))
continue
named = sorted({p for p in paths if p})
shown = ", ".join(named[:3]) + (f" and {len(named) - 3} more" if len(named) > 3 else "")
if key == "language_not_covered":
language, _, counts = detail.partition(":")
changed, _, total = counts.partition(":")
lines.append(
f"language not covered: {language} ({changed} changed file(s), {total} in the repository): "
"no engine frontend and no quick rules, so silence here is not a clean result"
)
elif key == "resolution":
lines.append(_RESOLUTION.get(detail, GLOSSARY["resolution"].format(detail=_words(detail))))
elif key == "stage_failed":
stage, _, error = detail.partition(":")
lines.append(f"the {_words(stage)} step failed ({error})")
elif key in GLOSSARY:
text = GLOSSARY[key].format(
n=len(paths),
paths=shown or "some files",
detail=_words(detail),
deadline=f"{deadline:g}" if deadline is not None else "configured",
)
lines.append(text)
else:
suffix = f" in {shown}" if shown else ""
lines.append(f"{_words(key if not detail else key + ':' + detail)}{suffix} (see the details file)")
return lines
src/openultrasast/push/hook/ousast-docker
Executable
+61
View File
@@ -0,0 +1,61 @@
#!/bin/sh
# OpenUltraSAST in Docker, callable from a Git hook. A script, not a shell function: Git runs
# hooks non-interactively and never sees functions from ~/.bashrc.
#
# OUSAST_COMMAND=/path/to/ousast-docker git push # or: ousast pre-push install --docker
#
# The repository is mounted READ-ONLY at its own absolute path and the artifact directory
# read-write at its own path, so every path in the arguments and in the output ("Details: ...")
# means the same thing inside and outside the container. The push input on stdin is passed
# through (-i). The container runs as your user, with no network and --memory 3g.
#
# Advisory like the hook: when Docker or the image is missing, it says the check did not run
# and exits 0. Pull the image once first; a hook never downloads ~3 GB on its own:
#
# docker pull ghcr.io/norandom/openultrasast:2.0.1
#
# --engine background does not work through this wrapper: the container ends with the hook.
image=${OUSAST_IMAGE:-ghcr.io/norandom/openultrasast:2.0.1}
if ! command -v docker >/dev/null 2>&1; then
echo "ousast-docker: check did not run: docker is not installed. Push continues."
exit 0
fi
if ! docker image inspect "$image" >/dev/null 2>&1; then
echo "ousast-docker: check did not run: image $image is not present." \
"Run 'docker pull $image' once. Push continues."
exit 0
fi
repo=$(pwd -P)
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
previous=
for argument in "$@"; do
if [ "$previous" = "--artifact" ]; then
artifact_dir=$(dirname -- "$argument")
fi
previous=$argument
done
mkdir -p -- "$artifact_dir" || exit 0
artifact_dir=$(cd -- "$artifact_dir" && pwd -P)
# A linked worktree keeps its objects in the main repository's git directory: mount that too.
common=$(git rev-parse --path-format=absolute --git-common-dir 2>/dev/null || echo "$repo/.git")
case "$common" in
"$repo"/*) extra_mount= ;;
*) extra_mount="-v $common:$common:ro" ;;
esac
# shellcheck disable=SC2086 # OUSAST_DOCKER_ARGS and extra_mount are deliberately word-split
docker run --rm -i --network none --memory "${OUSAST_DOCKER_MEMORY:-3g}" \
--user "$(id -u):$(id -g)" -e HOME=/tmp -e TMPDIR=/work --tmpfs /work:size=4g \
-v "$repo:$repo:ro" -w "$repo" -v "$artifact_dir:$artifact_dir" $extra_mount \
${OUSAST_DOCKER_ARGS:-} "$image" "$@"
status=$?
if [ "$status" -ge 125 ]; then
echo "ousast-docker: check did not run: the container could not start (docker exit $status). Push continues."
exit 0
fi
# Only an explicit blocking opt-in may turn the container's exit into a rejected push; an older
# image that rejects a newer argument must not block either.
if [ "$status" -ne 0 ] && [ "${OUSAST_PUSH_MODE:-advisory}" != blocking ]; then
echo "ousast-docker: the check exited with status $status; push continues (advisory)."
exit 0
fi
exit "$status"
src/openultrasast/push/hook/pre-push
Executable
+38
View File
@@ -0,0 +1,38 @@
#!/bin/sh
# OpenUltraSAST pre-push safety net. Advisory: nothing here rejects a push unless
# OUSAST_PUSH_MODE=blocking is set explicitly. Install with `ousast pre-push install`
# (or ops/install-pre-push from a checkout); remove with `ousast pre-push uninstall`.
# A prior hook is chained only after it was moved to the sibling path "$0.before-ousast".
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
prior="$0.before-ousast"
# The Docker wrapper installed beside this hook (`ousast pre-push install --docker`) is used
# unless OUSAST_COMMAND names another command; otherwise `ousast` from PATH.
if [ -z "${OUSAST_COMMAND:-}" ] && [ -x "$(dirname -- "$0")/ousast-docker" ]; then
OUSAST_COMMAND="$(dirname -- "$0")/ousast-docker"
fi
command=${OUSAST_COMMAND:-ousast}
if ! command -v "$command" >/dev/null 2>&1; then
echo "ousast: pre-push check did not run: '$command' is not on PATH." \
"Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues."
if [ -e "$prior" ] || [ -L "$prior" ]; then
exec "$prior" "$@"
fi
exit 0
fi
remote_name=$1
remote_url=$2
set -- pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}"
# Optional settings are passed only when set, so an older ousast (or image) still runs.
if [ -n "${OUSAST_PUSH_ENGINE:-}" ]; then
set -- "$@" --engine "$OUSAST_PUSH_ENGINE"
fi
if [ -n "${OUSAST_COMPARISON_BASE:-}" ]; then
set -- "$@" --comparison-base "$OUSAST_COMPARISON_BASE"
fi
if [ -e "$prior" ] || [ -L "$prior" ]; then
set -- "$@" --prior-hook "$prior"
fi
exec "$command" "$@" --remote "$remote_name" "$remote_url"
src/openultrasast/push/install.py
+117
View File
@@ -0,0 +1,117 @@
"""`ousast pre-push install|uninstall`: the hook as package data (task 17.6, Requirements 7.4, 9.6).
Works from a pip install: the hook script and the Docker wrapper ship inside the wheel. Git's
effective hook directory is resolved (including `core.hooksPath`) and never edited. An existing
hook is never overwritten silently: without `--force` installation refuses; with it the existing
hook moves to `pre-push.before-ousast` and is chained with the same input and arguments.
Uninstall removes only a hook this tool wrote and restores the chained hook.
"""
from __future__ import annotations
import os
import subprocess
from importlib import resources
from pathlib import Path
PRIOR_SUFFIX = ".before-ousast"
# First lines this tool has written into an installed hook: the current one and the 2026-09 one.
_MARKERS = (b"# OpenUltraSAST pre-push safety net.", b"# Explicitly copy using the non-overwriting recipe in ops/README.md.")
def packaged(name: str) -> bytes:
"""The shipped `pre-push` hook or `ousast-docker` wrapper."""
return resources.files("openultrasast.push").joinpath("hook", name).read_bytes()
def hooks_dir(repository: Path) -> Path:
out = subprocess.run(
["git", "-C", str(repository), "rev-parse", "--path-format=absolute", "--git-path", "hooks"],
capture_output=True,
check=False,
timeout=30,
)
if out.returncode != 0:
raise ValueError(f"{repository} is not a Git repository")
return Path(os.fsdecode(out.stdout.rstrip(b"\n")))
def ours(path: Path) -> bool:
try:
head = path.read_bytes()[:400]
except OSError:
return False
return any(marker in head for marker in _MARKERS)
def _write(path: Path, data: bytes) -> None:
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o755)
with os.fdopen(fd, "wb") as stream:
stream.write(data)
os.fchmod(stream.fileno(), 0o755)
def _exists(path: Path) -> bool:
return path.exists() or path.is_symlink()
def install(repository: Path, *, force: bool = False, docker: bool = False) -> tuple[int, list[str]]:
hooks = hooks_dir(repository)
hooks.mkdir(parents=True, exist_ok=True)
hook = hooks / "pre-push"
prior = hooks / ("pre-push" + PRIOR_SUFFIX)
wrapper = hooks / "ousast-docker"
lines: list[str] = []
if _exists(hook):
if not force:
what = "an OpenUltraSAST hook is already installed" if ours(hook) else "an existing pre-push hook"
return 1, [
f"Installation refused: {what} at {hook}.",
"Rerun with --force to replace ours, or to move a foreign hook to pre-push.before-ousast and chain it.",
]
if ours(hook):
hook.unlink()
lines.append(f"Replaced the OpenUltraSAST hook at {hook}.")
elif _exists(prior):
return 1, [f"Installation refused: {prior} already holds a chained hook; resolve it by hand first."]
else:
os.rename(hook, prior)
lines.append(f"Moved the existing hook to {prior}; it runs after the check with the same input.")
if docker:
if _exists(wrapper) and not (force and ours_wrapper(wrapper)):
return 1, [f"Installation refused: {wrapper} exists; rerun with --force to replace it."]
if _exists(wrapper):
wrapper.unlink()
_write(wrapper, packaged("ousast-docker"))
lines.append(f"Installed the Docker wrapper {wrapper} (image: ghcr.io/norandom/openultrasast:2.0.1; pull it once).")
_write(hook, packaged("pre-push"))
lines.insert(0, f"Installed {hook}")
lines.append("Advisory: the hook never blocks a push unless OUSAST_PUSH_MODE=blocking. Remove with 'ousast pre-push uninstall'.")
return 0, lines
def ours_wrapper(path: Path) -> bool:
try:
return b"OpenUltraSAST in Docker, callable from a Git hook" in path.read_bytes()[:400]
except OSError:
return False
def uninstall(repository: Path, *, force: bool = False) -> tuple[int, list[str]]:
hooks = hooks_dir(repository)
hook = hooks / "pre-push"
prior = hooks / ("pre-push" + PRIOR_SUFFIX)
wrapper = hooks / "ousast-docker"
if not _exists(hook):
return 1, [f"Nothing to remove: no pre-push hook at {hook}."]
if not ours(hook) and not force:
return 1, [f"Removal refused: {hook} was not written by OpenUltraSAST. Inspect it, or rerun with --force."]
hook.unlink()
lines = [f"Removed {hook}"]
if _exists(wrapper) and ours_wrapper(wrapper):
wrapper.unlink()
lines.append(f"Removed {wrapper}")
if _exists(prior):
os.rename(prior, hook)
lines.append(f"Restored the previous hook from {prior}.")
return 0, lines
src/openultrasast/push/policy.py
+11
View File
@@ -658,6 +658,16 @@ def _compare_evidence(
reason = "head_context_incomplete"
elif execution_budget is not None and time.monotonic() >= execution_budget.deadline_monotonic:
reason = "deadline_exhausted"
elif op.path in {context.decode_path(p) for p in context.added_paths}:
# The file did not exist in the base, so no base line or counterpart question can hold
# this operation (task 17.4, Requirement 9.4). Code moved here from elsewhere is still
# movement: an identical operation in any base answer keeps it out of `new`, and an
# unpaired add/delete in the same push already blocked above as context.
reason = "operation_moved_within_change"
if not any(b.operation == op.operation and b.mechanism == op.mechanism for b in base_ops):
reason = "file_added_unsupported_claim"
if change and finding.rung == Rung.ENTAILED and not op.discharged:
novelty, reason = "new", "file_added_in_head"
elif enumerated:
same = tuple(b for b in old if b.source == op.source)
reason = "operation_correspondence_unresolved"
@@ -926,6 +936,7 @@ def admit_candidates(candidates: Sequence[AdmissionCandidate], *, capabilities:
elif not delta.change_evidence or (delta.novelty, delta.reason) not in (
("new", "source_connection_absent_from_comparable_base"),
("new", "operation_absent_from_comparable_base"),
("new", "file_added_in_head"),
("worsened", "discharge_removed"),
):
reasons.append("change_unsupported")
src/openultrasast/push/quick.py
+130
View File
@@ -0,0 +1,130 @@
"""The quick-rule tier of pre-push: the fast first look (task 17.3, Requirement 9.3).
The shipped pattern rules run over the head side of the files a push changes, before the engine and
inside the same deadline, and keep the matches that fall on a changed line or inside a changed
function. They are pattern matches: shown as advisory, labeled apart from admitted alerts, never
admitted and never blocking. Requirement 3.2 still governs every engine candidate.
"""
from __future__ import annotations
import time
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
from typing import Literal
from openultrasast.model.contracts import ChangeContext
Basis = Literal["comparison", "last_commit"]
@dataclass(frozen=True)
class QuickTierResult:
status: Literal["completed", "not_run", "failed"]
basis: Basis
files: tuple[str, ...] = ()
bytes_read: int = 0
findings: tuple[Mapping[str, object], ...] = ()
reason: str | None = None
seconds: float = 0.0
basis_commit: str | None = None
def to_payload(self) -> dict[str, object]:
return {
"label": "quick rules: pattern matches on changed lines; advisory, not admitted, never blocking",
"status": self.status,
"basis": self.basis,
"basis_commit": self.basis_commit,
"files": list(self.files),
"bytes_read": self.bytes_read,
"findings": [dict(item) for item in self.findings],
"reason": self.reason,
"seconds": self.seconds,
}
def head_spans(context: ChangeContext) -> dict[str, list[tuple[int, int]]]:
"""Changed head lines per decoded path, for every changed file that still exists in head."""
deleted = set(context.deleted_paths)
spans: dict[str, list[tuple[int, int]]] = {}
for path in context.changed_paths:
if path not in deleted:
spans.setdefault(context.decode_path(path), [])
for span in context.spans:
if span.side == "head":
spans.setdefault(context.decode_path(span.path), []).append((span.start_line, span.end_line))
return spans
def run_quick_tier(
root: Path, spans: Mapping[str, Sequence[tuple[int, int]]], *, basis: Basis = "comparison", basis_commit: str | None = None
) -> QuickTierResult:
"""Pattern rules over the changed head files of a materialized snapshot at `root`.
Pure and bounded by its caller: it reads only the named files, runs no project code and calls no
model. `files`/`bytes_read` prove what it read, so a zero is never an unread input.
"""
from openultrasast.findings import PATTERN_RULES, quick_scan_findings
from openultrasast.mapping import analyze_entry_points, attach_reachability_hints
from openultrasast.policy import load_policy
from openultrasast.preprocess import build_file_target, detect_language
from openultrasast.rank import rank_targets
started = time.monotonic()
resolved = root.resolve()
paths = []
read = 0
for name in sorted(spans):
path = resolved / name
if path.is_symlink() or not path.is_file() or not path.resolve().is_relative_to(resolved):
continue
if detect_language(path) == "unknown":
continue
paths.append(path)
read += path.stat().st_size
targets = [build_file_target(resolved, path) for path in paths]
targets = attach_reachability_hints(targets, analyze_entry_points(resolved, targets))
rules = {rule.rule_id: rule for rule in PATTERN_RULES}
kept: list[Mapping[str, object]] = []
for finding in quick_scan_findings(resolved, targets, rank_targets(targets), PATTERN_RULES, load_policy()):
if finding.status == "shadow" or finding.line is None:
continue
changed = spans.get(finding.path, ())
where = _changed_scope(finding.line, changed, finding.reachability_evidence)
if where is None:
continue
rule_id = finding.finding_id.split(":", 1)[0]
rule = rules.get(rule_id)
kept.append(
{
"rule": rule_id,
"title": finding.title,
"cwe": rule.cwe if rule is not None else None,
"severity": finding.severity,
"path": finding.path,
"line": finding.line,
"function": finding.function_name,
"scope": where,
}
)
return QuickTierResult(
"completed",
basis,
tuple(target.path for target in targets),
read,
tuple(kept),
seconds=time.monotonic() - started,
basis_commit=basis_commit,
)
def _changed_scope(line: int, changed: Sequence[tuple[int, int]], hints: Sequence[Mapping[str, object]]) -> str | None:
if any(start <= line <= end for start, end in changed):
return "changed_line"
for hint in hints:
start, end = hint.get("line"), hint.get("end_line")
inside = isinstance(start, int) and isinstance(end, int) and start <= line <= end
if inside and any(a <= end and start <= b for a, b in changed): # type: ignore[operator]
return "changed_function"
return None
src/openultrasast/push/report.py
+137
View File
@@ -18,6 +18,7 @@ from typing import Literal
from openultrasast.model.contracts import ChangeContext, ExecutionBudget
from openultrasast.model.scan import ModelScanResult
from openultrasast.push.contracts import PushComparison, PushResolution, PushResult, SnapshotManifest
from openultrasast.push.explain import explain
from openultrasast.push.policy import AdmissionResult, _context_boundaries
from openultrasast.redaction import redact_secrets
@@ -44,6 +45,10 @@ class PushReport:
# Labeled M1a recorded-veto evaluation. Present only under the explicit replay flag;
# it is retained beside the decision and never feeds admission or enforcement.
experimental: Mapping[str, object] = field(default_factory=dict)
# The quick-rule tier (task 17.3): one payload per comparison it ran for; advisory only.
quick: tuple[Mapping[str, object], ...] = ()
# The background engine (task 17.5): what this run started, and earlier results shown now.
background: Mapping[str, object] = field(default_factory=dict)
def __post_init__(self) -> None:
object.__setattr__(self, "model_assistance", MappingProxyType(dict(self.model_assistance)))
@@ -133,6 +138,8 @@ def _write_artifact(report: PushReport, target: Path, temporary: Path, connectio
"timing_scope": "Caller stage measurements; publication and rendering duration is returned in ReportDelivery.",
"resolution": report.resolution.to_payload() if report.resolution else None,
"experimental": dict(report.experimental),
"quick_tier": [dict(item) for item in report.quick],
"engine_background": dict(report.background),
}
json.dump(payload, stream, ensure_ascii=True, allow_nan=False, indent=2)
stream.write("\n")
@@ -187,6 +194,10 @@ def _save(report: PushReport, target: Path, budget: ExecutionBudget) -> str | No
process.close()
# Distinct skip kinds shown in the terminal; the artifact keeps every reason.
_SKIP_LINES = 12
def _short(value: str, limit: int = 180) -> str:
# Bound before escaping: repository-controlled names/witnesses cannot add lines
# or terminal escapes. Full spelling and every witness remain in the artifact.
@@ -211,6 +222,121 @@ def _experimental_line(experimental: Mapping[str, object]) -> str:
)
# Vetoes that say only "this capability is not qualified yet". A candidate whose novelty is established
# and whose only vetoes are these is shown as an advisory engine finding (task 17.4, Requirement 9.4);
# any comparison, evidence or location veto keeps it out of the terminal.
CAPABILITY_VETOES = frozenset(
{
"capability_unavailable",
"capability_disabled",
"capability_unevaluated",
"capability_ambiguous",
"operation_semantics_mismatch",
"consequence_unsupported",
"repair_unsupported",
}
)
def _advisory_engine_lines(report: PushReport) -> list[str]:
shown: dict[str, list[str]] = {}
for disposition in report.admission.dispositions:
delta = disposition.candidate.delta
op = delta.head_operation
if (
disposition.admitted
or delta.novelty not in ("new", "worsened")
or not set(disposition.reasons) <= CAPABILITY_VETOES
or op is None
or delta.witness is None
or delta.defect_id in shown
):
continue
how = "new" if delta.novelty == "new" else "worsened"
shown[delta.defect_id] = [
f"- {_short(delta.family)} at {_short(op.path, 4096)}:{op.line} ({how}): {_short(redact_secrets(delta.witness))}"
]
if not shown:
return []
lines = [f"Engine: {len(shown)} finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert."]
for entry in list(shown.values())[:3]:
lines.extend(entry)
if len(shown) > 3:
lines.append(f"{len(shown) - 3} more advisory engine findings retained in the detailed result.")
return lines
def _previous_engine_lines(background: Mapping[str, object]) -> list[str]:
"""Background engine results from an earlier push, shown once (task 17.5)."""
lines: list[str] = []
earlier = background.get("previous")
for item in earlier if isinstance(earlier, list) else []:
if not isinstance(item, Mapping):
continue
advisory = item.get("advisory")
found = [a for a in advisory if isinstance(a, Mapping)] if isinstance(advisory, list) else []
state = (
"the engine did not run (Joern unavailable)"
if not item.get("engine_ran")
else f"{item.get('alerts', 0)} alert(s), {len(found)} advisory engine finding(s), coverage {item.get('coverage')}"
+ ("" if item.get("finished") else ", did not finish within its deadline")
)
head = str(item.get("head", ""))[:12]
lines.append(f"Engine result for {_short(head)} from the previous push: {_short(state, 300)}.")
for entry in found[:3]:
lines.append(f"- {_short(str(entry.get('family')))} at {_short(str(entry.get('path')), 4096)}:{entry.get('line')}")
lines.append(" Details: " + _short(str(item.get("artifact", "")), 4096))
return lines
def _background_lines(background: Mapping[str, object]) -> list[str]:
status = background.get("status")
head = _short(str(background.get("head", ""))[:12])
if status == "started":
return [
f"Engine: running in the background for {head} (deadline {background.get('deadline_seconds')} s); "
"the next push shows its result."
]
if status == "busy":
return [f"Engine: a background engine run for {head} is still busy, so none was started for this push."]
if status == "failed":
return [f"Engine: the background run could not be started ({_short(str(background.get('reason')))})."]
if status == "not_started":
return [f"Engine: no background run was started ({_short(str(background.get('reason')))})."]
return []
# Quick-rule matches shown per comparison; the artifact keeps every one.
_QUICK_LINES = 5
def _quick_lines(payload: Mapping[str, object]) -> list[str]:
"""The quick tier in the terminal: advisory pattern matches, never presented as alerts."""
files = payload.get("files")
count = len(files) if isinstance(files, list) else 0
read = f"{count} file(s), {payload.get('bytes_read', 0)} bytes read"
lines: list[str] = []
if payload.get("basis") == "last_commit":
commit = str(payload.get("basis_commit") or "")[:7]
lines.append(f"New branch: instead, quick rules checked the files its last commit changed (compared with {commit}).")
if payload.get("status") != "completed":
return [*lines, f"Quick rules did not run ({_short(str(payload.get('reason') or 'unknown'))})."]
findings = payload.get("findings")
found = [item for item in findings if isinstance(item, Mapping)] if isinstance(findings, list) else []
if not found:
return [*lines, f"Quick rules: no pattern match on changed code ({read})."]
lines.append(f"Quick rules: {len(found)} pattern match(es) on changed code ({read}). Advisory: not verified by the engine.")
for item in found[:_QUICK_LINES]:
cwe = f"{item.get('cwe')}, " if item.get("cwe") else ""
lines.append(
f"- {_short(str(item.get('rule')))} ({cwe}{_short(str(item.get('severity')))}) at "
f"{_short(str(item.get('path')), 4096)}:{item.get('line')}: {_short(str(item.get('title')))}"
)
if len(found) > _QUICK_LINES:
lines.append(f"{len(found) - _QUICK_LINES} more quick-rule matches retained in the detailed result.")
return lines
def render_report(report: PushReport, *, artifact: Path | None, error: str | None = None) -> str:
lines: list[str] = []
defects = report.admission.defects
@@ -246,6 +372,10 @@ def render_report(report: PushReport, *, artifact: Path | None, error: str | Non
)
if len(defects) > 3:
lines.append(f"{len(defects) - 3} more actionable defects retained in the detailed result.")
lines.extend(_previous_engine_lines(report.background))
lines.extend(_advisory_engine_lines(report))
for payload in report.quick:
lines.extend(_quick_lines(payload))
if not defects:
coverage = report.result.coverage_status
if coverage == "complete_within_scope":
@@ -260,8 +390,14 @@ def render_report(report: PushReport, *, artifact: Path | None, error: str | Non
notices = []
if report.result.coverage_status in ("incomplete", "unavailable"):
reasons = report.admission.coverage_reasons
# One plain line per distinct skipped check (Requirement 9.2); the artifact keeps the raw values.
deadline = report.timings.get("deadline_seconds")
for skipped in explain(reasons, deadline=float(deadline) if deadline is not None else None)[:_SKIP_LINES]:
lines.append("Skipped: " + _short(skipped, 400))
if any("deadline" in reason for reason in reasons):
notices.append("Analysis did not finish. Rerun with a longer analysis deadline.")
elif any(reason in ("cpg_build_failed", "cpg_unavailable") for reason in reasons):
notices.append("Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.")
elif any("capability" in reason for reason in reasons):
notices.append("Some checks are not enabled for normal alerts. Review capability coverage in the details.")
else:
@@ -272,6 +408,7 @@ def render_report(report: PushReport, *, artifact: Path | None, error: str | Non
notices.append("Details could not be saved. Choose an artifact path outside the analyzed repository.")
elif error:
notices.append("Details could not be saved. Retry with a writable artifact path and available reporting time.")
lines.extend(_background_lines(report.background))
if notices:
lines.append("Notice: " + " ".join(notices))
if report.experimental:
src/openultrasast/push/runner.py
+188 -59
View File
@@ -9,6 +9,7 @@ import os
import pickle
import selectors
import time
from collections import Counter
from collections.abc import Callable, Sequence
from contextlib import nullcontext
from dataclasses import asdict, replace
@@ -28,6 +29,7 @@ from openultrasast.preprocess import build_file_target, enumerate_source_files
from openultrasast.push.cache import ArtifactCache, SemanticKeys
from openultrasast.push.contracts import ComparisonAnalysis, PushComparison, PushResult, SnapshotManifest
from openultrasast.push.eligibility import fingerprint, load_registry
from openultrasast.push.explain import language_reason
from openultrasast.push.policy import (
ActionableDefect,
AdmissionCandidate,
@@ -40,6 +42,7 @@ from openultrasast.push.policy import (
dependency_gaps,
semantics_digest,
)
from openultrasast.push.quick import Basis, QuickTierResult, head_spans, run_quick_tier
from openultrasast.push.report import PushReport, ReportDelivery, ScanRecord, _completed_scan, deliver_report, render_report
from openultrasast.push.reuse import ReusingBackend, declaration_identity, discovery
from openultrasast.push.snapshot import SnapshotAdapter
@@ -206,6 +209,30 @@ def _record(
return report.to_payload()
def _uncovered_languages(manifest: SnapshotManifest, context: ChangeContext) -> list[str]:
"""`language_not_covered` reasons for changed languages with no engine frontend and no quick rules.
Only a language this push changes is named: a Go helper nobody touched is not this push's gap.
"""
from openultrasast.findings import PATTERN_RULES
from openultrasast.model.partitions import FRONTENDS
from openultrasast.preprocess import LANGUAGE_BY_EXTENSION
covered = set(FRONTENDS) | {language for rule in PATTERN_RULES if rule.status != "disabled" for language in rule.languages}
if any(not rule.languages for rule in PATTERN_RULES if rule.status != "disabled"):
return []
def language(path: str) -> str | None:
return LANGUAGE_BY_EXTENSION.get(os.path.splitext(path)[1].lower())
totals = Counter(language(os.fsdecode(bytes.fromhex(item.path_hex))) for item in manifest.files)
changed = Counter(language(context.decode_path(path)) for path in context.changed_paths if path not in context.deleted_paths)
return [
language_reason(name, count, totals.get(name, 0))
for name, count in sorted((k, v) for k, v in changed.items() if k is not None and k not in covered)
]
def _discovery_identity() -> str:
root = Path(__file__).resolve().parents[1]
# Discovery invokes the shipped mapper, preprocessing, layout and family specs.
@@ -241,6 +268,8 @@ def _analyze(
resolved: PushComparison | None = None,
record_vetoes: bool = False,
declarations: Path | None = None,
engine: Literal["inline", "off", "background"] = "inline",
quick_basis: Basis = "comparison",
) -> PushReport:
"""Run a single explicit comparison; exit policy never stands for coverage.
@@ -269,6 +298,7 @@ def _analyze(
comparison = resolved
context = None
experimental: dict[str, object] = {"label": LABEL, "flag": FLAG, "status": "not_evaluated"} if record_vetoes else {}
quick: QuickTierResult | None = None
timings: dict[str, float] = {
"deadline_seconds": config.deadline_seconds,
"cancellation_allowance_seconds": config.cancellation_allowance_seconds,
@@ -347,69 +377,84 @@ def _analyze(
context = adapter.compare(comparison, declaration_paths=tuple(set(head_declarations + base_declarations)), budget=budget)
context = replace(context, unresolved_boundaries=tuple(dict.fromkeys((*context.unresolved_boundaries, *reasons))))
reasons.extend(context.unresolved_boundaries)
reasons.extend(_uncovered_languages(tip.manifest, context))
stage_started = _stage(timings, stage, stage_started)
stage = "head"
head_scan = scan_repository(
tip.root,
head_regions,
backend=head_backend,
budget=scan_budget,
execution_budget=budget,
ranking_mode="evidence",
unit="repository",
population_complete=all(m.complete for m in manifests),
change_context=context,
)
records.append(ScanRecord(comparison, "head", head_scan))
stage = "quick"
# The fast first tier: pattern rules over the changed head lines, before the engine
# and inside the same deadline (Requirement 9.3). Advisory, never admitted.
try:
spans = head_spans(context)
basis_commit = comparison.base_oid if quick_basis == "last_commit" else None
quick = _prepare(lambda: run_quick_tier(tip.root, spans, basis=quick_basis, basis_commit=basis_commit), budget)
except Exception as error: # noqa: BLE001 -- a failed quick tier is a stated gap, never the end of the check
quick = QuickTierResult("failed", quick_basis, reason=type(error).__name__)
reasons.append("quick_rules_failed:" + type(error).__name__)
stage_started = _stage(timings, stage, stage_started)
stage = "base"
delta = compare_targeted_base(
old.root,
head=head_scan,
head_regions=head_regions,
base_regions=base_regions,
context=context,
backend=base_backend,
execution_budget=budget,
scan_budget=scan_budget,
head_semantics=provenance["semantics"],
base_semantics=provenance["semantics"],
cache=cache,
cache_semantics=cache_semantics,
)
if delta.base_scan is not None:
records.append(ScanRecord(comparison, "base", delta.base_scan))
reasons.extend(delta.coverage_reasons)
stage_started = _stage(timings, stage, stage_started)
stage = "admission"
candidates = tuple(
AdmissionCandidate(
c,
CapabilityKey(
c.head_operation.question.language if c.head_operation else "unknown",
"unspecified",
"unspecified",
c.family,
c.head_operation.mechanism if c.head_operation else "taint",
"unreviewed",
delta.semantics,
),
comparison,
dependency_gaps(c, context=context, head=head_scan, base=delta.base_scan),
if engine != "inline":
reasons.append("engine_" + engine)
if engine == "inline":
stage = "head"
head_scan = scan_repository(
tip.root,
head_regions,
backend=head_backend,
budget=scan_budget,
execution_budget=budget,
ranking_mode="evidence",
unit="repository",
population_complete=all(m.complete for m in manifests),
change_context=context,
)
for c in delta.candidates
)
admission = _prepare(lambda: _admit(candidates, provenance), budget)
stage_started = _stage(timings, stage, stage_started)
if record_vetoes:
stage, stage_started = "recorded_vetoes", time.monotonic()
assert context is not None
recorded = _prepare(
lambda: _record(head_scan, delta, context, comparison, tuple(reasons), admission, provenance, declarations),
budget,
)
experimental = {**recorded, "status": "evaluated"}
records.append(ScanRecord(comparison, "head", head_scan))
stage_started = _stage(timings, stage, stage_started)
stage = "base"
delta = compare_targeted_base(
old.root,
head=head_scan,
head_regions=head_regions,
base_regions=base_regions,
context=context,
backend=base_backend,
execution_budget=budget,
scan_budget=scan_budget,
head_semantics=provenance["semantics"],
base_semantics=provenance["semantics"],
cache=cache,
cache_semantics=cache_semantics,
)
if delta.base_scan is not None:
records.append(ScanRecord(comparison, "base", delta.base_scan))
reasons.extend(delta.coverage_reasons)
stage_started = _stage(timings, stage, stage_started)
stage = "admission"
candidates = tuple(
AdmissionCandidate(
c,
CapabilityKey(
c.head_operation.question.language if c.head_operation else "unknown",
"unspecified",
"unspecified",
c.family,
c.head_operation.mechanism if c.head_operation else "taint",
"unreviewed",
delta.semantics,
),
comparison,
dependency_gaps(c, context=context, head=head_scan, base=delta.base_scan),
)
for c in delta.candidates
)
admission = _prepare(lambda: _admit(candidates, provenance), budget)
stage_started = _stage(timings, stage, stage_started)
if record_vetoes:
stage, stage_started = "recorded_vetoes", time.monotonic()
assert context is not None
recorded = _prepare(
lambda: _record(head_scan, delta, context, comparison, tuple(reasons), admission, provenance, declarations),
budget,
)
experimental = {**recorded, "status": "evaluated"}
stage_started = _stage(timings, stage, stage_started)
stage = "cleanup"
timings[stage + "_seconds"] = time.monotonic() - stage_started
except Exception as error:
@@ -451,9 +496,67 @@ def _analyze(
snapshots=tuple(manifests),
change_context=context,
experimental=experimental,
quick=(quick.to_payload(),) if quick is not None else (),
)
def _last_commit_quick(
repository: Path, comparison: PushComparison, *, artifact: Path, config: PushConfig, budget: ExecutionBudget
) -> PushReport | None:
"""Quick rules over the files the pushed head's last commit changed; None for a root commit."""
try:
parent = SnapshotAdapter(repository, execution_budget=budget).resolve_replay(comparison.head_oid + "^", comparison.head_oid)
except Exception: # noqa: BLE001 -- a root commit has no parent to compare with
return None
resolved = PushComparison(comparison.head_oid, parent.base_oid, "new_branch_last_commit", comparison.refs)
assert resolved.base_oid is not None
return _analyze(
repository,
base=resolved.base_oid,
head=resolved.head_oid,
artifact=artifact,
config=config,
execution_budget=budget,
resolved=resolved,
engine="off",
quick_basis="last_commit",
)
def _background(
repository: Path,
pending: Sequence[tuple[str | None, str]],
artifact: Path,
deadline: float,
max_regions: int,
cache_dir: Path | None,
) -> dict[str, object]:
"""Earlier background results to show, and at most one new background engine run (task 17.5)."""
from openultrasast.push import background
state: dict[str, object] = {"previous": background.previous(artifact, repository)}
ready = [(base, head) for base, head in pending if base is not None]
if ready:
base, head = ready[0]
assert base is not None
state.update(
background.start(
repository,
base=base,
head=head,
artifact=artifact,
deadline_seconds=deadline,
max_regions=max_regions,
cache_dir=cache_dir,
)
)
if len(ready) > 1:
state["not_started"] = [h for _, h in ready[1:]] # one engine at a time
elif pending:
state.update(status="not_started", reason="no base revision to compare against")
return state
def _deliver(repository: Path, report: PushReport, artifact: Path, budget: ExecutionBudget) -> ReportDelivery:
if artifact.resolve().is_relative_to(repository.resolve()):
error = "artifact_inside_repository"
@@ -474,6 +577,8 @@ def replay(
model_config: Path | None = None,
record_vetoes: bool = False,
declarations: Path | None = None,
engine: Literal["inline", "off", "background"] = "inline",
background_deadline: float = 900.0,
) -> ReportDelivery:
config = config or PushConfig()
if declarations is not None and not record_vetoes:
@@ -491,7 +596,10 @@ def replay(
execution_budget=budget,
record_vetoes=record_vetoes,
declarations=declarations,
engine=engine,
)
pending = [(base, head)] if engine == "background" else []
report = replace(report, background=_background(repository, pending, artifact, background_deadline, max_regions, cache_dir))
from openultrasast.push.assistance import assist
report = assist(report, config=model_config, budget=budget)
@@ -511,6 +619,8 @@ def push(
cache_dir: Path | None = None,
model_config: Path | None = None,
execution_budget: ExecutionBudget | None = None,
engine: Literal["inline", "off", "background"] = "inline",
background_deadline: float = 900.0,
) -> ReportDelivery:
"""Consume one Git transaction. Resolution, every comparison and reporting share a deadline."""
config = config or PushConfig()
@@ -522,6 +632,7 @@ def push(
budget = execution_budget or ExecutionBudget(started + config.deadline_seconds, config.cancellation_allowance_seconds)
reports: list[PushReport] = []
analyses: list[ComparisonAnalysis] = []
fallbacks: list[PushReport] = []
reasons: list[str] = []
resolution = None
try:
@@ -535,6 +646,12 @@ def push(
if comparison.base_oid is None or time.monotonic() >= budget.deadline_monotonic:
analyses.append(ComparisonAnalysis(comparison, (), (), "unavailable"))
reasons.append("base_comparison_unavailable" if comparison.base_oid is None else "deadline_exhausted")
if comparison.base_oid is None:
# A new branch has no remote base. Say what was checked instead: the quick rules
# over the files its last commit changed (Requirement 9.2, 9.3).
fallback = _last_commit_quick(repository, comparison, artifact=artifact, config=config, budget=budget)
if fallback is not None:
fallbacks.append(fallback)
continue
report = _analyze(
repository,
@@ -547,6 +664,7 @@ def push(
cache_dir=cache_dir,
execution_budget=budget,
resolved=comparison,
engine=engine,
)
reports.append(report)
analyses.extend(report.result.analyses)
@@ -597,6 +715,17 @@ def push(
resolution=resolution,
snapshots=tuple(s for r in reports for s in r.snapshots),
change_contexts=tuple(r.change_context for r in reports if r.change_context is not None),
quick=tuple(q for r in (*reports, *fallbacks) for q in r.quick),
background=_background(
repository,
[(r.change_context.base_revision, r.change_context.head_revision) for r in reports if r.change_context]
if engine == "background"
else [],
artifact,
background_deadline,
max_regions,
cache_dir,
),
)
from openultrasast.push.assistance import assist
src/openultrasast/push/snapshot.py
+5
View File
@@ -249,6 +249,7 @@ class SnapshotAdapter:
boundaries = ["declaration_paths_unavailable"] if declaration_paths is None else []
changed: list[str] = []
deleted: list[str] = []
added: list[str] = []
spans: list[ChangedSpan] = []
renames: list[PathRename] = []
mappings: list[LineCorrespondence] = []
@@ -305,6 +306,9 @@ class SnapshotAdapter:
if status == "D":
assert path_base is not None
deleted.append(path_base.hex())
if status == "A":
assert path_head is not None
added.append(path_head.hex())
if status == "R":
assert path_base is not None and path_head is not None
renames.append(PathRename(path_base.hex(), path_head.hex()))
@@ -432,6 +436,7 @@ class SnapshotAdapter:
tuple(boundaries),
"filesystem-bytes-hex",
tuple(mappings),
tuple(added),
)
@contextmanager
src/openultrasast/reports.py
+26
View File
@@ -49,7 +49,10 @@ def write_markdown_report(
lines = ["# OpenUltraSAST Report", "", f"Findings: {len(findings)}", "", "## Inventory", ""]
if not findings:
uncovered = [item for item in degradations or () if item.get("reason") == "language_not_covered"]
lines.append("No quick-mode findings were emitted.")
for item in uncovered:
lines.append(f"Not a clean result for {item.get('language')} ({item.get('count')} files): no rules cover that language.")
lines.append("")
for finding in reasoned:
verification = verification_by_id.get(finding.finding_id)
@@ -446,6 +449,25 @@ _COVERAGE_DEGRADATIONS: dict[str, str] = {
"about the rest of the repository."
),
"region_failed": "The region `{path}` could not be arbitrated.",
# Recorded by cli.py and, until 2026-10-02, never rendered (pre-push 17.2, Requirement 9.2).
"cpg_unavailable": (
"The Joern engine is not installed here, so the model layer did not run and **no flow was followed**. "
"Install Joern (`joern-parse` on PATH) or run through the Docker image."
),
"learning_endpoint_unavailable": (
"No model endpoint is configured, so the graph's entailed findings are reported and the residual "
"`suspicion` questions went unasked."
),
"model_withheld_execution_budget": "The model was not asked, because the run's time budget leaves no room for it.",
"facts_unavailable": "The fact tables for the `{stage}` stage could not be loaded, so that stage decided nothing. {detail}",
"policy_invalid": "The `{stage}` policy could not be loaded, so that stage decided nothing. {detail}",
"max_findings_exceeded": "Only the first {requested} of {actual} findings are reported (`max_findings`).",
"hunter_model_unavailable": "The `{stage}` stage needs a model and none is available, so it was skipped.",
"sandbox_unavailable": "Docker is not available, so no finding was reproduced in the sandbox (REGRESS skipped).",
"language_not_covered": (
"language not covered: {language} ({count} files). No quick rules{engine} exist for it, so **silence about "
"these files is not a clean result**."
),
}
@@ -460,8 +482,12 @@ def _append_not_analysed(lines: list[str], degradations: Sequence[Mapping[str, o
for item in degradations:
template = _COVERAGE_DEGRADATIONS.get(str(item.get("reason") or ""))
if template is None:
# Never hide a recorded degradation because nobody wrote a sentence for it yet.
rendered.append(f"`{item.get('reason')}` (stage `{item.get('stage', 'unknown')}`)")
continue
fields = dict(item)
fields.setdefault("detail", "")
fields.setdefault("engine", "")
files = fields.get("files")
if isinstance(files, (list, tuple)):
fields["files"] = ", ".join(f"`{name}`" for name in files)
tests/test_push_background.py
+82
View File
@@ -0,0 +1,82 @@
"""Engine honesty within the deadline and the background engine (task 17.5, Requirement 9.5)."""
from __future__ import annotations
import json
import os
import time
from test_push_runner import history
from openultrasast.cpg.backend import NullBackend
from openultrasast.push import background
from openultrasast.push.runner import replay
def test_engine_off_runs_the_quick_tier_and_says_the_engine_was_skipped(tmp_path):
root, base, head, _ = history(tmp_path)
delivery = replay(root, base=base, head=head, artifact=tmp_path / "r.json", backend=NullBackend(), engine="off")
assert delivery.exit_code == 0
assert "Skipped: engine checks were switched off for this run (--engine off)" in delivery.text
data = json.loads((tmp_path / "r.json").read_text())
assert data["scans"] == [] and data["quick_tier"][0]["status"] == "completed"
def test_background_engine_result_is_shown_once_on_the_next_run(tmp_path):
root, base, head, _ = history(tmp_path)
artifacts = tmp_path / "artifacts"
first = replay(root, base=base, head=head, artifact=artifacts / "push-1.json", engine="background", background_deadline=60)
assert first.exit_code == 0
assert "Engine: running in the background for " + head[:12] in first.text
assert "Skipped: engine checks run in the background after this push" in first.text
started = json.loads((artifacts / "push-1.json").read_text())["engine_background"]
assert started["status"] == "started" and started["head"] == head
result = background.results_dir(artifacts / "push-1.json", root) / f"{head}.json"
deadline = time.monotonic() + 90
while not result.exists() and time.monotonic() < deadline:
time.sleep(0.2)
assert result.exists(), (background.results_dir(artifacts / "push-1.json", root) / f"{head}.log").read_text()
second = replay(root, base=base, head=head, artifact=artifacts / "push-2.json", backend=NullBackend(), engine="off")
assert f"Engine result for {head[:12]} from the previous push:" in second.text
assert str(result) in second.text
third = replay(root, base=base, head=head, artifact=artifacts / "push-3.json", backend=NullBackend(), engine="off")
assert "from the previous push" not in third.text
def test_one_background_engine_at_a_time(tmp_path):
root, base, head, _ = history(tmp_path)
artifact = tmp_path / "a" / "push.json"
directory = background.results_dir(artifact, root)
directory.mkdir(parents=True)
(directory / "running.json").write_text(json.dumps({"pid": os.getpid(), "head": "f" * 40}))
state = background.start(root, base=base, head=head, artifact=artifact, deadline_seconds=5)
assert state["status"] == "busy" and not (directory / f"{head}.log").exists()
delivery = replay(root, base=base, head=head, artifact=artifact, backend=NullBackend(), engine="background")
assert "is still busy, so none was started" in delivery.text
def test_previous_summarises_advisory_engine_findings(tmp_path):
root = tmp_path / "repo"
root.mkdir()
artifact = tmp_path / "out" / "push.json"
directory = background.results_dir(artifact, root)
directory.mkdir(parents=True)
delta = {
"family": "injection",
"novelty": "new",
"witness": "req.query.code -> eval",
"head_operation": {"path": "debug.js", "line": 2},
}
payload = {
"result": {"coverage_status": "incomplete"},
"admission": {
"defects": [],
"coverage_reasons": ["capability_unavailable"],
"dispositions": [{"admitted": False, "reasons": ["capability_unavailable"], "candidate": {"delta": delta}}],
},
}
(directory / ("a" * 40 + ".json")).write_text(json.dumps(payload))
(item,) = background.previous(artifact, root)
assert item["advisory"] == [{"family": "injection", "path": "debug.js", "line": 2, "novelty": "new"}]
assert item["finished"] and item["engine_ran"] and item["alerts"] == 0
assert background.previous(artifact, root) == []
tests/test_push_explain.py
+101
View File
@@ -0,0 +1,101 @@
"""Plain-language skip lines (pre-push 17.2, Requirement 9.2): nothing hidden, nothing opaque."""
import re
from dataclasses import replace
from test_push_report import budget, report_with
from openultrasast.push.explain import explain, language_reason
from openultrasast.push.report import deliver_report
# Coverage reasons recorded by the 2026-10-02 audit's live runs, verbatim in shape.
AUDIT = (
"7265636f7264732e7079:ambiguous_line_correspondence",
"base_counterpart_unresolved",
"cpg_build_failed",
"frontend_unsupported",
"analysis_incomplete",
"base_comparison_unavailable",
"eligibility:eligibility_provenance_mismatch",
*(f"dynamic_external_or_depth_context_unresolved:contributor-scan:{i:064x}" for i in range(20)),
"resolution:missing_base",
"admission_failed:TimeoutError",
"base_incomplete",
"comparison_unknown",
)
def test_audit_reasons_become_one_plain_line_each():
lines = explain(AUDIT, deadline=30)
text = "\n".join(lines)
assert "records.py" in text and "7265636f7264732e7079" not in text
assert not re.search(r"[0-9a-f]{64}", text)
assert "20 engine check(s) depend on dynamic or external code" in text
assert "Joern" in text and "new branch: the remote has no base" in text
assert "the admission step failed (TimeoutError)" in text
assert "analysis_incomplete" not in text and "analysis incomplete" not in text
# One line per distinct kind: twenty hashed reasons are one line, comparison_unknown folds into its cause.
assert len(lines) == 10
assert sum("could not be established" in line for line in lines) == 1
def test_every_glossary_reason_is_documented_for_users():
"""17.6: the skip-reason glossary in docs/scanning.md names every reason the terminal explains."""
from pathlib import Path
from openultrasast.push.explain import _NOVELTY_REASONS, _RESOLUTION, GLOSSARY
docs = (Path(__file__).resolve().parents[1] / "docs" / "scanning.md").read_text()
missing = [key for key in GLOSSARY if key not in docs]
missing += [f"resolution:{key}" for key in _RESOLUTION if f"`resolution:{key}`" not in docs]
assert not missing, missing
assert "base_incomplete" in docs and _NOVELTY_REASONS
def test_unknown_reason_is_still_printed():
assert explain(("some_future_reason",)) == ["some future reason (see the details file)"]
def test_deadline_line_names_the_deadline_and_the_background_option():
(line,) = explain(("deadline_exhausted",), deadline=30)
assert "30 s" in line and "background" in line
def test_language_not_covered_is_named_with_counts():
(line,) = explain((language_reason("go", 1, 12),))
assert line.startswith("language not covered: go (1 changed file(s), 12 in the repository)")
def test_terminal_shows_skip_lines_while_artifact_keeps_raw_reasons(tmp_path):
import json
report = report_with(0)
raw = ("7265636f7264732e7079:ambiguous_line_correspondence", f"dynamic_external_or_depth_context_unresolved:contributor-scan:{1:064x}")
report = replace(
report,
admission=replace(report.admission, coverage_reasons=raw),
result=replace(report.result, coverage_status="incomplete"),
)
delivered = deliver_report(report, tmp_path / "run.json", execution_budget=budget())
assert "Skipped: changed lines could not be matched one-to-one to base lines in records.py" in delivered.text
assert "7265636f" not in delivered.text and f"{1:064x}" not in delivered.text
assert delivered.text.count("Notice:") == 1
assert tuple(json.loads(delivered.artifact.read_text())["admission"]["coverage_reasons"]) == raw
def test_replay_names_a_changed_language_nothing_covers(tmp_path):
import json
from test_push_runner import history
from openultrasast.cpg.backend import NullBackend
from openultrasast.push.runner import replay
root, base, _, git = history(tmp_path)
(root / "run.go").write_text("package main\n")
git("add", "run.go")
git("commit", "-m", "go")
head = git("rev-parse", "HEAD")
delivery = replay(root, base=base, head=head, artifact=tmp_path / "r.json", backend=NullBackend())
assert "Skipped: language not covered: go (1 changed file(s), 1 in the repository)" in delivery.text
assert "language_not_covered:go:1:1" in json.loads((tmp_path / "r.json").read_text())["admission"]["coverage_reasons"]
tests/test_push_hook.py
+39
View File
@@ -105,3 +105,42 @@ def test_install_refuses_dangling_symlink(tmp_path):
result = subprocess.run([str(installer), str(root)], capture_output=True)
assert result.returncode != 0
assert not target.exists()
def test_missing_ousast_never_blocks_the_push(tmp_path):
"""17.1 / Req 9.1: a hook whose executable is absent allows the push and says the check did not run."""
root, _, head, git = history(tmp_path)
remote = tmp_path / "remote.git"
subprocess.run(["git", "init", "--bare", str(remote)], check=True, capture_output=True)
git("remote", "add", "origin", str(remote))
hooks = root / ".hooks"
hooks.mkdir()
git("config", "core.hooksPath", ".hooks")
shutil.copyfile(Path(__file__).resolve().parents[1] / "ops/pre-push", hooks / "pre-push")
(hooks / "pre-push").chmod(0o755)
env = {**os.environ, "OUSAST_COMMAND": "ousast-not-installed-anywhere", "OUSAST_ARTIFACT_DIR": str(tmp_path / "a")}
result = subprocess.run(["git", "-C", str(root), "push", "origin", "HEAD:refs/heads/main"], env=env, capture_output=True)
assert result.returncode == 0, result.stderr
assert b"pre-push check did not run" in result.stdout + result.stderr
assert git("ls-remote", str(remote)).startswith(head)
@pytest.mark.parametrize(
("extra", "expected"),
[
(["--deadline", "abc"], 0),
(["--cache-dir", "INSIDE"], 0),
(["--deadline", "abc", "--mode", "blocking"], 2),
],
)
def test_hook_mode_usage_errors_are_advisory_unless_blocking(tmp_path, extra, expected):
"""17.1 / Req 9.1: invalid settings in hook mode allow the push; explicit blocking keeps exit 2."""
import sys
root, _, _, _ = history(tmp_path)
extra = [str(root / "cache") if item == "INSIDE" else item for item in extra]
command = [sys.executable, "-m", "openultrasast.cli", "pre-push", str(root), "--artifact", str(tmp_path / "r.json")]
result = subprocess.run([*command, *extra, "--remote", "origin", "/tmp/remote"], input=b"", capture_output=True, timeout=60)
assert result.returncode == expected, result.stderr
if expected == 0:
assert b"pre-push check did not run" in result.stdout
tests/test_push_install.py
+125
View File
@@ -0,0 +1,125 @@
"""`ousast pre-push install|uninstall` and the hook-callable Docker wrapper (task 17.6, Req 7.4, 9.6)."""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
import pytest
from test_push_runner import history
from openultrasast.cli import main
from openultrasast.push.install import install, packaged, uninstall
OPS = Path(__file__).resolve().parents[1] / "ops"
def repo(tmp_path):
root, base, head, git = history(tmp_path)
git("config", "core.hooksPath", ".custom-hooks")
remote = tmp_path / "remote.git"
subprocess.run(["git", "init", "--bare", str(remote)], check=True, capture_output=True)
git("remote", "add", "origin", str(remote))
return root, git
def test_packaged_hook_and_wrapper_are_the_ops_scripts():
assert packaged("pre-push") == (OPS / "pre-push").read_bytes()
assert packaged("ousast-docker") == (OPS / "ousast-docker").read_bytes()
def test_install_uninstall_through_the_cli_honours_hooks_path(tmp_path, capsys):
root, git = repo(tmp_path)
assert main(["pre-push", "install", str(root)]) == 0
hook = root / ".custom-hooks" / "pre-push"
assert hook.read_bytes() == packaged("pre-push") and os.access(hook, os.X_OK)
assert git("config", "core.hooksPath") == ".custom-hooks"
assert main(["pre-push", "install", str(root)]) == 1 # already installed: refused without --force
assert "already installed" in capsys.readouterr().err
assert main(["pre-push", "install", str(root), "--force"]) == 0
assert main(["pre-push", "uninstall", str(root)]) == 0
assert not hook.exists()
def test_existing_hook_is_refused_then_chained_and_restored(tmp_path):
root, git = repo(tmp_path)
hooks = root / ".custom-hooks"
hooks.mkdir()
log = tmp_path / "prior-input"
original = f'#!/bin/sh\ncat > "{log}"\nexit 0\n'.encode()
(hooks / "pre-push").write_bytes(original)
(hooks / "pre-push").chmod(0o755)
code, lines = install(root)
assert code == 1 and "existing pre-push hook" in lines[0] and (hooks / "pre-push").read_bytes() == original
code, lines = install(root, force=True)
assert code == 0 and (hooks / "pre-push.before-ousast").read_bytes() == original
env = {**os.environ, "OUSAST_COMMAND": "ousast-not-installed-anywhere", "OUSAST_ARTIFACT_DIR": str(tmp_path / "a")}
pushed = subprocess.run(["git", "-C", str(root), "push", "origin", "HEAD:refs/heads/main"], env=env, capture_output=True)
assert pushed.returncode == 0 and "refs/heads/main" in log.read_text()
code, _ = uninstall(root)
assert code == 0 and (hooks / "pre-push").read_bytes() == original
assert not (hooks / "pre-push.before-ousast").exists()
code, lines = uninstall(root)
assert code == 1 and "not written by OpenUltraSAST" in lines[0]
def fake_docker(tmp_path, *, inspect=0, run=0):
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
record = tmp_path / "docker-args"
stdin = tmp_path / "docker-stdin"
script = f'#!/bin/sh\nif [ "$1" = image ]; then exit {inspect}; fi\nprintf "%s\\n" "$@" > "{record}"\ncat > "{stdin}"\nexit {run}\n'
(bin_dir / "docker").write_text(script)
(bin_dir / "docker").chmod(0o755)
return bin_dir, record, stdin
def test_docker_install_routes_the_hook_through_the_wrapper_with_stdin_and_mounts(tmp_path):
root, git = repo(tmp_path)
assert install(root, docker=True)[0] == 0
bin_dir, record, stdin = fake_docker(tmp_path)
artifacts = tmp_path / "artifacts"
env = {**os.environ, "PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}", "OUSAST_ARTIFACT_DIR": str(artifacts)}
env.pop("OUSAST_COMMAND", None)
pushed = subprocess.run(["git", "-C", str(root), "push", "origin", "HEAD:refs/heads/main"], env=env, capture_output=True)
assert pushed.returncode == 0, pushed.stderr
args = record.read_text().splitlines()
real = str(root.resolve())
assert args[:2] == ["run", "--rm"] and "-i" in args and "--network" in args and "none" in args
assert f"{real}:{real}:ro" in args and str(artifacts.resolve()) + ":" + str(artifacts.resolve()) in args
assert args[args.index("-w") + 1] == real
assert "ghcr.io/norandom/openultrasast:2.0.1" in args and "pre-push" in args and "--remote" in args
assert stdin.read_text().startswith("HEAD ") and "refs/heads/main" in stdin.read_text()
@pytest.mark.parametrize(("inspect", "run", "needle"), [(1, 0, "is not present"), (0, 125, "could not start")])
def test_wrapper_failures_never_block(tmp_path, inspect, run, needle):
bin_dir, _, _ = fake_docker(tmp_path, inspect=inspect, run=run)
env = {**os.environ, "PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}", "OUSAST_ARTIFACT_DIR": str(tmp_path / "artifacts")}
result = subprocess.run(
[str(OPS / "ousast-docker"), "pre-push", ".", "--artifact", str(tmp_path / "x" / "r.json")],
env=env,
cwd=tmp_path,
input=b"",
capture_output=True,
)
assert result.returncode == 0 and needle.encode() in result.stdout
@pytest.mark.parametrize(("mode", "status", "expected"), [("blocking", 1, 1), ("advisory", 1, 0), ("advisory", 2, 0)])
def test_only_explicit_blocking_passes_a_nonzero_exit_through(tmp_path, mode, status, expected):
bin_dir, _, _ = fake_docker(tmp_path, run=status)
env = {
**os.environ,
"PATH": f"{bin_dir}{os.pathsep}{os.environ['PATH']}",
"OUSAST_ARTIFACT_DIR": str(tmp_path / "artifacts"),
"OUSAST_PUSH_MODE": mode,
}
result = subprocess.run([str(OPS / "ousast-docker"), "pre-push", "."], env=env, cwd=tmp_path, input=b"", capture_output=True)
assert result.returncode == expected
def test_hook_passes_optional_settings_only_when_set():
text = (OPS / "pre-push").read_text()
assert '--engine "$OUSAST_PUSH_ENGINE"' in text and 'if [ -n "${OUSAST_PUSH_ENGINE:-}" ]' in text
tests/test_push_new_file.py
+96
View File
@@ -0,0 +1,96 @@
"""A file new in head is `new`, not `comparison unknown` (task 17.4, Requirement 9.4)."""
from __future__ import annotations
from dataclasses import replace
from test_push_admission import control
from test_push_policy import delta, scan
from test_push_report import budget, report_with
from openultrasast.model.contracts import ChangeContext, ChangedSpan
from openultrasast.push.policy import admit_candidates
from openultrasast.push.report import deliver_report
def added(*, boundaries=(), listed=True):
return ChangeContext(
"base",
"head",
("a.js",),
(),
(ChangedSpan("a.js", 1, 6, "head"),),
(),
(),
(),
tuple(boundaries),
added_paths=("a.js",) if listed else (),
)
def test_snapshot_records_added_paths_and_not_modified_ones(tmp_path):
import json
from test_push_runner import history
from openultrasast.cpg.backend import NullBackend
from openultrasast.push.runner import replay
root, _, base, git = history(tmp_path)
(root / "debug.js").write_text("eval(x)\n")
(root / "api.js").write_text("changed\n")
git("add", "debug.js", "api.js")
git("commit", "-m", "add and modify")
head = git("rev-parse", "HEAD")
replay(root, base=base, head=head, artifact=tmp_path / "r.json", backend=NullBackend())
context = json.loads((tmp_path / "r.json").read_text())["change_context"]
assert context["added_paths"] == [b"debug.js".hex()]
def test_operation_in_an_added_file_is_new_without_any_base_answer():
result = delta(scan(), None, added())
assert (result.novelty, result.reason) == ("new", "file_added_in_head")
assert result.change_evidence == ("head:a.js:1-6",)
def test_the_same_shape_without_the_added_fact_keeps_its_veto():
assert delta(scan(), None, added(listed=False)).novelty == "unknown"
def test_unpaired_add_and_delete_stays_unknown():
result = delta(scan(), None, added(boundaries=("unmatched_added_deleted_path_correspondence",)))
assert (result.novelty, result.reason) == ("unknown", "change_context_unresolved")
def test_an_operation_already_in_the_base_elsewhere_is_movement_not_new():
result = delta(scan(), scan(path="b.js", line=9), added())
assert (result.novelty, result.reason) == ("unknown", "operation_moved_within_change")
def test_unentailed_or_discharged_claims_in_added_files_are_not_new():
assert delta(scan(sanitized=True), None, added()).novelty == "unknown"
def test_admission_accepts_the_added_file_reason():
candidate, capability, _ = control()
new_file = replace(candidate, delta=replace(candidate.delta, reason="file_added_in_head"))
assert len(admit_candidates((new_file,), capabilities=(capability,)).defects) == 1
unqualified = admit_candidates((new_file,))
assert unqualified.dispositions[0].reasons == ("capability_unavailable",)
def test_unqualified_new_engine_finding_is_shown_as_advisory_not_alert(tmp_path):
report = report_with(1, admitted=False)
delivered = deliver_report(report, tmp_path / "r.json", execution_budget=budget())
assert delivered.exit_code == 0 and delivered.result.actionable_defect_ids == ()
assert "Engine: 1 finding(s) this push introduced. Advisory" in delivered.text
assert "- injection at handler.src:4 (new): request.input -> exec(command)" in delivered.text
def test_a_comparison_veto_keeps_an_engine_finding_out_of_the_terminal(tmp_path):
report = report_with(1, admitted=False)
disposition = report.admission.dispositions[0]
vetoed = replace(disposition, reasons=(*disposition.reasons, "comparison_unknown"))
report = replace(report, admission=replace(report.admission, dispositions=(vetoed,)))
delivered = deliver_report(report, tmp_path / "r.json", execution_budget=budget())
assert "Engine:" not in delivered.text
tests/test_push_quick.py
+54
View File
@@ -0,0 +1,54 @@
"""The quick-rule tier of pre-push (task 17.3, Requirement 9.3): fast, advisory, never an alert."""
from __future__ import annotations
import json
from test_push_runner import history
from openultrasast.cpg.backend import NullBackend
from openultrasast.push.quick import run_quick_tier
from openultrasast.push.runner import push, replay
HANDLER = "module.exports = function debug (req, res) {\n res.end(String(eval(req.query.code)))\n}\n"
def test_only_matches_on_changed_lines_are_kept_and_input_is_proven(tmp_path):
(tmp_path / "debug.js").write_text(HANDLER)
(tmp_path / "old.js").write_text("eval(x)\n")
hit = run_quick_tier(tmp_path, {"debug.js": [(2, 2)], "old.js": [(5, 5)]})
assert hit.status == "completed" and hit.files == ("debug.js", "old.js")
assert hit.bytes_read == len(HANDLER) + len("eval(x)\n")
assert {(f["rule"], f["path"], f["line"], f["scope"]) for f in hit.findings} >= {("js-eval", "debug.js", 2, "changed_line")}
assert all(f["path"] == "debug.js" for f in hit.findings)
miss = run_quick_tier(tmp_path, {"debug.js": [(3, 3)]})
assert miss.findings == () and miss.files == ("debug.js",)
def test_replay_shows_quick_matches_as_advisory_and_never_admits_or_blocks(tmp_path):
root, _, base, git = history(tmp_path)
(root / "debug.js").write_text(HANDLER)
git("add", "debug.js")
git("commit", "-m", "debug")
head = git("rev-parse", "HEAD")
artifact = tmp_path / "r.json"
delivery = replay(root, base=base, head=head, artifact=artifact, backend=NullBackend())
assert delivery.exit_code == 0 and delivery.result.actionable_defect_ids == ()
assert "Quick rules: " in delivery.text and "Advisory: not verified by the engine." in delivery.text
assert "- js-eval (CWE-95" in delivery.text and "debug.js:2" in delivery.text
data = json.loads(artifact.read_text())
(tier,) = data["quick_tier"]
assert tier["status"] == "completed" and tier["basis"] == "comparison" and tier["files"] == ["debug.js"]
assert tier["bytes_read"] == len(HANDLER) and "advisory" in tier["label"]
assert data["admission"]["defects"] == []
def test_new_branch_says_there_is_no_base_and_what_was_checked_instead(tmp_path):
root, _, head, git = history(tmp_path)
updates = f"refs/heads/feature {head} refs/heads/feature {'0' * 40}\n"
delivery = push(root, updates=updates, remote_name="origin", remote_url="/r", artifact=tmp_path / "p.json", backend=NullBackend())
assert delivery.exit_code == 0
assert "Skipped: new branch: the remote has no base to compare against" in delivery.text
assert "New branch: instead, quick rules checked the files its last commit changed" in delivery.text
(tier,) = json.loads((tmp_path / "p.json").read_text())["quick_tier"]
assert tier["basis"] == "last_commit" and tier["files"] == ["api.js"]
tests/test_push_vetoes.py
+21 -1
View File
@@ -247,8 +247,28 @@ def test_cli_refuses_the_flag_outside_explicit_replay(tmp_path, capsys):
from openultrasast.cli import main
root, base, head, _ = history(tmp_path)
# Hook mode refuses the flag without running anything; since 17.1 (Req 9.1) the refusal is advisory
# (exit 0 with a line), because a usage error in a hook must never reject the push.
hook = ["pre-push", str(root), "--remote", "origin", "/x", "--artifact", str(tmp_path / "x.json")]
refused = main([*hook, "--experimental-record-vetoes"])
captured = capsys.readouterr()
assert refused == 0 and "requires explicit --base/--head replay" in captured.err
assert "pre-push check did not run" in captured.out and not (tmp_path / "x.json").exists()
with pytest.raises(SystemExit):
main(["pre-push", str(root), "--remote", "origin", "/x", "--artifact", str(tmp_path / "x.json"), "--experimental-record-vetoes"])
main(
[
"pre-push",
str(root),
"--remote",
"origin",
"/x",
"--artifact",
str(tmp_path / "x.json"),
"--mode",
"blocking",
"--experimental-record-vetoes",
]
)
assert "requires explicit --base/--head replay" in capsys.readouterr().err
code = main(
[
tests/test_reports.py
+43
View File
@@ -654,3 +654,46 @@ def test_a_clean_scan_states_no_coverage_gaps(tmp_path) -> None: # type: ignore
degradations=[],
)
assert "### What could not be analysed" not in path.read_text()
def test_every_recorded_scan_degradation_is_rendered(tmp_path) -> None: # type: ignore[no-untyped-def]
"""pre-push 17.2 / Req 9.2: the eight degradations cli.py records and the report used to drop."""
from openultrasast.reports import write_markdown_report
reasons = [
{"stage": "model", "reason": "learning_endpoint_unavailable"},
{"stage": "model", "reason": "cpg_unavailable"},
{"stage": "overlay", "reason": "facts_unavailable", "detail": "missing table"},
{"stage": "obligations", "reason": "facts_unavailable"},
{"stage": "obligations", "reason": "policy_invalid", "detail": "bad toml"},
{"stage": "model", "reason": "model_withheld_execution_budget"},
{"stage": "budget", "reason": "max_findings_exceeded", "requested": 5, "actual": 9},
{"stage": "map", "reason": "hunter_model_unavailable"},
{"stage": "regress", "reason": "sandbox_unavailable"},
{"stage": "x", "reason": "not_yet_described"},
]
path = tmp_path / "report.md"
write_markdown_report([], path, [], degradations=reasons)
text = path.read_text()
for needle in (
"No model endpoint is configured",
"Joern engine is not installed",
"fact tables for the `overlay` stage",
"fact tables for the `obligations` stage",
"`obligations` policy could not be loaded",
"time budget leaves no room",
"first 5 of 9 findings",
"needs a model and none is available",
"Docker is not available",
"`not_yet_described`",
):
assert needle in text, needle
def test_quick_scan_names_a_language_nothing_covers(tmp_path, capsys) -> None: # type: ignore[no-untyped-def]
"""pre-push 17.2 / Req 9.2: a Go-only tree is not a silent zero."""
(tmp_path / "run.go").write_text('package main\n\nimport "os/exec"\n\nfunc run(c string) { exec.Command("sh", "-c", c).Run() }\n')
assert main(["scan", str(tmp_path), "--mode", "quick"]) == 0
assert "notice: language not covered: go (1 files)" in capsys.readouterr().out
report = next((tmp_path / ".openultrasast" / "runs").iterdir()) / "report.md"
assert "Not a clean result for go (1 files)" in report.read_text()