From a656fda2f46f29e423e2f647a11f21f7ba9b7429 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Fri, 2 Oct 2026 21:46:25 +0200 Subject: [PATCH] pre-push(17): operational-readiness task group and Requirement 9 Append task group 17 "Operational readiness (2026-10-02)" with one task per maintainer-approved item (never block, name every skip, quick tier, new-file classification, engine honesty and background mode, install/Docker/docs, proof), add Requirement 9 (9.1-9.6, marked added 2026-10-02, maintainer-approved), and check 13.4 and the parent of 14 as the audit found them done. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo --- .../specs/pre-push-safety-net/requirements.md | 37 +++++++++++++++ .kiro/specs/pre-push-safety-net/tasks.md | 45 ++++++++++++++++++- 2 files changed, 80 insertions(+), 2 deletions(-) diff --git a/.kiro/specs/pre-push-safety-net/requirements.md b/.kiro/specs/pre-push-safety-net/requirements.md index 6ddd60b..7587e23 100644 --- a/.kiro/specs/pre-push-safety-net/requirements.md +++ b/.kiro/specs/pre-push-safety-net/requirements.md @@ -147,6 +147,43 @@ with Express-style routes and middleware, including vulnerable, fixed and benign frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or unsupported-property results separately from demonstrated vulnerability detection. +## Requirement 9: Operate as an advisory safety net on any repository + +Added 2026-10-02, maintainer-approved, from the operational-readiness audit. The maintainer's +definition: a user installs the hook on ANY repository and gets honest, advisory-only results; +nothing ever blocks a push unless blocking was explicitly chosen. + +9.1 (added 2026-10-02, maintainer-approved) When the installed hook cannot run the check, because the +`ousast` executable is missing or an argument or setting is invalid, the hook shall allow the push and +print one line saying the check did not run and how to fix it; only an explicit `--mode blocking` +opt-in may turn an analysis result into a rejected push. + +9.2 (added 2026-10-02, maintainer-approved) When a check is skipped or incomplete, the terminal output +shall name each distinct reason in plain words, one line per reason, with repository paths decoded and +no opaque hash tokens; a language with no engine frontend and no quick rules shall be named with its +file count; a first push of a new branch shall say there was no base to compare against and what was +checked instead. The JSON artifact keeps the raw reason values. + +9.3 (added 2026-10-02, maintainer-approved) Before the engine, within the same deadline, the Safety Net +shall run the shipped quick pattern rules over the head side of the changed lines and show their +findings as advisory pattern matches, labeled separately from admitted alerts. This amends 3.2 for this +tier only: quick-rule findings are shown, never admitted, and never block. + +9.4 (added 2026-10-02, maintainer-approved) When a finding's operation lies in a file that is new in the +head revision, the comparison shall classify it as new rather than unknown; an engine finding whose +novelty is established but whose capability is not yet qualified shall be shown as an advisory engine +finding, not an admitted alert. Genuinely unknown comparisons keep their vetoes. + +9.5 (added 2026-10-02, maintainer-approved) When the engine cannot finish within the deadline, one line +shall say so; on explicit request (`--engine background`) the Safety Net shall start the engine +detached after the push, record its result in the artifact directory, and show it on the next run as +the engine result for that head from the previous push. + +9.6 (added 2026-10-02, maintainer-approved) The hook installer and remover shall ship in the wheel +(`ousast pre-push install|uninstall`), refuse to overwrite an existing hook unless forced, and a +hook-callable container wrapper shall pass the push input and artifact path through; the documentation +shall give a first-run guide, a glossary of skip reasons, a CI recipe and help text for every flag. + ## Accepted experimental acceptance profile These values are the initial implementation/evaluation profile accepted for task generation, not diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md index ff3d6f4..cc0c39e 100644 --- a/.kiro/specs/pre-push-safety-net/tasks.md +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -475,7 +475,7 @@ Preserve all completed task evidence below. - _Depends: 13.1_ - _Requirements: 6.2, 6.3, 8.1, 8.2_ -- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4) +- [x] 14. Ask every declared family whether it establishes anything (precondition for M4) - Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing. - [x] 14.1 Census the seven declared families on NodeGoat and VAmPI - Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only. @@ -575,7 +575,7 @@ Preserve all completed task evidence below. - _Boundary: Evaluation harness, full-run integration_ - _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_ - _Requirements: 6.2, 6.3, 8.1, 8.2_ -- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds +- [x] 13.4 Let a question inside one partition complete, whatever else the repository holds - **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan. - The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote. - **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it. @@ -587,6 +587,7 @@ Preserve all completed task evidence below. - _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical. - _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`. - _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_ + - _Checked 2026-10-02 per the operational-readiness audit:_ implemented and verified on PMPro (120 of 127, above). 13.3 stays open on its own blocker; this box was left unchecked only because it waits on it. - _Depends: 13.3_ - _Requirements: 2.1, 2.2, 6.3, 8.1_ @@ -619,6 +620,46 @@ Preserve all completed task evidence below. - _Requirements: 8.1, 8.2_ +- [ ] 17. Operational readiness (2026-10-02) + - **Named by the 2026-10-02 operational-readiness audit**, maintainer-approved as the first increment. The maintainer's definition: a user installs the hook on ANY repository and gets honest, advisory-only results; nothing ever blocks a push. The audit found the plumbing sound (snapshots, resolution, deadline accounting, advisory exit 0, the JSON artifact) and five things in the way: the hook can block a push, quick rules are not part of pre-push, the engine cannot finish a 26-file repository inside 30 s, the terminal hides every skip reason, and there is no supported install path without the source checkout. Before numbers for every item are the audit's live runs on records (Python), serve-static (JavaScript) and unrolled/secure (Go), each with one planted commit. + - This group is plumbing and honesty. It enables no capability, edits no rule for a planted sample, and leaves graph reuse (the engine speed work decided in 11.6) as a later task. + - _Requirements: 9.1–9.6_ +- [ ] 17.1 Never block a push the user did not choose to block + - The installed hook exits 0 with one line when `ousast` is missing (the check did not run, and how to install). In hook (stdin) mode, an argument or usage error (an invalid `OUSAST_PUSH_DEADLINE`, a cache directory inside the repository) is advisory: one line and exit 0. `--mode blocking` stays the explicit opt-in that can reject a push, and keeps its usage errors. The prior-hook input-retention failure keeps exit 1, because it is the chained hook's verdict that cannot be obtained, not ours. + - Files: `ops/pre-push`, `src/openultrasast/cli.py`. Tests: `tests/test_push_hook.py` (a push with `ousast` absent from PATH succeeds and prints the line; a bad deadline in hook mode exits 0; blocking mode keeps exit 2). + - Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0. + - _Requirements: 9.1, 5.3, 5.4_ +- [ ] 17.2 Name every skipped check in plain words + - One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: ( files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`). + - Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`. + - Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch. + - _Requirements: 9.2, 5.2, 8.2_ +- [ ] 17.3 Run the quick rules as the fast first tier + - Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block. + - Files: `src/openultrasast/push/quick.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_quick.py` (new). + - Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it. + - _Requirements: 9.3_ +- [ ] 17.4 Classify a file new in head as new, not unknown + - `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto. + - Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`. + - Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB). + - _Requirements: 9.4, 3.4_ +- [ ] 17.5 Say when the engine cannot finish, and offer it in the background + - One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work. + - Files: `src/openultrasast/push/background.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/cli.py`, `ops/pre-push`. Tests: `tests/test_push_background.py` (new). + - Evidence: a hook push with `OUSAST_PUSH_ENGINE=background` followed by a second push showing the previous result. + - _Requirements: 9.5, 4.1, 4.2_ +- [ ] 17.6 Install, Docker and documentation for a first-time user + - `ousast pre-push install|uninstall [--force] [--docker] [REPO]` ships the hook as package data, honours `core.hooksPath`, refuses an existing hook unless `--force` (which chains it as `pre-push.before-ousast`), and works from a pip install. `ops/ousast-docker` is a hook-callable script (not a shell function) that passes stdin and the artifact path through. The `openultrasast:dev` image is rebuilt only if `df -h /` shows at least 4 GB free; otherwise the rebuild command is documented and the record says it was not rebuilt. `docs/scanning.md` gains a first-run guide, a glossary of skip reasons, a GitHub Actions recipe and full `--help` text; README tiers 1 and 2 updated, pulling `ghcr.io/norandom/openultrasast:2.0.1` where the wrapper allows. + - Files: `src/openultrasast/push/hook/pre-push`, `src/openultrasast/push/install.py` (new), `src/openultrasast/cli.py`, `ops/ousast-docker`, `docs/scanning.md`, `README.md`, `ops/README.md`. Tests: `tests/test_push_install.py` (new), `mkdocs build --strict`. + - Evidence: install from the package into a scratch clone, push through it, uninstall restores the prior hook. + - _Requirements: 9.6, 7.4_ +- [ ] 17.7 Prove it on the audit's three repositories + - Rerun records, serve-static and unrolled/secure (same planted commits) with the new build in quick mode and with the hook installed, plus one Docker engine run on serve-static if free memory >= 3 GB and free disk >= 1 GB. Prove each run read its input (files, bytes). + - Files: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json`. Evidence: before (the audit) and after: exit codes, wall time against the deadline, findings shown, every skip line printed. A planted sample still missed is a coverage gap, recorded, never a rule edit. + - _Requirements: 9.1–9.6, 6.3_ + + ## Implementation Notes - Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.