From 7e3b3bbc8b95b0bf947b5e9746382fcf9d8cfdb8 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Sat, 12 Sep 2026 18:27:14 +0200 Subject: [PATCH] docs(pre-push-safety-net): record approved design and implementation plan --- .kiro/specs/contributor-scan/design.md | 24 ++ .kiro/specs/contributor-scan/spec.json | 13 +- .kiro/specs/flow-aware-ranking/brief.md | 68 ++++ .kiro/specs/flow-aware-ranking/spec.json | 11 +- .kiro/specs/pre-push-safety-net/brief.md | 75 ++++ .kiro/specs/pre-push-safety-net/design.md | 339 ++++++++++++++++++ .kiro/specs/pre-push-safety-net/evaluation.md | 82 +++++ .../specs/pre-push-safety-net/requirements.md | 159 ++++++++ .kiro/specs/pre-push-safety-net/research.md | 142 ++++++++ .kiro/specs/pre-push-safety-net/spec.json | 54 +++ .../specs/pre-push-safety-net/task-review.md | 56 +++ .kiro/specs/pre-push-safety-net/tasks.md | 217 +++++++++++ .kiro/steering/overview.md | 12 + .kiro/steering/roadmap.md | 67 ++++ .kiro/steering/safety-net.md | 82 +++++ 15 files changed, 1398 insertions(+), 3 deletions(-) create mode 100644 .kiro/specs/pre-push-safety-net/brief.md create mode 100644 .kiro/specs/pre-push-safety-net/design.md create mode 100644 .kiro/specs/pre-push-safety-net/evaluation.md create mode 100644 .kiro/specs/pre-push-safety-net/requirements.md create mode 100644 .kiro/specs/pre-push-safety-net/research.md create mode 100644 .kiro/specs/pre-push-safety-net/spec.json create mode 100644 .kiro/specs/pre-push-safety-net/task-review.md create mode 100644 .kiro/specs/pre-push-safety-net/tasks.md create mode 100644 .kiro/steering/safety-net.md diff --git a/.kiro/specs/contributor-scan/design.md b/.kiro/specs/contributor-scan/design.md index f57149d..ce8b6e7 100644 --- a/.kiro/specs/contributor-scan/design.md +++ b/.kiro/specs/contributor-scan/design.md @@ -1,5 +1,29 @@ # Design Document: contributor-scan +## Integration amendment — 2026-09-12 (draft, unapproved) + +The maintainer has clarified the product as an actionable pre-push safety net across supported +ecosystems. The proposed successor `../pre-push-safety-net/design.md` consumes this driver/backend +without replacing its ranker: the ranker remains the scope mechanism, and vendor code stays out +of both targets and graphs. Existing approvals below apply to the prior design, not this amendment. + +Proposed shared seams needing requirements/design review before implementation: + +- Accept an absolute deadline across graph build, overlays, retries and queries, not only region/call caps. +- Accept an optional validated reusable graph lease while preserving ordinary fresh-build scans; + backend remains authoritative for artifact identity, census, compatibility and cleanup. +- Return exact selected/deferred question IDs and context boundaries from ranker decisions; + callers must not reconstruct exclusions by slicing an earlier order. +- Record engine, fact, query, layout and ranking provenance. Align Docker/Compose/ops pins with + the evaluated baseline before packaging parity claims. +- Resolve task 5.6's family-specific sanitizer semantics before admitting affected capabilities to + a low-noise hook; ranker improvements do not repair detector semantics. + +The push spec owns immutable Git inputs, comparison/admission policy, persistence and presentation. +This spec owns reusable engine contracts and family correctness. No hook imports belong inside +the detector, no second heuristic scope selector is introduced, and no task is marked complete by +this documentation amendment. Revalidate normal scans and repository benchmarks when seams change. + ## Overview `model-grounded-detection` built an arbiter and measured it at 90.0% pair-correct / 6.7% leak on vibe-py. It diff --git a/.kiro/specs/contributor-scan/spec.json b/.kiro/specs/contributor-scan/spec.json index 4ec5260..f6ae709 100644 --- a/.kiro/specs/contributor-scan/spec.json +++ b/.kiro/specs/contributor-scan/spec.json @@ -1,7 +1,7 @@ { "feature_name": "contributor-scan", "created_at": "2026-09-08T20:00:00Z", - "updated_at": "2026-09-09T12:30:00Z", + "updated_at": "2026-09-12T15:35:00Z", "language": "en", "phase": "implementation", "approvals": { @@ -22,5 +22,14 @@ "depends_on": [ "model-grounded-detection" ], - "approval_note": "2026-09-08 brief drafted after the maintainer asked what the spec state was and whether anything was forgotten. It was: model-grounded-detection is 16/16 complete and its arbiter is measured at 86.7% pair-correct / 6.7% leak, but the model layer has ZERO production callers -- ousast scan does not use it, so the feature is complete and unshipped at once. That is a scope gap in the predecessor (it had no task saying 'make this reachable', and its byte-identical-gates requirement is satisfied precisely because nothing is wired). This spec covers scan integration, whole-repo scanning against known vulnerable repositories, the deferred memory-family decision, and records the pipeline, ops and corpus work delivered outside any spec. Not approved; requirements not generated. ; 2026-09-08 requirements generated at the maintainer's direction ('we follow the spec and build integration afterwards'). ; 2026-09-09 requirements approved and design generated ('let us continue with these'): the model layer joins as one stage inside MAP beside the obligations checker, one CPG per scan, regions from entry points, a total budget rather than a per-region cap. Phase 2 (the first run on a pinned real repository) can invalidate the rest and comes before shipping. ; 2026-09-09 tasks approved by the maintainer ('approve the tasks and start group 1'); implementing group 1 (regions, driver, MAP stage). ; 2026-09-09 group 2 complete except 2.8/2.9/2.13-2.16, and it reshaped the phase as intended. Nine engine defects found by measurement and none by review, gates byte-identical through every fix -- which says the pair corpus could not have caught any of them. Three requirements amendments are DRAFTED AND UNAPPROVED at the end of requirements.md: (A) Req 5 should state what a report may repeat, because libpng emitted 198 findings with nothing in the spec forbidding it, and the model layer's own suspicion band is still 38 of vampi's 60; (B) a family that cannot decide a class must say so, because c/memory scored zero true positives on 89k lines of C for a structural reason and a silent scan reads as a clean bill of health; (C) Req 4.1 is satisfied by a checkout that cannot demonstrate detection at all, so it should require one that can. Separately, two task groups appended to tasks.md without passing requirements or design have been LIFTED OUT to the new finding-feedback-loop spec, where they belong. ; 2026-09-09 the three group-2 amendments APPROVED by the maintainer and folded into the numbered requirements: Req 5.5 (a report proportionate to the evidence behind a finding), Req 5.6 (a family that cannot decide a class states the limit, and no scan presents absence of findings as evidence of absence), Req 4.6 (at least one checkout carrying a vulnerability a family can decide in principle). Req 5.5 is half implemented by task 2.11 -- pattern matches -- and task 2.17 was added for the model layer's own suspicion band, which is 38 of vampi's 60 findings and is an enumerated candidate the graph could not decide with a judge's opinion attached rather than a reasoned claim." + "approval_note": "2026-09-08 brief drafted after the maintainer asked what the spec state was and whether anything was forgotten. It was: model-grounded-detection is 16/16 complete and its arbiter is measured at 86.7% pair-correct / 6.7% leak, but the model layer has ZERO production callers -- ousast scan does not use it, so the feature is complete and unshipped at once. That is a scope gap in the predecessor (it had no task saying 'make this reachable', and its byte-identical-gates requirement is satisfied precisely because nothing is wired). This spec covers scan integration, whole-repo scanning against known vulnerable repositories, the deferred memory-family decision, and records the pipeline, ops and corpus work delivered outside any spec. Not approved; requirements not generated. ; 2026-09-08 requirements generated at the maintainer's direction ('we follow the spec and build integration afterwards'). ; 2026-09-09 requirements approved and design generated ('let us continue with these'): the model layer joins as one stage inside MAP beside the obligations checker, one CPG per scan, regions from entry points, a total budget rather than a per-region cap. Phase 2 (the first run on a pinned real repository) can invalidate the rest and comes before shipping. ; 2026-09-09 tasks approved by the maintainer ('approve the tasks and start group 1'); implementing group 1 (regions, driver, MAP stage). ; 2026-09-09 group 2 complete except 2.8/2.9/2.13-2.16, and it reshaped the phase as intended. Nine engine defects found by measurement and none by review, gates byte-identical through every fix -- which says the pair corpus could not have caught any of them. Three requirements amendments are DRAFTED AND UNAPPROVED at the end of requirements.md: (A) Req 5 should state what a report may repeat, because libpng emitted 198 findings with nothing in the spec forbidding it, and the model layer's own suspicion band is still 38 of vampi's 60; (B) a family that cannot decide a class must say so, because c/memory scored zero true positives on 89k lines of C for a structural reason and a silent scan reads as a clean bill of health; (C) Req 4.1 is satisfied by a checkout that cannot demonstrate detection at all, so it should require one that can. Separately, two task groups appended to tasks.md without passing requirements or design have been LIFTED OUT to the new finding-feedback-loop spec, where they belong. ; 2026-09-09 the three group-2 amendments APPROVED by the maintainer and folded into the numbered requirements: Req 5.5 (a report proportionate to the evidence behind a finding), Req 5.6 (a family that cannot decide a class states the limit, and no scan presents absence of findings as evidence of absence), Req 4.6 (at least one checkout carrying a vulnerability a family can decide in principle). Req 5.5 is half implemented by task 2.11 -- pattern matches -- and task 2.17 was added for the model layer's own suspicion band, which is 38 of vampi's 60 findings and is an enumerated candidate the graph could not decide with a judge's opinion attached rather than a reasoned claim.", + "pending_amendments": [ + { + "date": "2026-09-12", + "document": "design.md", + "topic": "Pre-push integration: shared deadline, reusable graph lease, exact ranker scope and provenance; preserve vendor exclusion.", + "approved": false, + "ready_for_implementation": false + } + ] } diff --git a/.kiro/specs/flow-aware-ranking/brief.md b/.kiro/specs/flow-aware-ranking/brief.md index 30d2f35..2ce9fc3 100644 --- a/.kiro/specs/flow-aware-ranking/brief.md +++ b/.kiro/specs/flow-aware-ranking/brief.md @@ -2,6 +2,74 @@ **Status:** brief only, unapproved. +## Current amendment — 2026-09-12 + +This section supersedes forward-looking interpretations below; the iteration history is retained. +The product goal is a scalable, actionable pre-push security safety net as AI accelerates code changes. +PMPro and WordPress are regression workloads within that goal, not its scope boundary. See +`../pre-push-safety-net/` for proposed push requirements and design, and +`../../steering/safety-net.md` for the maintainer's product direction. + +**The ranker remains the scope mechanism.** It selects and orders the regions/questions that +receive analysis. The push adapter supplies change and affected first-party context; it does not +create a competing scope heuristic. Third-party vendor code remains excluded from both targets +and graphs. Tests remain available but unshipped, as already decided. Missing external semantics +need explicit summaries or unresolved coverage, never automatic vendor reintroduction. + +### Verified repository facts and bounded interpretations + +- Latest recorded phase-2 budgets: PMPro **82**, WP Statistics **91**, MW WP Form **54**, VAmPI **16**. + The `<50` gate remains unmet. The evidence passes recorded no unanswered pairs. These are stored + ranking measurements, not fresh detection/precision results or a pre-push latency demonstration. +- **Equal-score ties are not the whole remaining problem.** At fixed scores, perfect tie-breaking + still needs at least **68** regions for PMPro and **86** for WP Statistics (67/85 strictly + higher-scored same-tier regions precede the targets). An LLM restricted to equal-score ties + cannot pass `<50`. Within-tier reordering across scores or a new feature is a distinct experiment. +- Docker, Compose and provisioning still default to 4.0.623; the latest measurements use 4.0.625. + Align the shipped baseline before claiming deployment parity. New files restored by the engine + upgrade are legitimate coverage, not a ranker regression to tune away. +- Ordinary CLI scans do not enable `ScanBudget.order_by_evidence`; preserve the experimental status + until the chosen acceptance profile passes. Record mode explicitly in every future comparison. +- `position.py` currently equates an unmatched target function with `reached_transitively` and + omits it from `budget_at_recall`. Require a queried witness for transitive detection; otherwise + keep the target unresolved in the denominator. This currently affects VAmPI's SQLI accounting. +- Graph construction precedes region selection. Ranking improves query allocation but does not + itself make graph builds incremental. The push design must measure both costs. + +### Revised experimental contract (draft, awaiting approval) + +**Maintainer clarification:** PMPro remains the chosen core-optimization example. Require a +JavaScript/Node API transfer workload, especially Express routes/middleware, to test that those +optimizations generalize. Frontend/framework facts map concrete semantics to common evidence; +core ranking and scope must not gain PHP-specific branches or language-specific score bonuses. +Equivalent normalized evidence must produce equivalent tier/score decisions. See +`../pre-push-safety-net/evaluation.md` for the PMPro/Node/Python/C workload roles, proposed NodeGoat +integration candidate, and separate untouched Node evaluation. C/C++ is valuable for checking +shared machinery and abstraction limits; libpng's envelope is not a positive detection result. + +1. Freeze engine, facts, layout/scope, ranker mode and workload before comparing an ordering change. + Record selected/deferred question IDs and unanswered queries. Separate engine and scope changes + from ranking improvements; vendor exclusion is an explicit boundary change. +2. Keep `budget_at_recall` as a scheduling diagnostic, not the product gate. Measure actual + vulnerable/fixed outcomes, actionable-alert precision, benign-push interruption, supported recall, + completed coverage, and total time on the same workload. No unanswered target disappears. +3. Keep the repeatedly inspected PHP plugins as regression workloads. Use untouched repositories + and benign/security-changing pushes for new evaluations, with per-language/framework/family + outcomes. Broader C memory reasoning is not solved by ordering taint questions better. +4. Before another LLM iteration, distinguish changing equal-score ties from reordering a whole tier. + Compare the candidate with the frozen heuristic at equal total cost, including model latency. + Protect obligation/configuration work: a taint-only score must not silently starve those families. +5. Include a recorded exploration allocation and sufficiently small fully evaluated workloads in + the first learned-ranking experiment, rather than deferring selection-bias measurement until + after phase 3. Use a recorded seed; report unresolved/unexamined population and sampling limits. +6. Ranking can decide where to look, never whether to alert. The hook's admission policy requires + supported change attribution, a useful witness and repair direction even for an entailed result. + Advisory mode must meet that same bar. No model may change facts, scope exclusions or rungs. + +This amendment does not approve the brief or its next phases. Reconcile requirements/design before +generating implementation tasks; preserve prior measured iterations as evidence rather than relabeling +them approved specifications. + ## Why now > "we need a taint flow aware ranker, that is capable to put high risk areas into tiers" diff --git a/.kiro/specs/flow-aware-ranking/spec.json b/.kiro/specs/flow-aware-ranking/spec.json index b4ec0ae..fc9e430 100644 --- a/.kiro/specs/flow-aware-ranking/spec.json +++ b/.kiro/specs/flow-aware-ranking/spec.json @@ -1,7 +1,16 @@ { "name": "flow-aware-ranking", + "language": "en", "status": "brief", "approved": false, + "updated_at": "2026-09-12T15:35:00Z", + "related_specs": [ + "pre-push-safety-net" + ], + "amendment_note": "2026-09-12: preserve the ranker as scope mechanism and vendor exclusion from targets/graphs; correct tie-only and benchmark-recall interpretations; place ranking inside a broader actionable pre-push safety net. Brief amendment remains unapproved; no historical approvals inferred.", "summary": "Rank (region, family) pairs by cheap evidence of a possible taint flow, so an expensive query is spent where it can pay.", - "depends_on": ["contributor-scan"] + "depends_on": [ + "contributor-scan" + ], + "cross_language_note": "2026-09-12: maintainer endorsed the direction and clarified PMPro as core development case, JavaScript/Express Node APIs as required transfer workload, and C/C++ as additional shared-machinery/abstraction-limit evidence. Preserve language-agnostic core decisions; framework facts may differ. Detailed amended requirements/design remain pending review." } diff --git a/.kiro/specs/pre-push-safety-net/brief.md b/.kiro/specs/pre-push-safety-net/brief.md new file mode 100644 index 0000000..d5e60c5 --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/brief.md @@ -0,0 +1,75 @@ +# Brief: pre-push-safety-net + +Status: requirements, design and tasks approved on 2026-09-12; group 1 implementation started. Language: English. + +## Problem + +AI increases the rate of code changes. Developers need a bounded pre-push check that catches +actionable security regressions without a recurring queue of speculative SAST warnings. The +maintainer explicitly rejects noisy advisory output as well as noisy blocking output. + +## Current State + +The scan driver and Joern arbiters exist, with demonstrated PHP/Python cases. There is no pre-push +snapshot resolver, change context for the ranker, persistent graph cache, or end-to-end deadline. +The driver builds a whole-root graph before cutting the region budget. The latest PMPro ranking +experiment alone spends about 130 seconds building and 296 seconds extracting evidence. A manually +selected two-file slice finding a CVE does not demonstrate automatic incremental analysis. + +## Desired Outcome + +A developer pushes committed code and receives either a concise actionable security finding or a +bounded result with explicit coverage. Unrelated working-tree changes, backlog findings, and model +speculation do not become push warnings. The same analysis can be invoked explicitly for diagnosis. + +## Approach + +Resolve pushed snapshots, feed changed and affected first-party context to the existing ranker, +reuse compatible results, and run admitted detector capabilities within a total deadline. The ranker +remains the scope mechanism. Exclude third-party vendor code from targets AND graphs as already decided. +Compare relevant base/head evidence +before calling a defect new. Apply a separate actionability policy after arbitration. Evaluate on +security and benign changes across repositories/frameworks, with cold and warm caches separated. + +## Scope + +- In: resulting pushed tips, affected context, caching, bounded execution, change-related actionable + alerts, coverage status, optional enforcement, evaluation and reproducible local artifacts. +- Out: every historical commit introduced by a push, whole-repository certification, style rules, + dependency-vulnerability inventory, automatic source edits, mandatory cloud/LLM services, + prompt optimization, and new memory-safety abstractions. + +## Boundary Candidates + +One product boundary owns the pre-push transaction from Git input to user outcome. It consumes the +existing ranker's scope and analysis engine through explicit scope/budget/results contracts. Reusable ranking and +family semantics remain with their existing specs rather than being reimplemented in the hook. + +## Upstream / Downstream + +- Upstream: `contributor-scan` engine/packaging/family correctness; existing facts and rungs. +- Adjacent: `flow-aware-ranking` schedules work; `zero-with-a-reason` diagnoses missing work; + `finding-feedback-loop` owns durable human review and later learning. +- Downstream: developer pre-push workflow and explicit local/CI replay of the same snapshot contract. + +## Constraints and Initial Evaluation Targets + +- User direction: actionability and low noise are mandatory in every mode. +- Proposed rollout: advisory first; blocking is explicit opt-in after per-capability validation. +- Proposed latency experiment: 30-second warm-push p95 on a declared reference workload/hardware, + with a configurable deadline and 2-second cancellation/report allowance. This is an unproven + design target, not a demonstrated SLA. Cold runs use the same deadline and disclose incompleteness. +- No promise of all-language coverage; capabilities are admitted per language/framework/family. +- Initial work must establish feasibility and noise before investing in an LLM ranker. + +## Cross-language amendment — 2026-09-12 + +Keep PMPro as the primary worked optimization example, and require a JavaScript/Express/Node API +transfer experiment before claiming the core improvement generalizes. Abstract evidence, ranking, +scope and arbitration contracts remain shared. Language/frontend/framework facts supply concrete +semantics; missing semantics remain a stated limit. Do not add PHP-specific core ranking shortcuts. + +See `evaluation.md` for the proposed workload matrix: NodeGoat as an Express wiring/teaching +candidate, a separate untouched Node API repository, existing Python regressions, and libpng as +a C envelope/limit control. C/C++ detection is measured only for a labeled property the chosen +abstraction can decide; bounds reasoning is not inferred from successful taint analysis. diff --git a/.kiro/specs/pre-push-safety-net/design.md b/.kiro/specs/pre-push-safety-net/design.md new file mode 100644 index 0000000..27632da --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/design.md @@ -0,0 +1,339 @@ +# Design: pre-push-safety-net + +Status: approved for task generation on 2026-09-12 following the maintainer's "ok, next kiro phase". +Requirements and tasks are approved; group 1 implementation was authorized by "approve." on +2026-09-12. Runtime and latency claims remain unverified. + +## Overview + +The hook is a bounded transaction around the existing ranker and detector. It analyzes immutable +pushed tips, supplies change context to the ranker, and admits only actionable security regressions +to the terminal. The ranker remains the scope mechanism. Third-party vendor code stays outside +targets and graphs. WordPress is one regression workload, not the architecture's organizing unit. + +The design separates three decisions: the ranker selects questions, the arbiter establishes modeled +claims, and hook policy decides whether a supported change-related claim merits an interruption. +An entailed rung is necessary for initial admission but is not sufficient by itself: a wrong fact, +an irrelevant old defect, or an unactionable witness can still produce noise. + +### Goals and Non-Goals + +- Deliver specific security findings with witness, change relationship, location, and repair direction. +- Bound the entire push and retain exact coverage; reuse matching analysis work without stale verdicts. +- Measure recall, noise, coverage and latency together across supported languages/frameworks. +- Do not build a second ranker/scope heuristic, an incremental CPG engine, a daemon, an automatic fixer, + or a mandatory model service. Do not scan intermediate pushed history in this feature. + +## Boundary Commitments + +### This Spec Owns + +- Git push inputs, immutable snapshots, comparison identity, and change context passed to the ranker. +- Local artifact reuse, the shared push deadline, delta attribution and hook alert admission. +- Concise rendering, explicit finding/coverage/push dispositions, and integration instructions. + +### Out of Boundary + +- Ranking features, weights, tier definitions, and exploration: `flow-aware-ranking` owns these. +- Family facts, sanitizers, graph construction semantics and generic driver correctness: + `contributor-scan` owns these. Hook integration may extend contracts but may not fork the driver. +- Broader telemetry mechanisms: `zero-with-a-reason`; durable dismissals and learning: + `finding-feedback-loop`. Initial rollout does not require either to be complete. +- Vendor source analysis, new C memory abstractions, dependency CVE inventories, whole-tree assurance. + +### Allowed Dependencies + +Use Git, Python's standard library, `preprocess`, `mapping`, `model/regions`, `model/layout`, +`model/scan`, `model/evidence`, `cpg/backend`, existing arbiters, redaction and report records. +No new mandatory package or service. Dependency direction is existing engine/types -> push adapters +and policy -> push runner -> CLI. The engine never imports the push package; shared deadline and +artifact contracts live at the engine boundary. Ranker internals never depend on Git hook objects. + +### Revalidation Triggers + +Changes to engine/frontend/overlay versions, facts/sanitizers, layout, scope rules, ranker mode, +query depth, deduplication, finding identity, or alert admission invalidate relevant cached results +and require the security/fixed/benign push suite. Ranker changes also require untouched evaluation. +Driver contract changes revalidate ordinary `ousast scan`, coverage reports, and repository benchmarks. + +## Architecture + +### Existing Architecture Analysis + +`preprocess_repository` reads the live filesystem and labels it with HEAD; this is not a pushed +snapshot. `scan_repository` builds a full-root CPG before selecting regions. `ScanBudget` limits +regions/model calls, not total elapsed time. Backend graphs are temporary, with per-command timeouts +and cleanup. `order_by_evidence` is implemented but disabled by the ordinary CLI caller. + +Consequently the current ranker does not bound graph-build cost. A faster ordering alone cannot +establish a fast pre-push hook, and passing fewer regions must not be described as incremental parsing. + +```mermaid +flowchart TD + A[Git ref updates] --> B[Immutable tips and comparison context] + B --> C[First-party inputs with vendor exclusions] + C --> D[Compatible graph or bounded graph build] + B --> E[Changed and affected context] + D --> F[Existing ranker selects scope] + E --> F + F --> G[Existing arbiters and targeted base comparison] + G --> H[Change attribution and actionability admission] + H --> I[Compact hook result and detailed artifact] + D --> J[Coverage and deadline accounting] + F --> J + G --> J + J --> I +``` + +The deadline surrounds the entire flow. Cache lookup does not bypass identity checks; a miss does +not trigger unbounded work. The default path is local and deterministic. An optional LLM may reorder +within allowed ranker constraints or explain existing evidence; it cannot mint facts or admit alerts. + +### Language and Framework Contract + +Requirements 8.1, 8.2, 8.3, 8.4 make abstract analysis a transfer contract, not a claim of automatic +language coverage. Frontends and versioned facts identify sources, sink argument roles, carried +values, dispatch, guards, obligations and configuration properties. The common evidence vector, +ranker scope decisions and arbiters consume these representations. Core ranking must not inspect +repository names, PHP syntax, WordPress calls or Express API names to assign priority. Equivalent +normalized evidence receives the same core tier and score regardless of origin; policy admission +also requires the independently evaluated capability for that origin. + +Existing seams are `mapping.py` (route/middleware mapping), `ruleset/semantic/*.toml` and +`ruleset/obligations/*.toml` (reviewed language/framework facts), `model/specs.py` (abstract question +construction), `model/evidence.py` (shared ranking), and `cpg/queries/` (graph queries). Reuse them; +do not add a universal source IR or move framework checks into the hook. Frontend normalization +belongs at the backend/query boundary and must document unsupported constructs. Existing +language-specific integrations require audit; this design does not claim they already conform. + +For Express, verify route registration and ordered middleware, request input, imported handlers, +service calls, callbacks and async boundaries. A function named `authenticate` does not establish +that its guard governs an operation. Require the same guard/obligation evidence as in other +frameworks. `node_modules` and other declared third-party trees stay outside targets AND graphs; +explicit framework/library summaries describe relevant behavior without analyzing vendor source. +Do not exclude first-party JavaScript merely because it is browser code: record its partition and +capability separately from the server. TypeScript support also needs frontend evidence, not an +assumption based on normalizing its language label to JavaScript. + +PMPro is the development workload; Node/Express checks transfer of the frozen core. Framework/fact +adaptations are recorded separately from zero-change transfer, and any case used for repairs becomes +regression data. C and C++ use the same scheduling/reporting contract but may need different abstract +domains: taint transfer, guard dominance and numeric bounds are distinct capability claims. Keep +libpng as an envelope/unsupported-bounds control until a suitable domain is implemented and measured. +Concrete workload roles and required paired changes are in `evaluation.md`. + +### Technology Stack + +| Layer | Choice | Role | +|---|---|---| +| Push input | Installed Git, object-format-aware commands | Resolve refs, trees and immutable blobs | +| Runtime | Existing Python >=3.11 CLI and standard library | Transaction, cache, deadline, policy | +| Analysis | Existing Joern backend; evaluated baseline 4.0.625 | Graph and arbiters, version recorded | +| Storage | Local content-addressed directories and JSON manifests | Atomic reusable artifacts, no database | +| Presentation | Existing finding records plus compact hook renderer | Separate findings and coverage | + +Align Docker, Compose and provisioning with the evaluated Joern version before any deployment +claim. The old defaults are 4.0.623. Pin alignment is predecessor packaging work, not completed here. + +## File Structure Plan + +New files (proposed, not created by this documentation change): + +| Path | Responsibility | +|---|---| +| `src/openultrasast/push/snapshot.py` | Parse ref updates, resolve bases, materialize blobs, produce change context | +| `src/openultrasast/push/cache.py` | Artifact keys, validity, atomic publication and eviction | +| `src/openultrasast/push/policy.py` | Delta identity, actionability admission and push disposition | +| `src/openultrasast/push/runner.py` | Shared deadline, ranker/engine orchestration, per-ref result aggregation | +| `src/openultrasast/push/report.py` | Compact terminal result and reproducible push manifest | +| `benchmarks/push/README.md` and `benchmarks/push/measure.py` | Labeled push recipes and joint quality/latency measurement | +| `tests/test_push_snapshot.py`, `tests/test_push_cache.py`, `tests/test_push_policy.py`, `tests/test_push_runner.py` | Contract and integration checks below | + +Modified files: `cli.py` and `config.py` add explicit push command/configuration; +`model/regions.py` carries change context without Git-specific logic; `model/scan.py` consumes the +ranker's selected questions and returns exact selected/deferred IDs, deadline and reusable-graph +contracts; `cpg/backend.py` owns graph artifact validation and process cancellation. Existing +`model/evidence.py` changes only through the ranking spec. Packaging pin fixes remain in existing +Docker/Compose/ops paths. `README.md` documents opt-in integration without overwriting existing hooks. + +Cross-language validation reuses `tests/test_model_evidence.py` for equivalent normalized evidence, +`tests/test_model_scan.py` for shared scope/coverage behavior, and existing mapping/fact tests for +frontend adaptation. Reviewed workload recipes use `benchmarks/repos/*.toml` and the proposed +`benchmarks/push/` harness. Necessary mapping/fact/query fixes stay within the engine/ranking specs; +they do not become JavaScript-specific branches in `push/`. + +## Components and Interfaces + +### Snapshot Adapter + +`PushUpdate(local_ref: str, local_oid: str, remote_ref: str, remote_oid: str)` records every input +line. `PushComparison(head_oid: str, base_oid: str | None, base_reason: str, refs: tuple[str, ...])` +records immutable identity. Object IDs are validated through Git; no hard-coded SHA-1 length. + +For existing refs, compare the supplied old and new tips, including force-pushes. For a new branch, +use a configured comparison ref resolved to an OID and its unique merge base with the pushed tip; +record that this is a chosen baseline, not the previous remote branch. No configured/unique/local +base means comparison incomplete. Do not substitute HEAD or a remote-tracking ref silently. No +automatic fetch inside the hook. Peel annotated tags to commits; other objects are unsupported. +Deletion updates require no target scan. Deduplicate identical target graphs, but retain separate +comparisons when the same target has different bases. Multiple refs share one deadline. + +Materialize tracked blobs directly into isolated scratch storage; do not checkout over the user's +tree or run filters/build scripts. Use NUL-delimited tree/diff records and argument arrays. Include +deletions as impact seeds and retain rename attribution. Symlinks are not followed outside the +snapshot; gitlinks, absent LFS content and unsupported objects produce explicit coverage limits. +Target-tree content is the analysis population; intermediate history is out of scope. + +### Ranker Scope Contract + +The snapshot adapter provides `ChangeContext`: base/head identities, changed/deleted paths and spans, +manifest/config changes, and known affected first-party relationships. It supplies evidence, not +an alternative inclusion policy. The existing ranker produces `ScopeDecision`: selected question +IDs, their priority/evidence, deferred question IDs with reasons, and unresolved context boundaries. +Identifiers include unit/language/path/function/family to avoid collisions across partitions. + +Third-party vendor exclusions apply before both preprocessing targets and graph construction. +Tests remain available but unshipped, consistent with the approved layout decision. The ranker +cannot reintroduce vendor sources. Known external summaries are versioned facts; without a relevant +summary, a claim depending on excluded internals is unresolved. Do not assume a missing external +call is safe. Entry/callee, field, hook and removed-guard context must be represented in the ranked +question; an unchanged sink may be affected by changed reachability or sanitization. + +Graph construction and question scope are distinct costs. Initial graph input is the first-party +repository per supported frontend, or an explicitly declared analysis unit with recorded boundaries. +Do not create an additional heuristic source-slicer. A mixed-language repository has explicit +frontend partitions and coverage per partition; unsupported cross-language edges remain unresolved. +Use old graph relationships only as prioritization hints after a code change, never as current proof. +If a ranked question's necessary first-party context is absent, expand its declared unit within the +remaining budget or report it unresolved. No claim of exact tier-0 pruning extends beyond the +modeled graph/facts/scope; unanswered evidence is never tier 0. + +### Cache and Engine Boundary + +Graph key: digest of every included blob/path and applicable declarations, frontend/engine version, +overlay semantics/options, language, and exclusions. Query key additionally includes graph key, +question/context/depth, facts/sanitizers, query code and relevant analysis configuration. Result key +also includes ranking/admission versions and any model/prompt identity. Comparison results include +both base and head identities. Persist manifests with source byte counts and graph census. + +Only validated, complete artifacts can satisfy complete-result lookups. Partial artifacts may be +retained for diagnostics but never promoted by cache presence. Publish atomically from private +temporary paths under a per-key lock; another push may wait only within its deadline. Apply a +configured cache size limit and evict unleased entries. No live graph mutation or dependence on +CPG node IDs remaining stable across builds. A changed input invalidates that graph; incremental +Joern mutation is neither assumed nor promised. Unchanged declared units and identical snapshots +can reuse graphs; changed units rebuild. Measure whether this is enough before claiming scale. + +The maintainer confirmed interest in persistence followed by differential updates on 2026-09-12. +Persistence is the first optimization; differential graph mutation remains a separate future +experiment. Its acceptance gate must compare against a cold rebuild on identical inputs and +semantics, including changed/deleted/renamed declarations, affected call edges and dataflow +overlays, and invalidation of dependent query/results artifacts. Updating only changed file nodes +is insufficient. No graph partition may claim complete cross-partition analysis without validated +relationship handling. This sequencing does not expand the approved implementation into a new +incremental engine. + +The backend owns `GraphArtifact` validation/lifecycle. The generic driver accepts an optional +validated graph lease and an absolute monotonic deadline, preserving its ordinary fresh-build path. +Queries operate on an isolated lease if loading can modify a graph. Every subprocess timeout is +bounded by remaining transaction time, including fallback frontend, overlay, shard and retry work. +Cancel process groups, close leases, then persist coverage within the cancellation allowance. + +### Delta and Actionability Policy + +Admission requires all of: an enabled/evaluated capability, `model_entailed` or a future actually +produced `execution_confirmed` result, a locatable witness, applicable family semantics, no unresolved +dependency that undermines the claim, supported change attribution, and a concrete repair direction. +No production caller currently reaches `execution_confirmed`; it is not a prerequisite or a promise. + +Findings are keyed by family/mechanism and stable source/operation identity, with rename/span mapping; +line numbers alone do not establish novelty. Re-query relevant base counterparts under the same +semantics and sufficient comparable context. A missing old finding from a truncated/failed scan is +unknown, not proof that the head introduced a bug. A removed guard or newly connected source can +establish a worsened flow at an unchanged sink. Unknown novelty stays in the diagnostic artifact +with one aggregate comparison-coverage notice. Initial checkout is not automatic acceptance of debt. + +Deduplicate by defect, retaining distinct witnesses and ref associations. Explain input/operation, +security consequence, why this change matters and a specific repair locus; do not require automatic +patch generation. Templates grounded in facts/witnesses are sufficient. Admission rejects generic +"sanitize input" advice without a supported context/operation. An LLM explanation must cite the +same witness and cannot raise the rung or invent exploitability. + +### Result and Hook Adapter + +`PushResult` separates `finding_status` (`actionable` / `none`), `coverage_status` (`complete_within_scope` +/ `incomplete` / `unavailable` / `not_applicable`) and `push_disposition` (`allow` / `block`). Zero +selected questions cannot mean complete when applicable questions were deferred or context is unknown. +The detailed manifest retains per-ref/per-capability coverage, supported and unsupported populations, +all admitted defects, rejected-candidate reasons, exact scope and timings. + +Initial profile default: advisory, with the same admission policy as blocking. Blocking is opt-in and +returns nonzero only for admitted new/worsened findings; incomplete coverage defaults to allow plus +a notice, with a separately configurable strict coverage policy. Show up to three concise defect +summaries plus the remaining count/artifact path; the cap never changes the exit decision. Do not +render the full SAST backlog. No actionable findings: at most one bounded status line. Incomplete: +one notice naming the primary reason and a concrete retry/diagnostic action. Unchanged results may +be reused, but a still-relevant admitted defect is not dismissed merely because it was printed before. + +Expose a proposed `ousast pre-push` command receiving Git's stdin and remote arguments, plus explicit +base/head replay. Hook installation is explicit: preserve `core.hooksPath`, existing hook exit +semantics, and stdin for other consumers. Provide an integration snippet and refuse automatic +overwrite of an existing hook. Analysis runs no project code and performs no source modification. + +## Requirements Traceability + +| Requirements | Design elements | +|---|---| +| 1.1, 1.2, 1.3, 1.4 | Snapshot Adapter: immutable objects, per-ref comparisons, special update cases | +| 2.1, 2.2, 2.3, 2.4 | Ranker Scope Contract: change context, vendor exclusion, boundary/guard handling | +| 3.1, 3.2, 3.3, 3.4, 3.5 | Delta and Actionability Policy: admission, witnesses, comparison and deduplication | +| 4.1, 4.2, 4.3, 4.4 | Cache and Engine Boundary: shared deadline, complete artifact identity and leases | +| 5.1, 5.2, 5.3, 5.4, 5.5 | Result and Hook Adapter: separate statuses, compact rendering, explicit policies | +| 6.1, 6.2, 6.3, 6.4, 6.5 | Testing Strategy and Rollout: capability admission and joint gates | +| 7.1, 7.2, 7.3, 7.4 | Local runtime, model redaction/budget, manifests, non-destructive integration | +| 8.1, 8.2, 8.3, 8.4 | Language and Framework Contract; frozen-core transfer and workload matrix in evaluation.md | + +## Error Handling + +Unreadable input or failed census invalidates the affected graph. Missing base invalidates novelty +comparison. Missing context invalidates affected checks. Deadline exhaustion preserves completed +findings and records deferred work. These states remain visible even when the hook permits the push. +If result persistence fails, print one artifact-write notice and retain the real finding/coverage +decision in the terminal; do not fabricate a successful cache entry or hang while retrying. + +## Testing Strategy and Rollout + +1. Snapshot contracts (1.1–1.4, 7.1, 7.4): dirty trees, non-HEAD pushes, multiple refs with different + bases, force-push/new-branch/missing-object cases, tags, deletion-only pushes, unusual paths, + symlinks/gitlinks, and coexistence with an existing hook. +2. Ranker/context tests (2.1–2.4): handler-to-unchanged-sink, field/hook flows, removed guards, + first-party cross-file context, vendor source absent from both graph and targets, unresolved + vendor boundary, polyglot partition coverage. No second selection heuristic in the push adapter. +3. Admission tests (3.1–3.5, 5.1–5.5): high rung with wrong/unsupported semantics is rejected; + unchanged backlog and missing-base claims stay out; new supported defects include actionable + witnesses; one defect reached by many regions prints once; advisory and blocking share admission. +4. Deadline/cache tests (4.1–4.4, 7.3): source/context/fact/query/version invalidation, corrupt/partial + entries, concurrent pushes, timeout during build/overlay/query/retry, process-group termination, + no repeated per-ref budget reset, exact selected/deferred IDs rather than slicing old order. +5. Real push evaluation (6.1–6.5, 8.1–8.4): use PMPro for core development, then freeze the core + and replay labeled introducing/fixing/benign changes on a JavaScript/Express Node API workload. + Preserve Python regressions and a separate untouched Node API repository; training applications + and cases used to fix mapping are not fresh holdouts. Include injection, path and authorization/ + configuration where supported, plus middleware and cross-module/async controls. Test equivalent + normalized evidence for identical core decisions. Keep libpng's envelope and unsupported bounds + separate from a future C/C++ positive control. See `evaluation.md` for selection and labeling gates. + +Measure the accepted experimental acceptance profile in requirements on declared hardware and fixed workloads. +Warm means prior reusable work from the preceding state, not only repeating the identical tip. +Report identical-tip hits separately from a one-function change and a dependency/configuration +change. Include first installs, cold graphs, growing repository size, multi-ref pushes and incomplete +outcomes. Freeze the applicable population before ranker selection; deferred questions count against +completion. Report p50/p95 plus timeout counts, observed precision with confidence intervals, recall, +benign interruption rate and fixed-side silence. A low-noise score with low recall/coverage fails. + +Rollout order: reconcile engine pins and benchmark honesty; prove one end-to-end change/witness/fix +with the existing ranker; establish deadline/cache feasibility; run untouched and benign evaluation; +admit passing capabilities in advisory mode; consider opt-in blocking only after these gates pass. +LLM ranking is a later experiment, not a prerequisite. If realistic changed-unit rebuilds cannot +meet the deadline/coverage gates, keep the hook experimental and revisit the graph strategy with +measurements; do not solve the miss by silently shrinking context or lowering alert standards. diff --git a/.kiro/specs/pre-push-safety-net/evaluation.md b/.kiro/specs/pre-push-safety-net/evaluation.md new file mode 100644 index 0000000..cd279ff --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/evaluation.md @@ -0,0 +1,82 @@ +# Evaluation: core optimization and cross-language transfer + +Status: evaluation contract accepted with requirements/design for task generation, 2026-09-12. Implements requirements 6.1–6.5 and 8.1–8.4. +No new repository has been pinned, scanned or admitted by this document. + +## Workload roles + +| Workload | Role | Required evidence | +|---|---|---| +| Existing pinned PMPro | Primary development and cost-attribution example | Known CVE retained; fixed/benign controls; total build/evidence/query cost | +| Existing WP Statistics and MW WP Form | PHP structural regressions | Hook/field flows preserved; vendor exclusion does not become a ranking trick | +| JavaScript/Express Node API | Required transfer and hook integration workload | Same frozen core; route/middleware/handler/service context; vulnerable/fixed/benign changes | +| Separate untouched Node API repository | Generalization evaluation | Predeclared labels/population; no tuning on its results; all misses and gaps retained | +| Existing pinned VAmPI | Python and authorization regression | Actual transitive SQLI witness plus obligation findings; no omitted position targets | +| Existing pinned libpng | C graph/cost and capability-limit control | Envelope measured; unsupported arithmetic/bounds explicitly reported | +| Future C or C++ positive control | Additional abstract-property transfer | Reviewed vuln/fix for a supported property; C++ frontend behavior verified independently | + +## Recommended JavaScript starting point + +**OWASP NodeGoat is the proposed first Express wiring/teaching candidate.** It is a deliberately +vulnerable Node application with remediation teaching material, and its package manifest declares +Express. It exercises server-side code without making an entire browser application the primary +measurement unit. [Project](https://github.com/OWASP/NodeGoat), +[package manifest](https://github.com/OWASP/NodeGoat/blob/master/package.json). + +This is candidate selection, not detection evidence. Before adding an executable recipe, pin an +immutable commit, verify readable source/license bytes, identify the exact affected operation and +its caller context, and review the vulnerable and fixed forms. An authored repair must be labeled +as authored; do not present it as an upstream CVE fix. Do not assume the tutorial's vulnerability +categories correspond to supported arbiters: a MongoDB operator-injection case is not automatically +covered by SQL string-taint facts. Select a supported positive control and retain other cases as gaps. + +NodeGoat is intentionally educational and will be inspected during integration. It cannot establish +production generalization on its own. Its inspected manifest uses an older Express 4 dependency +range; it is not evidence of coverage for current module/async/framework patterns. Require those +patterns in the additional Node workload and record its exact versions. Reserve an independently selected Node API repository with +reviewable security-fix history before the transfer result is used to tune the core. Selection must +not depend on getting a favorable scanner score. No npm installation or project execution is needed +to establish the static input; runtime controls, when needed, belong in a separate controlled harness. + +## Paired changes and negative controls + +1. Request input crosses route -> imported handler -> service -> supported dangerous operation; + a correct family-specific repair stops the finding on the fixed side. +2. A removed authorization/ownership guard exposes an unchanged operation. A guard that is present + but does not govern the operation must not discharge it; correctly ordered governing middleware must. +3. Callback/async data transfer preserves a supported source-to-operation witness. Unresolved dynamic + calls are counted as a context gap, not silently treated as no flow. +4. Input validation for an unrelated context does not cleanse SQL/command/path operations. Path + normalization alone is not assumed to establish confinement to an allowed root. Test both the + real repair and a plausible but insufficient repair. +5. Benign refactoring, renaming, formatting, parameter binding and unaffected backlog do not generate + new hook alerts. Retain benign push counts so low noise cannot be inferred only from security cases. +6. `node_modules` and other declared vendor files are absent from both graph and targets, with + first-party callers still represented. Missing external semantics remain visible coverage limits. + +Do not force all these shapes into one repository. Keep authored cross-language contract tests +separate from reviewed repository changes, and preserve their provenance in the scorecard. + +## Freeze, transfer, classify + +1. Freeze core ranker features/weights, arbiter/query semantics, engine version, budget, and existing + framework facts after the PMPro experiment. Run Node with available adapters first; save that result. +2. Classify each miss before changing anything: unread input/instrument failure, frontend normalization, + missing framework fact, ranker scope/budget, wrong abstract semantics, or unsupported abstract domain. +3. If a framework fact or frontend adaptation is needed, review it separately and rerun both languages. + Report adapted transfer separately from the unchanged-core result. A core change starts a new + version and requires a fresh untouched evaluation; it is not hidden in a "JavaScript config" label. +4. Compare actual findings, fixed-side silence, actionable precision, benign interruptions, coverage, + and elapsed cost together. Equivalent normalized evidence must produce identical core tier/score + results; equivalent syntax across languages is not required and is not presumed. +5. Do not generalize a C envelope result to C/C++ detection. Select a positive control only after its + property fits a declared arbiter; numerical bounds/overflow needs a suitable domain and separate + validation. Do not expand taint sources or sanitizer names to simulate arithmetic reasoning. + +## First bounded implementation experiment + +Replay one supported PMPro change/fix and one Node/Express change/fix through the same ranker, +graph exclusion policy, deadline and admission contract. Add at least one benign change per workload. +Produce one joint scorecard before any further core optimization. A missing Node frontend, broken +middleware mapping or wrong sanitizer model is an explicit finding of this experiment, not a reason +to remove Node from the evaluation or silently fall back to a PHP-only release claim. diff --git a/.kiro/specs/pre-push-safety-net/requirements.md b/.kiro/specs/pre-push-safety-net/requirements.md new file mode 100644 index 0000000..4894e7a --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/requirements.md @@ -0,0 +1,159 @@ +# Requirements: pre-push-safety-net + +Status: approved for task generation on 2026-09-12 following the maintainer's "ok, next kiro phase". The acceptance profile defines evaluation targets, not measured achievements. + +## Introduction and Boundary + +OpenUltraSAST helps developers catch concrete security regressions in the resulting commits they +push. Advisory findings must be actionable; nonblocking speculation is still noise. The hook owns +the push transaction and its report, using the existing analysis engine. It does not certify the +repository, scan every intermediate commit, add new detector semantics, or automatically fix code. + +## Requirement 1: Analyze the code being pushed + +1.1 When a push contains ref updates, the Safety Net shall analyze each distinct resulting commit +snapshot and retain its association with every supplied ref update. + +1.2 While analysis runs, the Safety Net shall leave the working tree, index, refs, and existing hook +behavior unchanged and shall exclude uncommitted content from the analyzed snapshot. + +1.3 If an update deletes a ref, targets an unsupported object, or lacks a usable comparison base, +the Safety Net shall record that disposition without presenting an empty comparison as a successful check. + +1.4 When a new branch, merge, force-push, rename, or deletion changes the comparison, the Safety Net +shall record the exact compared revisions and the rule used to choose them. + +## Requirement 2: Analyze affected security behavior + +2.1 When code changes, the Safety Net shall supply changed sites and affected first-party caller, +callee, data, guard, dispatch, and configuration context to the ranker, which shall remain the +mechanism selecting the regions and questions to analyze. + +2.2 If a dependency cannot be resolved or included within the budget, the Safety Net shall identify +the affected checks as incomplete rather than treating omitted context as safe. + +2.3 When constructing analysis targets or graphs, the Safety Net shall exclude third-party vendor +code and record those boundaries, along with any test, generated-file, submodule, or language exclusions +and their effect on supported coverage; missing vendor semantics shall require an explicit summary +or an unresolved result rather than reintroducing vendor code. + +2.4 When a guard or sanitizer is removed, the Safety Net shall evaluate affected unchanged operations +where supported rather than requiring the vulnerable operation itself to occur on an added line. + +## Requirement 3: Produce actionable security alerts + +3.1 When the Safety Net emits a normal hook alert, it shall include a supported security consequence, +an exact location, a witness connecting the relevant input or obligation to the operation, a supported +relationship to the proposed change, and a concrete repair direction. + +3.2 If a candidate is only suspicion, unsupported intent inference, a style concern, or an +uncalibrated capability result, the Safety Net shall omit it from normal hook alerts and retain its +disposition in an opt-in diagnostic artifact. + +3.3 When multiple regions identify the same defect, the Safety Net shall emit one defect summary +with its relevant locations rather than repeated warnings. + +3.4 If an existing defect is unchanged by the push, the Safety Net shall omit it from normal hook +alerts; if evidence cannot establish whether a defect is new or worsened, it shall report that +comparison gap as coverage information without claiming a new vulnerability. + +3.5 Where a model assists ranking or explanation, the Safety Net shall require the same alert +admission evidence as in a run without that model. + +## Requirement 4: Bound cost and reuse valid work + +4.1 When a check begins, the Safety Net shall apply one configurable elapsed-time budget across +all ref updates and stages, including preparation, retries, analysis, optional services, and reporting. + +4.2 If the deadline expires, the Safety Net shall cancel outstanding work, persist completed +evidence and incomplete coverage, and return within a separately measured cancellation allowance. + +4.3 When reusing an analysis result, the Safety Net shall verify that its input, relevant context, +engine, facts, query semantics, scope, and policy remain compatible with the requested check. + +4.4 If a cached artifact is partial, corrupt, incompatible, or unavailable, the Safety Net shall +recompute within the remaining budget or return incomplete coverage without reusing a clean verdict. + +## Requirement 5: Keep the push experience proportionate + +5.1 When no actionable defect is found in the completed supported checks, the Safety Net shall +emit at most one compact status line that describes the checked scope without calling the repository secure. + +5.2 If analysis is incomplete or unavailable, the Safety Net shall emit a separate compact operational +notice with one concrete next action and a detailed artifact reference, without a per-file warning flood. + +5.3 Where advisory mode is selected, the Safety Net shall allow the push while applying the full +actionability requirements to every displayed finding. + +5.4 Where blocking mode is explicitly enabled, the Safety Net shall block for admitted new or +worsened defects; an incomplete scan shall follow a separately declared coverage policy. + +5.5 When producing a result, the Safety Net shall preserve separate finding, coverage, and push +disposition fields so allowing a push cannot be interpreted as complete analysis. + +## Requirement 6: Earn admission through evaluation + +6.1 When evaluating admission of a detector capability to the hook, the project shall evaluate its language, +framework, and family on labeled vulnerable/fixed changes and benign pushes, including a repository +not used to develop or tune that capability. + +6.2 When reporting hook quality, the project shall report actionable-alert precision, recall on +supported labeled changes, fixed-side silence, benign-push interruption rate, coverage completion, +and cold/warm latency with sample counts, uncertainty, and all unresolved cases. + +6.3 If a target cannot be located, a query is unanswered, or a transitive path is unproven, the +evaluation shall retain that case in the declared population and identify the unresolved result. + +6.4 When comparing rankers, the evaluation shall hold scope, engine, facts, workload, and budget +constant, record the actual deployed ranking mode, and evaluate completed detections as well as rank. + +6.5 When evaluating release readiness of the hook, the project shall require passing predeclared precision, recall, +coverage, and latency gates together; a policy that achieves silence by examining or alerting on nothing +shall fail that evaluation. + +## Requirement 7: Preserve local control and reproducibility + +7.1 When running the default hook, the Safety Net shall use locally available analysis without +requiring a model endpoint, downloading an engine, fetching source, or running project build scripts. + +7.2 Where an external model is explicitly configured, the Safety Net shall apply the existing +redaction policy, record model/prompt provenance, and include its cost and latency in the budget. + +7.3 When saving an artifact, the Safety Net shall record input identities, scope and exclusions, +engine/fact/query/policy versions, exact selected and deferred questions, outcomes, and timings. + +7.4 When installing or removing the hook explicitly, the Safety Net shall preserve existing hooks +and document the integration and removal steps. + +## Requirement 8: Transfer the core analysis across languages + +8.1 When equivalent source, sink, guard, obligation and carried-value evidence is supplied from +different languages or frameworks, the Safety Net shall apply the same core scope, ranking and +admission rules without repository-specific or language-specific priority exceptions. + +8.2 When a frontend or framework cannot supply evidence needed for a supported check, the Safety Net +shall identify that capability gap rather than interpreting missing evidence as a safe program. + +8.3 When evaluating a core optimization developed on PMPro, the project shall freeze the core +ranking features, weights and decision rules and evaluate transfer on a JavaScript Node API workload +with Express-style routes and middleware, including vulnerable, fixed and benign changes. + +8.4 When publishing cross-language results, the project shall distinguish unchanged-core transfer, +frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or +unsupported-property results separately from demonstrated vulnerability detection. + +## Accepted experimental acceptance profile + +These values are the initial implementation/evaluation profile accepted for task generation, not +measured achievements or a demonstrated SLA. Freeze them before evaluating a candidate; report +failures without lowering the bar after inspecting holdout results. + +- Advisory first; blocking opt-in; incomplete coverage allows the push with one notice by default. +- Warm latency p95 <= 30 seconds, cancellation/report allowance <= 2 seconds; cold latency reported separately. +- >= 95% completed supported checks on the declared representative push workload. +- >= 95% observed actionable-alert precision with counts and confidence interval; zero known false + alerts on the fixed/benign regression suite. Small samples do not establish broad precision. +- >= 90% recall on the predeclared supported security-change suite; unsupported classes remain in + coverage reporting, not reclassified after a miss. Publish per-capability results, not only an aggregate. +- No capability is admitted with zero positive controls, zero reviewed emitted alerts, or no untouched + evaluation repository. Failed profiles remain experimental and silent in the normal hook. diff --git a/.kiro/specs/pre-push-safety-net/research.md b/.kiro/specs/pre-push-safety-net/research.md new file mode 100644 index 0000000..cd66c91 --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/research.md @@ -0,0 +1,142 @@ +# Research and design decisions: pre-push-safety-net + +## Summary + +- Date: 2026-09-12. Discovery scope: complex integration of the existing ranker and detector with Git. +- User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case. +- Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism; + third-party vendor code must remain outside targets and graphs. +- Requirements and design are proposals, not approved or implemented. No hook latency claim is made. + +## Research Log + +### Repository state and measurement limits + +Reviewed through `a8069ac`, including `60963f5` (overlay reuse/scoped seeds), `9f881b4` +(closure workaround bounded by upstream version), `60d13d7` (evidence ordering), `3b3b96b` +(parser-script readability probe), `b821ac6` (layout), and the latest ranking baselines. + +| Workload | Original position budget | 4.0.625 + layout budget | Build / evidence seconds | +|---|---:|---:|---:| +| PMPro | 391 | 82 | 129.9 / 295.7 | +| WP Statistics | 504 | 91 | 45.4 / 68.7 | +| MW WP Form | 513 | 54 | 45.2 / 80.3 | +| VAmPI | 16 | 16 | 5.6 / 18.4 | + +Source: `benchmarks/ranking/baselines/2026-09-11-evidence-rank-layout.json`, read and parsed during +the review. All four recorded `pairs_unanswered=0`; these are historical artifacts, not fresh runs. +WP Statistics's scope shrank after vendor removal. VAmPI's transitive SQLI target is excluded from +the position budget, so the value is not verified all-target recall. A 72-minute whole-PMPro scan +recorded in `flow-aware-ranking/brief.md` demonstrates a finishing scan, not hook feasibility or +precision of its 250 entailed findings. The focused review run passed 87 existing tests; it did not +rebuild Joern graphs or evaluate hook behavior. + +The latest ranker head has 67 strictly higher-scored same-tier regions ahead of PMPro's CVE and +85 ahead of WP Statistics's. Reordering equal scores alone cannot beat budgets 68 and 86. MW WP +Form has 39 strictly higher-scored regions, so its corresponding lower bound is 40. These are +arithmetic bounds from the stored head, not a measured new ranking algorithm. + +### Existing seams + +- `preprocess.py:preprocess_repository` reads filesystem bytes and then records HEAD; snapshot + identity must be established before calling this path for a push. +- `model/scan.py:scan_repository` builds a graph before applying `max_regions`. Its evidence order + is experimental and `cli.py` currently does not enable it. Region budget is not graph cost. +- `model/layout.py` supplies vendor exclusions and test/shipped classification. Preserve it. +- `cpg/backend.py` has process-group cancellation, census and parser readability checks, but fresh + temporary graphs and per-command rather than per-push deadlines. +- `cli.py` samples unjudged paths by slicing the original region order; the new contract must return + exact selected/deferred IDs from the ranker/driver instead of reconstructing them by count. +- `model/specs.py:taint_specs` still flattens sanitizers across a language. Contributor task 5.6 is + a prerequisite to admitting capabilities whose correctness depends on context-specific cleansing. +- `reports.py:scan_exit_code` has broad scan policies; the hook needs an independent admission and + coverage decision, without changing the meaning of ordinary scan output. +- Docker, Compose and ops default to 4.0.623, whereas the evaluated baseline is 4.0.625. + +### Git contract + +Git supplies ref-update lines to pre-push and uses its exit status to allow or abort the operation. +The design consumes the supplied object identities and handles creation/deletion explicitly; it +does not infer the pushed tree from the checkout. [Official hook contract](https://git-scm.com/docs/githooks#_pre_push) + +Use machine-readable diffs between recorded snapshots, with rename/deletion attribution and +NUL-delimited paths. A chosen new-branch comparison base is recorded separately from a previous +remote tip. [Official diff documentation](https://git-scm.com/docs/git-diff) + +### Graph reuse and slicing + +Joern documents data-flow and usage slices extracted from a CPG; data-flow slicing can itself be +expensive. This does not establish cheap incremental updates to a changed source tree. The design +therefore reuses exact compatible artifacts and rebuilds changed units, with latency as an explicit +feasibility gate. [Official CPG slicing documentation](https://docs.joern.io/cpg-slicing/) + +## Architecture Pattern Evaluation + +| Option | Benefit | Limitation | Decision | +|---|---|---|---| +| Whole scan on every push with a better ranker | Reuses current driver | Graph build precedes rank selection; measured costs exceed a short hook | Baseline/control only | +| Changed-file-only graph | Small apparent input | Can lose unchanged callers, guards, fields and dispatch context | Reject as a completeness claim | +| Ranker scope + first-party graph reuse + total deadline | Preserves current architecture and explicit vendor boundary | Changed graph units still rebuild; speed must be measured | Proposed | +| New incremental graph engine or mandatory daemon | Potential reuse across edits | Unverified backend semantics and additional lifecycle complexity | Defer until measurements require it | + +## Design Decisions + +1. **Preserve scope ownership.** Change context is input to the existing ranker, not a new selector. + The push layer executes its decisions and records deferred work. Vendor code never returns as + a side effect of context expansion. Unknown vendor behavior uses declared summaries or remains unresolved. +2. **Separate claim from interruption.** Existing rungs describe analysis evidence. Hook admission + additionally requires evaluated capability, change attribution and actionability. This meets the + user's low-noise requirement without pretending an LLM confidence score proves a bug. +3. **Compare evidence, not just finding lists.** A missing base finding can mean truncated analysis. + Targeted comparable base checks are required for novelty, including removed-guard cases. +4. **Adopt Git and the existing engine.** Build only the push transaction, cache and admission adapters. + Do not add an IR, database or service merely to describe future scale. +5. **Measure honest warm work.** Repeating an identical tip is a cache-hit control; warm changed-code + pushes are the product workload. Separate both from cold builds and dependency/configuration changes. + +## Risks and review decisions + +- Proposed 30-second target and enforcement defaults await review; neither is inferred user approval. +- Exact graph reuse may help repeated pushes more than everyday edits. The feasibility experiment + must expose this and can stop rollout; timeout notices alone do not make a useful safety net. +- Initial high-precision admission can lower recall. Joint recall/coverage/noise gates prevent + suppressing everything from masquerading as success. +- Unknown dynamic and cross-language edges remain real limits; ranking cannot manufacture context. +- Existing inspected plugins are not fresh holdouts. Fix scope and evaluation before another tuning cycle. + +## Review record + +### Cross-language clarification + +The maintainer explicitly retained PMPro as the core optimization example and requested transfer +to common JavaScript/Node APIs (Express in particular), with C/C++ as a useful additional example. +Added requirement 8 and the Language and Framework Contract to make agnostic analysis testable. +`evaluation.md` separates development, framework integration, untouched evaluation and C envelope +roles. NodeGoat is a proposed Express integration candidate, not a pinned or measured workload. + +Local inspection found existing request sources and eval/exec/query/path/outbound sinks in +`ruleset/semantic/javascript.toml`, JavaScript obligation facts, and Express-style route/middleware +mapping in `mapping.py`. This establishes reusable implementation seams, not passing end-to-end +coverage. The five current repository recipes contain PHP, Python and C workloads but no JavaScript +recipe. Existing `libpng.toml` explicitly marks its bounds CVE out of scope and measures only envelope. +That distinction is preserved. No new core algorithm, frontend or recipe is implemented here. + +Fresh amendment validation read the six current spec documents, confirmed 35 unique EARS criteria +with explicit design traceability, parsed the amended metadata, and checked the cross-language and +vendor contracts. The command and `git diff --check` both exited 0. This validates documents only; +Node/Express transfer remains the proposed experiment, not a measured result. + +### Earlier validation + +Fresh document validation read all five spec files, parsed metadata, checked all 31 unique EARS +criteria against design traceability, and confirmed populated boundary/file/testing sections. +An initial mechanical check rejected two criteria starting with "Before"; both were rewritten as +event-driven EARS statements and the repeated check passed (exit 0). `git diff --check` passed. +Existing contributor/ranker approval fields were compared with HEAD and remained unchanged; +new requirements/design approvals remain false. Stored ranking-head arithmetic independently +reproduced the equal-score tie bounds of 68, 86 and 40. + +The content review checked boundary ownership, failure handling, vendor exclusion, ranker scope, +and independent finding/coverage/push status. Draft defaults remain clearly labeled. Runtime +integration, performance and capability admission remain unverified work; no code tests were +needed for these documentation-only amendments. diff --git a/.kiro/specs/pre-push-safety-net/spec.json b/.kiro/specs/pre-push-safety-net/spec.json new file mode 100644 index 0000000..c074c08 --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/spec.json @@ -0,0 +1,54 @@ +{ + "feature_name": "pre-push-safety-net", + "created_at": "2026-09-12T15:35:00Z", + "updated_at": "2026-09-12T16:20:40.814596Z", + "language": "en", + "phase": "implementation", + "approvals": { + "requirements": { + "generated": true, + "approved": true + }, + "design": { + "generated": true, + "approved": true + }, + "tasks": { + "generated": true, + "approved": true + } + }, + "ready_for_implementation": true, + "implementation_approval_note": "2026-09-12: maintainer explicitly said 'approve.' in response to approval of the generated task plan and starting implementation with group 1. All tasks are approved; this implementation run starts with tasks 1.1\u20131.4. No task is complete until its fresh-evidence and reviewer gates pass.", + "depends_on": [ + "contributor-scan" + ], + "related_specs": [ + "flow-aware-ranking", + "zero-with-a-reason", + "finding-feedback-loop" + ], + "approval_note": "2026-09-12: user requested amendments and help moving the design forward for a scalable pre-push security safety net, broader than WordPress; clarified that advisory output must be actionable and not SAST noise. Requirements and a proposed design were drafted together under that explicit request. No requirements, design, tasks, enforcement policy, or latency target are marked approved. Phase remains requirements-generated until human review. Existing predecessor approvals are unchanged. ; 2026-09-12: after the amended requirements/design and cross-language evaluation were presented, the maintainer said \"ok, next kiro phase\". Requirements and design are now approved for task generation; tasks remain unapproved. Earlier unapproved statements above describe the historical drafting state.", + "cross_language_note": "2026-09-12: maintainer endorsed the direction and clarified PMPro as core development case, JavaScript/Express Node APIs as required transfer workload, and C/C++ as additional shared-machinery/abstraction-limit evidence. Preserve language-agnostic core decisions; framework facts may differ. Detailed amended requirements/design remain pending review.", + "approval_history": [ + { + "at": "2026-09-12T15:56:00Z", + "source": "ok, next kiro phase", + "action": "Accepted the presented amended requirements and design for task generation; task approval and implementation remain pending." + }, + { + "at": "2026-09-12T16:20:40.814596Z", + "source": "approve.", + "action": "Accepted the generated task plan and authorized starting group 1 (1.1\u20131.4). Implementation started; tasks require independent review and fresh verification before completion." + } + ], + "task_review": { + "verdict": "PASS", + "independent": true, + "repair_passes": 1, + "major_groups": 8, + "executable_subtasks": 27, + "requirements_covered": 35, + "record": "task-review.md" + } +} diff --git a/.kiro/specs/pre-push-safety-net/task-review.md b/.kiro/specs/pre-push-safety-net/task-review.md new file mode 100644 index 0000000..e6b0146 --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/task-review.md @@ -0,0 +1,56 @@ +# Task-plan review: pre-push-safety-net + +Date: 2026-09-12. Scope: design readiness and implementation task graph, not runtime validation. + +## Authorization and phase + +After the amended requirements/design and JavaScript transfer plan were presented, the maintainer +said **"ok, next kiro phase"**. This was applied as approval of those concrete documents for task +generation. Requirements/design are approved; generated tasks remain unapproved and implementation +has not begun. The initial acceptance profile is an evaluation target, not a demonstrated SLA. + +## Design readiness assessment + +**GO for task generation.** The design retains the existing ranker as scope owner, excludes vendor +code from targets/graphs, preserves language-agnostic evidence contracts, and separates graph +claims from actionable change-related alerts. Boundaries, integration paths and all 35 numeric +requirements have concrete design counterparts. No fundamental design gap was found in this review. + +Three material risks are explicit work and release gates, not assumed solved: + +1. Whole/changed-unit graph cost may not meet the hook deadline: measure the uncached baseline + before cache work, then realistic warm edits rather than identical-tip hits alone (5.2, 8.2). +2. Frontend/framework or sanitizer semantics may prevent an actionable claim: retain the gap, + keep the affected capability experimental, and route new engine work to its owning spec (3.3, 8.3). +3. Repeatedly inspected examples do not prove transfer: freeze the core and retain untouched + evidence per language/framework/family, with noise/recall/coverage gates together (1.2, 8.3). + +## Independent task-graph sanity review + +Reviewer received the draft task plan and paths to requirements, design, evaluation and task rules, +without a parent-generated coverage summary. The first verdict was **NEEDS_FIXES**: + +- Cache reuse can complete more work within the same deadline. Task 6.4 must require evidence/ + admission equality for the same completed questions; complete-scope equality belongs to a controlled + sufficient-budget comparison, not every deadline-limited run. +- Requirement 8.1 needs explicit paired normalized PHP/JavaScript contract tests, including equivalent + capability eligibility and an unsupported-origin negative control (3.2, 4.2). +- A Node holdout cannot admit PHP or Python capabilities. Untouched evaluation is required for each + language/framework/family; missing evidence leaves that capability experimental (1.2, 8.3). + +All three were repaired once. The same independent reviewer read the repaired draft and returned +**PASS**, with no remaining important task-graph issue. No return to design was required. + +## Mechanical and scope checks + +- Eight major groups and 27 executable subtasks; one parallel candidate (3.1), otherwise sequential + because the work shares backend, ranker, cache and integration contracts. +- All 35 requirement IDs mapped; every executable subtask has an observable completion condition, + responsibility boundary, and valid explicit dependencies where needed. +- No task is checked off; no acceptance-critical tests are optional. No deployment is included. +- The initial experiment is 5.2; persistent caching follows it. Normal alert eligibility is gated + at 8.3, so experimental diagnostics do not become noisy advisory output during development. +- Final file/metadata validation and `git diff --check` are the fresh document checks. No runtime + code or tests changed, and no graph/performance measurements were rerun during task generation. + +Next phase: human task approval, then implementation starting with group 1. diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md new file mode 100644 index 0000000..a1af1c3 --- /dev/null +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -0,0 +1,217 @@ +# Implementation Plan: pre-push-safety-net + +Status: task plan approved by the maintainer's "approve." on 2026-09-12. Implementation has +started with group 1 (tasks 1.1–1.4); each task remains pending until independent review and fresh verification pass. + +## Execution contract + +- Preserve the ranker as the scope mechanism and vendor exclusion from targets AND graphs. +- Use PMPro for core development and a JavaScript/Express workload for transfer. Keep the core + frozen for the first Node measurement; separate adapter repairs from core changes. +- Each executable subtask is a bounded work unit targeting 1–3 hours of engineering, excluding + unattended measurement time. If a new abstract capability is needed, record the gap and return + it to the owning engine/ranking spec; do not hide that implementation in an integration task. +- Cross-spec engine changes below are explicit integrations of the accepted push contracts. + They do not approve unrelated contributor/ranker backlog or automatically complete its tasks. +- Development runs retain experimental candidate/admission diagnostics without enabling normal + hook alerts. A capability becomes eligible for normal alerts only after task 8.3 passes its gates. +- The first end-to-end milestone is 5.2, before persistent caching and hook installation polish. + Negative performance evidence there informs caching; a broken instrument or absent supported + positive control requires predecessor remediation before claiming that milestone successful. +- Sequential order is the default. Only 3.1 may run alongside group 2 after its explicit prerequisite; + shared backend/driver, cache and report edits otherwise remain serial. No optional acceptance tests. + +- [ ] 1. Establish the runtime, evaluation inputs and shared contracts +- [ ] 1.1 Reproduce the evaluated engine environment + - Align the shipped image, Compose and host-install defaults with Joern 4.0.625; preserve version override and checksum verification. + - Exercise PHP with the native interpreter and the installed JavaScript frontend; open a source and the frontend parser where applicable, recording byte counts and graph census. + - Done when a local image smoke produces valid small PHP and JavaScript graph/query payloads with recorded versions, and an unreadable source fails explicitly. This builds locally; it does not deploy. + - _Boundary: Cache and Engine Boundary, packaging integration_ + - _Requirements: 7.1, 7.3, 8.2_ +- [ ] 1.2 Prepare labeled development and reserved evaluation changes + - Pin a readable PMPro change/fix and a supported Node/Express change/fix, plus benign controls; verify license and source bytes before accepting recipes. + - Use NodeGoat only if its selected mechanism fits a current arbiter; label authored repairs as authored. Reserve an independently selected Node API repository with modern module/async context and reviewable history before tuning on transfer results. + - Keep Python regression and libpng envelope recipes, explicit supported/unsupported populations, ref identities and development/holdout roles in machine-readable evaluation inputs. + - Reserve untouched evaluation for each language/framework/family intended for admission; the Node holdout cannot qualify PHP or Python. Missing per-capability evaluation leaves it experimental and does not delay the initial PMPro/Node diagnostic experiment. + - Done when recipe validation resolves immutable inputs and labels or names an explicit missing prerequisite; no favorable scanner result is required to select a recipe. + - _Boundary: Evaluation harness_ + - _Requirements: 6.1, 6.3, 8.3, 8.4_ +- [ ] 1.3 Make the baseline and push scorer fail honestly + - Retain unmatched targets and unanswered queries in the declared population; require a queried witness before labeling a target transitively detected. + - Emit actual vulnerable/fixed outcomes beside rank, selected mode, engine/facts/scope and elapsed stage costs. Freeze a versioned workload/profile before measuring. + - Done when a deliberately missing target and failed query cannot improve reported recall, while a verified transitive positive is counted with its witness. + - _Boundary: Evaluation harness, repository benchmark integration_ + - _Requirements: 6.2, 6.3, 6.4, 6.5, 7.3_ +- [ ] 1.4 Establish typed push and engine integration contracts + - Add importable push components and validated configuration for comparison base, shared deadline/cancellation allowance, local cache limit, advisory/blocking and separate coverage policy. + - Define ownership of update/comparison, change context, selected/deferred scope, graph identity/lease and separate finding/coverage/push statuses as described in design. + - Keep generic engine contracts outside the push package; leave ordinary scan defaults compatible and capabilities experimental until evaluated. + - Done when contract/config round trips preserve all identities/statuses, invalid values are rejected, and the engine has no dependency on the push package. + - _Boundary: Push runner, Cache and Engine Boundary, CLI/config integration_ + - _Requirements: 4.1, 4.3, 5.3, 5.4, 5.5, 7.1, 7.3_ + +- [ ] 2. Resolve immutable pushed content +- [ ] 2.1 Resolve every supplied ref update and comparison + - Parse Git's complete update input using object-format-aware validation; resolve existing tips, new-branch configured bases, force-pushes, merges, peeled tags, deleted refs and unsupported objects. + - Deduplicate equal targets without merging different base comparisons; retain every ref association. Do not silently substitute HEAD or fetch missing objects. + - Done when multi-ref and missing-base fixtures produce exact comparison identities and explicit dispositions, including deletion-only no-analysis results. + - _Boundary: Snapshot Adapter_ + - _Requirements: 1.1, 1.3, 1.4, 7.1_ +- [ ] 2.2 Materialize isolated object snapshots + - Read immutable tracked blobs into bounded scratch storage using machine-readable paths; do not checkout into the live tree or run filters, project scripts or network requests. + - Preserve path identity and report symlinks, gitlinks, unavailable LFS blobs and unsupported content; never follow source outside the snapshot. + - Done when non-HEAD pushes from a dirty checkout read only the intended object bytes and leave index, files and refs byte-identical after success, cancellation and failure. + - _Boundary: Snapshot Adapter_ + - _Requirements: 1.1, 1.2, 1.3, 2.3, 7.1, 7.3_ +- [ ] 2.3 Attach changed spans and deletion/rename context + - Compare recorded base/head objects using NUL-safe path handling; retain removed spans, renamed identities and declaration/config changes. + - Supply change context to the shared ranker contract rather than selecting an independent changed-files-only scope. + - Done when renamed functions retain correspondence and a removed guard remains an impact seed for an unchanged operation, with missing correspondence explicitly unresolved. + - _Boundary: Snapshot Adapter_ + - _Requirements: 1.4, 2.1, 2.4, 3.4_ + +- [ ] 3. Integrate bounded execution and ranker scope with the existing engine +- [ ] 3.1 (P) Enforce one remaining-time budget in the backend + - Thread the absolute deadline through readability probes, frontend builds, overlays, census, query batches, fallbacks and retries; no phase gets a fresh full budget. + - Bound process-group termination and lease/scratch cleanup by the cancellation allowance, replacing any longer fixed cleanup waits on the push path. + - Done when controlled hanging subprocesses at build/overlay/query stages terminate without orphan children inside the declared allowance, while ordinary scans preserve their prior behavior. + - _Boundary: Cache and Engine Boundary, generic driver integration_ + - _Depends: 1.4_ + - _Requirements: 4.1, 4.2, 7.3_ +- [ ] 3.2 Return the ranker's exact selected and deferred work + - Integrate the existing ranker through explicit mode selection and shared question identity; record selected/deferred IDs, evidence and reasons directly from the final order. + - Preserve unanswered evidence as unknown rather than tier zero; report per-family coverage so taint ranking cannot masquerade as authorization/configuration coverage. + - Add paired PHP/JavaScript normalized-evidence contract tests asserting identical tier/score and selection decisions for equivalent input and budgets, without repository/language priority exceptions. + - Done when changing order produces matching execution and manifest IDs, with no reconstruction from an earlier list, and normalized cross-language equivalence tests pass. + - _Boundary: Ranker Scope Contract, generic driver integration_ + - _Requirements: 2.1, 2.2, 6.4, 7.3, 8.1, 8.2_ +- [ ] 3.3 Connect change context to supported first-party relationships + - Use existing entry/callee, field, hook and guard relationships to supply affected context to ranked questions, including unchanged sinks after source/guard changes. + - Preserve current abstract semantics; absent dynamic or external relationships remain unresolved rather than guessed safe. A required new feature or fact is reported to its owning spec. + - Done when controlled cross-file and removed-guard cases reach the relevant question/witness, while an unresolved boundary cannot produce a complete negative. + - _Boundary: Ranker Scope Contract, Snapshot Adapter integration_ + - _Depends: 2.3, 3.2_ + - _Requirements: 2.1, 2.2, 2.4, 8.1, 8.2_ +- [ ] 3.4 Apply vendor exclusion and explicit language partitions + - Reuse declared layout exclusions before target enumeration and graph build; retain unshipped first-party tests and separate browser/server or language capability coverage. + - Build supported frontend partitions explicitly; preserve unknown cross-language and vendor semantics without restoring excluded sources. + - Done when PHP and Node graphs/targets contain no declared vendor code, first-party callers remain visible, and unsupported partitions appear as coverage limits rather than clean results. + - _Boundary: Ranker Scope Contract, Cache and Engine Boundary integration_ + - _Requirements: 2.2, 2.3, 7.3, 8.2, 8.4_ + +- [ ] 4. Implement change attribution and actionable output +- [ ] 4.1 Compare targeted base/head evidence for defect novelty + - Establish stable mechanism/operation identity with rename/span mapping and collect comparable base evidence under the same semantics and remaining deadline. + - Distinguish new, worsened, unchanged and unknown; an absent finding from a failed/truncated base is unknown. Support removed guards and new source connections at unchanged sinks. + - Done when benign movement and unchanged backlog do not become new defects, while a supported removed-guard case has a concrete change-to-witness relationship. + - _Boundary: Delta and Actionability Policy, engine comparison integration_ + - _Depends: 2.3, 3.3, 3.4_ + - _Requirements: 2.4, 3.1, 3.4, 4.1_ +- [ ] 4.2 Enforce capability and actionability admission + - Require evaluated capability provenance, applicable semantics, a sufficient rung, exact witness/location, supported change attribution and a concrete repair direction. + - Keep experimental, suspicious, wrongly sanitized and unknown-novelty candidates in diagnostic records; do not let model confidence, a sanitizer name or printed-before state admit/dismiss a defect. + - Deduplicate by defect while retaining relevant witnesses and refs; distinguish experimental evaluation from normal hook eligibility. + - Add equivalent PHP/JavaScript normalized claims with equally eligible capabilities and assert equal admission; add an unsupported-origin control whose coverage/eligibility differs without changing core scores. + - Done when the policy rejects an entailed but unsupported claim and admits fully supported synthetic controls equivalently across origins and advisory/blocking modes; real capabilities remain disabled pending 8.3. + - _Boundary: Delta and Actionability Policy_ + - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 5.3, 5.4, 8.1_ +- [ ] 4.3 Render compact results and preserve complete artifacts + - Render up to three actionable summaries plus remaining count; emit at most one no-finding status and one aggregated incomplete notice with a useful next action. + - Persist all defects, candidate dispositions, exact scope, identities, capability/ranker provenance and timings with separate finding/coverage/push status. + - Done when the display cap does not change the push decision, incomplete analysis cannot read as secure, and artifact-write failure produces a bounded notice while retaining the real decision. + - _Boundary: Result and Hook Adapter_ + - _Requirements: 3.1, 3.2, 3.3, 5.1, 5.2, 5.5, 7.3_ + +- [ ] 5. Prove the first end-to-end experiment before cache optimization +- [ ] 5.1 Wire explicit base/head replay through the full transaction + - Connect immutable snapshots, ranker decisions, existing arbiter, targeted comparison, admission diagnostics and compact artifacts through an explicit local replay command. + - Apply the shared budget to preparation and both revisions; no model/service or installed hook is required. All runtime errors preserve coverage state. + - Done when a controlled security change and fixed twin traverse the production runner through the existing ranker and produce replayable results with truthful deadlines. + - _Boundary: Push runner, CLI, Snapshot Adapter, engine and policy integration_ + - _Depends: 1.3, 3.1, 3.4, 4.3_ + - _Requirements: 1.1, 2.1, 3.1, 3.4, 4.1, 4.2, 5.5, 7.1, 7.3_ +- [ ] 5.2 Measure PMPro and frozen-core Node transfer + - Replay each prepared security/fixed/benign case, first using the declared hook deadline; record timeouts and stage costs. A separately named extended lab run may obtain witnesses but cannot satisfy the hook latency gate. + - Freeze the core after PMPro, run Node with existing facts, and classify instrument, mapping, fact, scope and domain gaps without tuning on an untouched holdout. + - Done when one joint artifact reports actual evidence and outcomes for both workloads, with unresolved cases retained and an explicit feasibility decision; no normal capability is enabled here. + - _Boundary: Evaluation harness, full-run integration_ + - _Depends: 1.2, 1.3, 5.1_ + - _Requirements: 6.2, 6.3, 6.4, 6.5, 8.1, 8.2, 8.3, 8.4_ + +- [ ] 6. Reuse valid graphs and results without stale conclusions +- [ ] 6.1 Validate graph artifact identity and leases + - Have the backend own source/declaration/exclusion digests, frontend/engine/overlay options, graph census and complete/partial identity. + - Permit validated graph leases in the generic driver while preserving the fresh-build path; isolate potentially mutating loads and retain bounded cleanup. + - Done when identical valid inputs can reuse a graph while any changed graph input or failed census rejects the lease; original cached artifacts survive query loading unchanged. + - _Boundary: Cache and Engine Boundary, generic driver integration_ + - _Requirements: 4.2, 4.3, 4.4, 7.3_ +- [ ] 6.2 Publish and evict local artifacts safely + - Add private temporary publication, per-key bounded locking, atomic completed manifests, cache size enforcement and unleased eviction. + - Retain partial records only as diagnostics; never interpret a directory or failed write as a complete result. Do not rebuild graphs incrementally by mutating old node identities. + - Done when concurrent writers, corrupt entries and interrupted publication yield either a validated complete artifact or an explicit miss without exceeding the remaining lock budget. + - _Boundary: Cache and Engine Boundary, push cache integration_ + - _Depends: 6.1_ + - _Requirements: 4.1, 4.3, 4.4, 7.3_ +- [ ] 6.3 Key query and comparison results by all relevant semantics + - Include graph identity, question/context/depth, facts/sanitizers/query versions and configuration; include ranking/admission/model provenance for final results and both revision identities for comparisons. + - Keep incomplete query/comparison results out of complete lookups; preserve current eligibility checks after reuse. + - Done when source, context, fact, query, mode, policy and optional model changes invalidate the appropriate layer, while an identical compatible request returns the same evidence and provenance. + - _Boundary: Push cache, policy integration_ + - _Requirements: 3.4, 4.3, 4.4, 6.4, 7.2, 7.3_ +- [ ] 6.4 Integrate cache reuse into the shared push deadline + - Reuse identical snapshots/unchanged declared units across refs without merging different comparisons; changed units rebuild and stale graph relationships remain hints only. + - Account for lookup, lock wait, lease, build/query and result persistence in the same deadline; preserve completed outcomes on cancellation. + - Done when cached and uncached replay produce equal evidence/admission for the same completed questions under compatible semantics, with full equivalence under a sufficiently large controlled budget. Under the hook deadline, verify truthful extra completion/deferred scope from cache reuse rather than requiring identical scope; no stale answer survives a dependency/configuration edit. + - _Boundary: Push runner, cache and engine integration_ + - _Depends: 5.1, 6.2, 6.3_ + - _Requirements: 1.1, 2.2, 4.1, 4.2, 4.3, 4.4, 7.3_ + +- [ ] 7. Connect the real push interface without adding noise +- [ ] 7.1 Execute Git's full push input and enforcement policy + - Extend the replay command to consume all update lines and remote arguments, aggregating per-ref comparisons under one deadline and preserving independent coverage and finding statuses. + - Apply advisory, opt-in blocking and separate incomplete-coverage policy; keep external services disabled by default and normal alerts limited to admitted capabilities. + - Done when controlled multi-ref push invocations exercise all allow/block/status combinations, including deletion-only, missing-base and no-work cases, without resetting budgets. + - _Boundary: CLI/config, Push runner and Result and Hook Adapter integration_ + - _Depends: 2.1, 4.2, 4.3, 6.4_ + - _Requirements: 1.1, 1.3, 1.4, 4.1, 5.1, 5.2, 5.3, 5.4, 5.5, 7.1_ +- [ ] 7.2 Provide non-destructive hook integration and removal + - Supply an executable integration snippet/installation path with explicit invocation, preserving existing hooks, hook path configuration, remote arguments and stdin for other consumers. + - Refuse automatic overwrite; retain an existing hook's nonzero result even when the Safety Net runs advisory. Include concrete integration/removal instructions with the delivered hook interface. + - Done when a disposable Git remote accepts/rejects pushes correctly with an existing stdin-reading hook, and removal restores the preceding behavior without touching source or refs. + - _Boundary: Result and Hook Adapter, CLI integration_ + - _Requirements: 1.2, 5.3, 5.4, 7.4_ +- [ ] 7.3 Keep optional model assistance within existing evidence rules + - Reuse explicit endpoint configuration and redaction only when enabled; propagate the remaining deadline to model calls and record model/prompt/cost provenance. + - Restrict explanations to existing witnesses and preserve deterministic no-model behavior; do not implement a new learned ranker or prompt optimizer in this task. + - Done when a delayed or misleading model cannot extend the push budget, leak unredacted input in the tested path, raise a rung or bypass actionability admission. + - _Boundary: Push runner, existing model and policy integration_ + - _Requirements: 3.5, 4.1, 4.2, 7.1, 7.2, 7.3_ + +- [ ] 8. Validate usefulness, runtime and cross-language admission +- [ ] 8.1 Exercise complete push failure and regression paths + - Combine dirty/non-HEAD/multi-ref/new-branch/force/tag/deletion pushes with unusual paths, external content boundaries, failed census, missing comparisons and interrupted persistence. + - Verify no orphan children, no overwritten live state, no vendor code in graphs, exact scope reports and no false clean result from skipped work. + - Done when the integration suite covers the critical flows through the real CLI/backend seams and preserves ordinary scan behavior. + - _Boundary: Push runner integration tests_ + - _Requirements: 1.1, 1.2, 1.3, 1.4, 2.2, 2.3, 4.2, 4.4, 5.2, 5.5, 7.1, 7.4, 8.2_ +- [ ] 8.2 Measure realistic cache and deadline feasibility + - Measure identical-tip reuse, a warm one-function edit, dependency/configuration edits, cold starts, repository growth and multi-ref workloads on declared hardware. + - Publish p50/p95, timeouts, cancellation allowance, completed/deferred population and stage costs; the denominator is fixed before ranker selection. + - Done when the versioned scorecard evaluates the accepted latency/coverage gates without hiding timeouts or calling repeated-identical-tip hits representative warm edits; a failed gate remains NO-GO for rollout. + - _Boundary: Evaluation harness, runtime integration_ + - _Requirements: 4.1, 4.2, 6.2, 6.3, 6.5, 7.3_ +- [ ] 8.3 Evaluate independent transfer and admit only passing capabilities + - Run the frozen candidate on reserved Node changes and benign/fixed controls; preserve PHP/Python regressions, including a verified VAmPI transitive target, and report libpng envelope/unsupported bounds separately. + - Score actual actionable alerts, reviewed precision with counts/intervals, supported recall, benign interruptions and fixed-side silence alongside latency and coverage. Record adapter changes separately; a core retune requires a new untouched evaluation. + - Generate versioned capability eligibility only when all predeclared gates pass; no positive controls, no reviewed alerts, no untouched workload, wrong family semantics or unsupported properties prevent admission. + - Require untouched evaluation for the particular language/framework/family being admitted. Node results cannot qualify PHP/WordPress or Python; capabilities without their own untouched evidence remain experimental with an explicit disposition. + - Done when the policy consumes a reproducible PASS/NO-GO capability artifact tied to exact semantics and rejects stale eligibility; a NO-GO result never silently enables advisory alerts. + - _Boundary: Evaluation harness, Delta and Actionability Policy integration_ + - _Depends: 1.2, 1.3, 5.2, 7.3, 8.2_ + - _Requirements: 3.1, 3.2, 6.1, 6.2, 6.3, 6.4, 6.5, 8.1, 8.2, 8.3, 8.4_ +- [ ] 8.4 Verify the packaged end-to-end outcome and acceptance coverage + - Run the required regression checks and a local packaged smoke over supported security/fixed/benign pushes; verify no-model startup, preservation of hook behavior and truthful partial results. + - Reconcile every acceptance criterion with fresh evidence and record unresolved capability/performance blockers separately from completed plumbing. + - Done when a reproducible validation artifact states GO only with all mandatory integration and joint quality/latency gates satisfied; otherwise it names the remaining NO-GO conditions without marking the feature complete. + - _Boundary: Packaged runtime, full feature validation integration_ + - _Requirements: 3.1, 3.4, 5.1, 5.2, 5.3, 5.4, 5.5, 6.5, 7.1, 7.3, 7.4, 8.3, 8.4_ diff --git a/.kiro/steering/overview.md b/.kiro/steering/overview.md index 1b9c242..7ff6d49 100644 --- a/.kiro/steering/overview.md +++ b/.kiro/steering/overview.md @@ -1,5 +1,17 @@ # Overview — what this is, and what is actually true of it +**Current direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated +development across supported ecosystems. See `safety-net.md` and the proposed `pre-push-safety-net` +spec. The ranker remains the scope mechanism; third-party vendor code stays outside targets and +graphs. Advisory must meet the same actionability bar as blocking. + +**Historical scoreboard below:** it was written September 9. Later commits record a completing +whole-PMPro scan and Joern 4.0.625 ranking measurements (82/91/54 on the three plugins), while +default installation pins remain 4.0.623. The pre-push workflow, changed-code latency and actionable +precision are not demonstrated by those results. In particular, the manually selected two-file +slice below is a useful experiment, not evidence that an automatic push scope would select the +same context. Current scope/status corrections live in `flow-aware-ranking/brief.md` and the roadmap. + **Written 2026-09-09.** Fourteen specs have accumulated and the roadmap still described a pre-Joern world, so this is the synthesis above them: what the system does, what has been measured, and what is not true yet. Every number here comes from a committed artifact in `benchmarks/measurements/`. diff --git a/.kiro/steering/roadmap.md b/.kiro/steering/roadmap.md index 36861ff..1b4ed39 100644 --- a/.kiro/steering/roadmap.md +++ b/.kiro/steering/roadmap.md @@ -1,5 +1,72 @@ # Roadmap +## Current direction and next review — 2026-09-12 + +The product is an **actionable pre-push security safety net for AI-accelerated development**, across +supported languages/frameworks. WordPress is one regression workload. Advisory must be useful and +low noise; moving speculative SAST output into a nonblocking hook does not meet the goal. See +`safety-net.md` for current steering. The earlier narrative below is retained as dated history and +does not override this direction or the latest measurements. + +### Approach decision + +Keep the existing ranker as the scope mechanism. Supply it with pushed-change and affected +first-party context. Third-party vendor code remains outside both targets and graphs. Reuse +compatible graph/results artifacts and apply a total deadline; treat unknown external semantics +as a coverage boundary. Require change attribution, witness and repair direction before a finding +earns normal hook visibility. A second heuristic scope selector and an assumed incremental Joern +engine are out of scope. + +### Current evidence and limitations + +- Whole-PMPro analysis has completed with its known CVE; the earlier "does not finish" diagnosis + is superseded. The recorded 72-minute scan is not a pre-push performance result. +- The latest Joern 4.0.625 + layout ranking budgets are PMPro 82, WP Statistics 91, MW WP Form 54, + VAmPI 16. `<50` remains unmet; VAmPI's transitive SQLI is not measured by this position metric. +- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing + ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller. +- Docker/Compose/ops still default to 4.0.623. Pin and provenance alignment precede deployment claims. +- `contributor-scan` has 24/40 checked tasks; historical checkboxes are not rewritten by this review. + `flow-aware-ranking` is still an unapproved brief despite implementation history. + +## Existing Spec Updates + +- [ ] `contributor-scan` — review the draft driver/backend integration amendment; reconcile packaging + pins, family-specific sanitizer task 5.6, exact scope reporting and repository baseline honesty. + Preserve previously approved work; the new amendment is not implicitly approved. +- [ ] `flow-aware-ranking` — review the amended experiment contract: ranker owns scope, vendor + boundary preserved, missing targets stay unresolved, actual detection and cost accompany rank, + untouched evaluation and exploration precede learned-ranking claims. +- [ ] `zero-with-a-reason` — reconcile the broader funnel proposal with the hook's consumed coverage + contract; it is adjacent work, not a requirement to finish every diagnostics mechanism first. + +## Specs (dependency order) + +- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact + reuse, total deadline, change attribution, actionability admission and compact hook behavior. + Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated + (8 groups, 27 executable subtasks), independently reviewed and approved; group 1 implementation started. + +### Next concrete step + +The maintainer approved `pre-push-safety-net/tasks.md` and starting group 1. Requirements and design +were accepted when the maintainer requested the next Kiro phase; the 27-subtask plan passed independent +sanity review. Complete group 1 (reproducible runtime, readable PMPro/Node inputs, honest scoring +and shared contracts). The first complete experiment is 5.2, before persistent caching and hook +installation work. Keep PMPro as the primary core-optimization example, then freeze the core and +prove a changed security flow and its fixed twin on a JavaScript/Express Node API workload through +the same ranker, with a real deadline and vendor-free graph. Include benign pushes and retain Python +regressions. `pre-push-safety-net/evaluation.md` proposes NodeGoat for integration, a separate untouched +Node API repository for generalization, and libpng for C envelope/abstraction-limit evidence. Shared +abstract evidence and decisions are mandatory; framework facts/adaptations are reported separately. +Measure realistic changed-code cache behavior before more ranker tuning. Broader runtime and +capability admission follow that result; an LLM ranker is optional. + +## Historical roadmap narrative — September 9–11 + +The sections below preserve prior plans and measurements. Statements about the current bottleneck, +PHP coverage, rollout order or feature readiness are superseded by the current section above. + **Rewritten 2026-09-09.** The previous version (kept as `roadmap-2026-09-04.md`) described a pre-Joern world — "adjudication is still Python-`ast` plus a tree-sitter CLI stub", "Joern-as-required deferred". Joern was adopted, the model layer was built, wired and measured, and two repositories have been scanned. See diff --git a/.kiro/steering/safety-net.md b/.kiro/steering/safety-net.md new file mode 100644 index 0000000..8778520 --- /dev/null +++ b/.kiro/steering/safety-net.md @@ -0,0 +1,82 @@ +# Product direction: a security safety net before push + +Updated 2026-09-12 from the maintainer's explicit direction. This direction supersedes the +WordPress-first release framing in older steering; historical measurements remain historical. + +## Purpose + +AI accelerates code production. OpenUltraSAST should help developers catch concrete security +regressions before pushing, with enough evidence to understand and repair them. It is also a +teaching system for understanding what abstract analysis can establish and where it stops. + +The maintainer's acceptance criterion is an actionable safety net, not a code nag. Advisory mode +does not lower the evidence bar. A nonblocking stream of speculative findings still fails the product. + +## Product principles + +- Make new or worsened security defects visible; do not repeatedly present unrelated backlog. +- Every normal hook alert needs a supported security claim, a connection to the proposed change, + an exact location, and a concrete repair direction. A confidence label alone is insufficient. +- Keep suspicion and uncalibrated detector output in opt-in diagnostic artifacts. +- Separate finding disposition from coverage. An incomplete scan gets a concise operational notice, + not a fabricated vulnerability or a clean verdict. Silence is bounded by the recorded scope. +- The ranker is the scope mechanism: it selects and orders the regions/questions receiving analysis. + Change information supplies its context; there is no second heuristic scope selector in the hook. + Ranking never establishes a defect or earns permission to alert. +- Third-party vendor code is excluded from both analysis targets and graphs. This is an explicit + product boundary. Flows requiring its internals need declared summaries or remain unresolved; + context expansion must not silently reintroduce vendor sources. +- Preserve vulnerability-family semantics: escaping for one context does not discharge another. +- Reuse results only when code, context, analysis semantics, and scope still match. +- A push deadline covers all work, including snapshot preparation, graph construction, retries, + queries, optional model calls, and reporting. A region count is not a runtime bound. + +## Scope and evidence + +PMPro remains the primary worked example for core optimization. JavaScript/Node APIs, especially +Express routing and middleware, are a required transfer workload alongside Python; they must not +be deferred until a PHP-specific implementation has hardened. Polyglot repositories and common +injection, path, authorization, and configuration scenarios belong in the evaluation according to +demonstrated capabilities. Parsing a language does not imply support for its security properties. +C/C++ supplies an additional shared-machinery and abstraction-limit workload; C bounds/arithmetic +reasoning remains a separate abstraction question. + +Language-agnostic analysis is an architectural requirement: frontends and versioned framework facts +map concrete code to shared sources, sinks, carried values, guards/obligations, configuration values +and evidence. Core ranking, scope selection and arbitration consume those contracts. No PMPro paths, +WordPress API names, Express method names or language-specific score boosts belong in core ranking. +Equivalent normalized evidence must produce equivalent core tier/score decisions. This is a contract +to verify, not an assumption that every frontend already produces equally complete graphs. + +Optimize on PMPro, freeze the core, and measure transfer to Node/Express before further tuning. +Report changes needed in framework facts separately from core changes. Once an example informs a +change it becomes development/regression data; keep a separate untouched repository for evaluation. + +Measure precision and actionability of alerts, recall on labeled security changes, fixed-side +silence, benign-push interruption rate, completed coverage, and cold/warm latency. Report sample +counts and uncertainty. More findings or a better rank position is not product success. + +Keep the repeatedly inspected plugins as regression workloads. Evaluate new ranking or alert +policies on untouched repositories and benign changes as well. A policy that suppresses all alerts +does not pass: useful recall and coverage are required alongside low noise. + +## Specification boundaries + +- `pre-push-safety-net`: immutable pushed snapshots, ranker integration, reusable results, total budget, + change attribution, actionability admission, and concise hook behavior. +- `contributor-scan`: reusable scan driver/backend, family correctness, packaging, repository validation. +- `flow-aware-ranking`: reusable scheduling of questions and honest ranking experiments. +- `zero-with-a-reason`: broader pipeline diagnostics; the hook consumes concrete coverage information. +- `finding-feedback-loop`: review ledger, dismissals, and later feedback-driven evolution. + +The hook does not require an LLM ranker or an optimizer. Start with the existing detector, earn +alert admission per capability, and add scheduling sophistication only when it improves the measured +developer outcome. Do not rebuild a custom IR, add a second engine, or assume incremental Joern +updates as a shortcut to an unmeasured latency promise. + +## Approval state + +The product direction above is user-requested. The amended requirements, design and initial +evaluation profile in `pre-push-safety-net` were approved for task generation on 2026-09-12 +("ok, next kiro phase"). Task approval remains pending. This is not authorization to implement +or a claim that hook performance or capability admission has been demonstrated.