Open_Ultra_SAST/.kiro/specs/pre-push-safety-net/research.md

Research and design decisions: pre-push-safety-net

Summary

• Date: 2026-09-12. Discovery scope: complex integration of the existing ranker and detector with Git.
• User direction: a scalable pre-push safety net as AI accelerates development; WordPress is one case.
• Clarifications: advisory must be actionable, not SAST noise; the ranker is the scope mechanism; third-party vendor code must remain outside targets and graphs.
• Requirements, design and tasks are now approved, and groups 1–3 are verified complete after the approved frontend-retention repair; see tasks.md for verified completion. The discovery log below preserves the earlier drafting state and measurements. No hook latency claim is made.

Research Log

Repository state and measurement limits

Reviewed through a8069ac, including 60963f5 (overlay reuse/scoped seeds), 9f881b4 (closure workaround bounded by upstream version), 60d13d7 (evidence ordering), 3b3b96b (parser-script readability probe), b821ac6 (layout), and the latest ranking baselines.

Workload	Original position budget	4.0.625 + layout budget	Build / evidence seconds
PMPro	391	82	129.9 / 295.7
WP Statistics	504	91	45.4 / 68.7
MW WP Form	513	54	45.2 / 80.3
VAmPI	16	16	5.6 / 18.4

Source: benchmarks/ranking/baselines/2026-09-11-evidence-rank-layout.json, read and parsed during the review. All four recorded pairs_unanswered=0; these are historical artifacts, not fresh runs. WP Statistics's scope shrank after vendor removal. VAmPI's transitive SQLI target is excluded from the position budget, so the value is not verified all-target recall. A 72-minute whole-PMPro scan recorded in flow-aware-ranking/brief.md demonstrates a finishing scan, not hook feasibility or precision of its 250 entailed findings. The focused review run passed 87 existing tests; it did not rebuild Joern graphs or evaluate hook behavior.

The latest ranker head has 67 strictly higher-scored same-tier regions ahead of PMPro's CVE and 85 ahead of WP Statistics's. Reordering equal scores alone cannot beat budgets 68 and 86. MW WP Form has 39 strictly higher-scored regions, so its corresponding lower bound is 40. These are arithmetic bounds from the stored head, not a measured new ranking algorithm.

Existing seams

• preprocess.py:preprocess_repository reads filesystem bytes and then records HEAD; snapshot identity must be established before calling this path for a push.
• model/scan.py:scan_repository builds a graph before applying max_regions. Its evidence order is experimental and cli.py currently does not enable it. Region budget is not graph cost.
• model/layout.py supplies vendor exclusions and test/shipped classification. Preserve it.
• cpg/backend.py has process-group cancellation, census and parser readability checks, but fresh temporary graphs and per-command rather than per-push deadlines.
• cli.py samples unjudged paths by slicing the original region order; the new contract must return exact selected/deferred IDs from the ranker/driver instead of reconstructing them by count.
• model/specs.py:taint_specs still flattens sanitizers across a language. Contributor task 5.6 is a prerequisite to admitting capabilities whose correctness depends on context-specific cleansing.
• reports.py:scan_exit_code has broad scan policies; the hook needs an independent admission and coverage decision, without changing the meaning of ordinary scan output.
• Docker, Compose and ops default to 4.0.623, whereas the evaluated baseline is 4.0.625.

Git contract

Git supplies ref-update lines to pre-push and uses its exit status to allow or abort the operation. The design consumes the supplied object identities and handles creation/deletion explicitly; it does not infer the pushed tree from the checkout. Official hook contract

Use machine-readable diffs between recorded snapshots, with rename/deletion attribution and NUL-delimited paths. A chosen new-branch comparison base is recorded separately from a previous remote tip. Official diff documentation

Graph reuse and slicing

Joern documents data-flow and usage slices extracted from a CPG; data-flow slicing can itself be expensive. This does not establish cheap incremental updates to a changed source tree. The design therefore reuses exact compatible artifacts and rebuilds changed units, with latency as an explicit feasibility gate. Official CPG slicing documentation

Architecture Pattern Evaluation

Option	Benefit	Limitation	Decision
Whole scan on every push with a better ranker	Reuses current driver	Graph build precedes rank selection; measured costs exceed a short hook	Baseline/control only
Changed-file-only graph	Small apparent input	Can lose unchanged callers, guards, fields and dispatch context	Reject as a completeness claim
Ranker scope + first-party graph reuse + total deadline	Preserves current architecture and explicit vendor boundary	Changed graph units still rebuild; speed must be measured	Proposed
New incremental graph engine or mandatory daemon	Potential reuse across edits	Unverified backend semantics and additional lifecycle complexity	Defer until measurements require it

Design Decisions

1. Preserve scope ownership. Change context is input to the existing ranker, not a new selector. The push layer executes its decisions and records deferred work. Vendor code never returns as a side effect of context expansion. Unknown vendor behavior uses declared summaries or remains unresolved.
2. Separate claim from interruption. Existing rungs describe analysis evidence. Hook admission additionally requires evaluated capability, change attribution and actionability. This meets the user's low-noise requirement without pretending an LLM confidence score proves a bug.
3. Compare evidence as well as finding lists. A missing base finding can mean truncated analysis. Targeted comparable base checks are required for novelty, including removed-guard cases.
4. Adopt Git and the existing engine. Build only the push transaction, cache and admission adapters. Do not add an IR, database or service merely to describe future scale.
5. Measure honest warm work. Repeating an identical tip is a cache-hit control; warm changed-code pushes are the product workload. Separate both from cold builds and dependency/configuration changes.

Risks and review decisions

• Proposed 30-second target and enforcement defaults await review; neither is inferred user approval.
• Exact graph reuse may help repeated pushes more than everyday edits. The feasibility experiment must expose this and can stop rollout; timeout notices alone do not make a useful safety net.
• Initial high-precision admission can lower recall. Joint recall/coverage/noise gates prevent suppressing everything from masquerading as success.
• Unknown dynamic and cross-language edges remain real limits; ranking cannot manufacture context.
• Existing inspected plugins are not fresh holdouts. Fix scope and evaluation before another tuning cycle.

Review record

Cross-language clarification

The maintainer explicitly retained PMPro as the core optimization example and requested transfer to common JavaScript/Node APIs (Express in particular), with C/C++ as a useful additional example. Added requirement 8 and the Language and Framework Contract to make agnostic analysis testable. evaluation.md separates development, framework integration, untouched evaluation and C envelope roles. NodeGoat is a proposed Express integration candidate, not a pinned or measured workload.

Local inspection found existing request sources and eval/exec/query/path/outbound sinks in ruleset/semantic/javascript.toml, JavaScript obligation facts, and Express-style route/middleware mapping in mapping.py. This establishes reusable implementation seams, not passing end-to-end coverage. The five current repository recipes contain PHP, Python and C workloads but no JavaScript recipe. Existing libpng.toml explicitly marks its bounds CVE out of scope and measures only envelope. That distinction is preserved. No new core algorithm, frontend or recipe is implemented here.

Fresh amendment validation read the six current spec documents, confirmed 35 unique EARS criteria with explicit design traceability, parsed the amended metadata, and checked the cross-language and vendor contracts. The command and git diff --check both exited 0. This validates documents only; Node/Express transfer remains the proposed experiment, not a measured result.

Earlier validation

Fresh document validation read all five spec files, parsed metadata, checked all 31 unique EARS criteria against design traceability, and confirmed populated boundary/file/testing sections. An initial mechanical check rejected two criteria starting with "Before"; both were rewritten as event-driven EARS statements and the repeated check passed (exit 0). git diff --check passed. Existing contributor/ranker approval fields were compared with HEAD and remained unchanged; new requirements/design approvals remain false. Stored ranking-head arithmetic independently reproduced the equal-score tie bounds of 68, 86 and 40.

The content review checked boundary ownership, failure handling, vendor exclusion, ranker scope, and independent finding/coverage/push status. Draft defaults remain clearly labeled. Runtime integration, performance and capability admission remain unverified work; no code tests were needed for these documentation-only amendments.

Group 2 implementation: immutable Git reads

The local Git runtime is 2.43.0. Snapshot reads must suppress replacement objects and implicit promisor fetching, including on Git versions that predate newer lazy-fetch controls. The protocol allow-list is the stronger boundary: an empty GIT_ALLOW_PROTOCOL forbids transports even when repository configuration permits an individual protocol. Git documents this precedence in its environment-variable reference. Regression fixtures must prove supplied-object identity and lack of helper execution; setting environment variables alone is not evidence of either property.

The snapshot adapter provides change evidence, not a second region selector. Source materialization, changed spans, and correspondence must remain language agnostic; unknown parser or correspondence boundaries cannot become a successful negative.

Group 3: frontend census invalidates completion

Tasks 3.1–3.3 passed independent review and real engine checks. Task 3.4 passed projection/ranking unit checks but failed a packaged graph census: 92 readable JavaScript source bytes included a 41-byte test file that never reached the graph. Direct frontend invocation reproduced the omission. Two unconditional upstream test filters require an owning foundation repair; see ../contributor-scan/frontend-retention-prerequisite.md. The runtime failure is not overridden by passing mocks, static checks or a populated server graph.

Group 3 completed after frontend and coverage repairs

The approved contributor-scan 2.18 adaptation preserves test directories and filename suffixes through both pinned JavaScript frontend filters. Task 3.4 then passed independent review with exact graph-path census, including PHP aggregation. Review also repaired unreadable discovery inputs throwing rather than reporting incomplete coverage. A synthetic graph file must never compensate for an omitted real source, and a wrapper must not discard evidence supplied by a frontend. See implementation-group-3.md for final evidence. Eleven of 27 tasks are complete; change attribution, actionability admission and compact output are next.

Group 4 comparison controls (2026-09-13)

Real JavaScript dominance rows distinguish a guarded base operation from an unguarded head operation while retaining a guarded sibling. The direct task 4.1 comparator can consume these answers with declared questions and immutable Git span correspondence. This does not establish automatic context projection for that family: the existing driver's change-context evidence pass currently projects taint questions, and a dominance-only scan with change_context reports context_projection_unavailable:contributor-scan. Preserve that limitation in the full replay integration at 5.1; any extension belongs to the contributor-scan context contract, not a hook-specific graph or detector workaround.

Duplicate identical source lines also remain ambiguous to the snapshot adapter. Controls with unique lines prove the narrow supported correspondence path; they do not establish general identity across arbitrary refactoring. Changed source text alone is insufficient to distinguish a new source from a benign identifier rename. Those comparisons remain unknown. A completed comparable base with no flow to an unchanged mapped sink supplies a supported new-connection control instead.