Open_Ultra_SAST/.kiro/specs/pre-push-safety-net/validation-group-8.md

# Packaged acceptance validation, group 8

## Validation Report
- DECISION: NO-GO
- FEATURE_COMPLETE: false
- BLOCKED_TASKS: 8.3, independent reviewed populations and verified VAmPI transitive evidence are missing; useful completion and runtime gates fail.
- OWNERSHIP: UPSTREAM for engine/census/context prerequisites; LOCAL for evaluation population and capability qualification.
- UPSTREAM_SPEC: contributor-scan; see `pre-push-runtime-followup.md` there.

This report distinguishes implemented safety contracts from demonstrated useful detection.
All 35 criteria below are mapped. A contract test does not qualify a language or framework,
and this report makes no rollout or production-readiness claim. Task 8.4 permits this
explicit NO-GO outcome; task 8.3 and the feature remain incomplete.

## Acceptance coverage

Paths in the evidence column refer to `tests/` unless otherwise indicated. Current
full-suite evidence covers the code at the group 8.3 implementation commit. Packaged
receipts below exercise the built release with its installed NO-GO registry.

| Criterion | Evidence | Assessment |
|---|---|---|
| 1.1 | `test_push_snapshot.py`, `test_push_interface.py`, `test_push_regression_integration.py` | Exact immutable comparisons/ref associations verified by controls. |
| 1.2 | Snapshot/integration tests; packaged disposable-hook smoke | Working source/index/refs and preceding hook preserved. |
| 1.3 | Snapshot and multi-ref tests | Delete/missing-base/unsupported dispositions remain explicit. |
| 1.4 | Snapshot/context tests | Comparison rules, force/new-branch/tag/rename identities retained. |
| 2.1 | Context/scope integration tests; real Node/VAmPI artifacts | Ranker owns selection; useful context incomplete on real workloads. |
| 2.2 | Scope, runner and failed-census controls | Omitted context is an explicit gap; real dynamic dependencies remain unresolved. |
| 2.3 | `test_push_regression_integration.py`, partition/backend tests | Physical first-party filtering verified; external semantics remain unresolved. |
| 2.4 | Removed-guard context/admission tests | Contract verified; real dominance context projection remains unqualified. |
| 3.1 | `test_push_admission.py`, `test_push_report.py`, `test_push_eligibility.py` | Actionability admission enforced; zero qualified real capabilities. |
| 3.2 | Admission/eligibility tests; installed NO-GO registry | Unreviewed/unsupported candidates do not become normal alerts. |
| 3.3 | Admission/report tests | Duplicate witness paths produce one defect summary. |
| 3.4 | Delta/admission/report tests; real same-snapshot VAmPI | Unchanged or unknown novelty cannot create a new-defect alert. |
| 3.5 | Model-assistance tests | Model selection cannot create evidence or change admission. |
| 4.1 | Execution-budget and multi-ref tests; real timeout receipts | Single absolute budget retained; useful completion gate fails. |
| 4.2 | Cancellation/publication controls and 21-case runtime profile | Partial artifacts retained; measured cancellation/report allowance below 2s. |
| 4.3 | Cache invalidation/reuse tests and packaged group 6 control | Exact input/context/semantic compatibility required. |
| 4.4 | Corrupt/partial/cache-failure controls | No partial or stale clean verdict reused. |
| 5.1 | Report/CLI tests and packaged output | Compact supported-scope status; no repository-security claim. |
| 5.2 | Timeout artifacts and report tests | Compact operational notice with next action/artifact, no per-file warning flood. |
| 5.3 | CLI/admission tests; packaged advisory/prior-hook smoke | Advisory permits push, preserves preceding rejection, same evidence bar. |
| 5.4 | Multi-ref/CLI policy tests | Blocking and incomplete-coverage policies separate; real blocking qualification absent. |
| 5.5 | Result contracts, artifact tests and real timeout artifacts | Finding/coverage/push disposition remain separate. |
| 6.1 | Input validator and frozen profiles | BLOCKED: independent Node/PHP/Python reviewed populations missing. |
| 6.2 | Versioned runtime/qualification scorecards | Counts, intervals, undefined precision, unresolved cases and latency retained; gates fail. |
| 6.3 | Scorer controls and real VAmPI replay | Missing target/question/transitive proof remains unresolved in denominator. |
| 6.4 | Frozen provenance receipts, deployed evidence ranking mode | No comparative ranker retune claimed; fixed core preserved across PHP/Node. |
| 6.5 | Eligibility recomputation and NO-GO decisions | Joint gates enforced; no runtime/quality qualification earned. |
| 7.1 | Packaged network-disabled no-model runs | No endpoint, engine download, source fetch or project build required by hook. |
| 7.2 | Explicit/delayed/misleading-model controls | Redaction, model provenance and shared budget verified on controlled path. |
| 7.3 | Full raw bundles, graph/query cache identity tests | Input, selected/deferred questions, outcomes, exclusions and stage identities retained. |
| 7.4 | Hook tests; packaged installer/refusal/removal smoke | Explicit integration preserves hooks and configuration. |
| 8.1 | Equivalent-normalized-evidence ranking/admission controls | Shared core rules; no PHP/Node priority exceptions introduced. |
| 8.2 | Runtime smoke and missing-census controls; real Node/VAmPI gaps | Missing frontend/context evidence is not interpreted as safe. |
| 8.3 | Frozen six-case packaged PHP/Node profiles | Transfer attempted unchanged; all 30s cases time out. Useful transfer unproven. |
| 8.4 | Separate development/regression/envelope profiles | No core/fact/adapter retune; libpng arithmetic/bounds unsupported; no C++ detection claim. |

## Integration and boundary assessment

The implemented route remains Snapshot Adapter → ranker-owned scope → existing scan
engine/cache → delta/actionability admission → report/hook. Persistent eligibility
is a trusted installed input to the actionability boundary. It recomputes the existing
evaluation scorer and checks exact semantics; repository configuration and optional
models cannot supply eligibility. Immutable graph leases and answer compatibility are
kept separate from change attribution. No incremental graph mutation, second engine,
new IR, rank boost or downstream detector workaround was introduced in group 8.

Concrete remaining gaps are engine lifecycle/census completeness and context/witness
projection, not evidence that a new ranker would solve the measured failure. Shared
scan changes belong to contributor-scan and must trigger downstream replay revalidation.
The file structure extends the approved evaluation/admission boundary with
`push/eligibility.py` and its installed JSON resource. Ordinary scan behavior remains
covered by the full suite and canonical detector/map/pair gates.

## Required next work

1. In contributor-scan, reconcile named readable first-party files with the Node graph
   census, then measure/amortize frontend, overlay, census and query process startup.
   Preserve vendor exclusion and complete/corrupt cache distinctions.
2. Resolve supported first-party context and retain actual query-origin/path witnesses.
   Reproduce the three VAmPI targets with exact question accounting; a raw SQLI finding
   does not establish transitive reachability or push attribution.
3. Freeze independently reviewed Node API vulnerable/fixed/benign cases with the required
   middleware/module/async shapes; obtain separate PHP/Python qualification populations.
   Do not tune on reserved Ghost before its population is fixed.
4. Refreeze the candidate after owner fixes and rerun all seven runtime workload classes,
   cross-language regressions and independent actionable-alert review. Require the
   unchanged 30s warm p95, 2s cancellation allowance, 95% completion/observed precision,
   90% supported recall and zero known fixed/benign false alerts together.
5. Regenerate eligibility only for exact passing language/framework/family semantics,
   then rerun packaged validation. Keep normal capabilities disabled until then.

## Fresh verification evidence

- Full suite: `pytest -q`, **1317 passed, 9 skipped**, 115.80s, exit 0. No production
  source changed after this run; subsequent edits are validation tooling and records.
- Ruff check and formatting pass (262 files); mypy passes (116 source files);
  compileall exits 0. No new placeholder markers or concrete secrets in reviewed code.
- Detection regression gate: 44/47 recall, 0/44 false positives, PASS. Map gate: PASS.
  Local pair gate: 3/3 pair-correct, 28/28 labeled recall, no fixed-side leaks, PASS.
  These corpus controls do not qualify real pre-push language/framework capabilities.
- Docker build exits 0, image
  `4cca6d20c5c9bfdbfa60c4e8fc991e096bc9b2f35096bb92cc92440e99f946d4`.
- Packaged offline smoke exits 0 as UID 1000. Installed registry recomputes as NO-GO,
  zero enabled capabilities. Both real disposable hook acceptance/rejection and removal
  controls pass; exact stdin/remote args and prior state are verified.
- Final packaged six-case PHP/Node replay completes its instrument with exit 1 and
  an explicit NO-GO decision. All underlying advisory CLI invocations exit 0 and save
  partial artifacts; all six time out in 30.560–30.614s. Every snapshot has positive
  byte receipts. No target check or real actionable alert is qualified. Preparation
  timeout provenance remains unavailable where it was not measured.
- Versioned record: `benchmarks/measurements/2026-09-13-push-packaged-validation.json`.
  Its 38-file raw bundle is 827,483 bytes, SHA-256
  `ad1bc15791636868a65a5cbb93b84d1a08dc6c94be0eb2f8f03890936720cc79`.
  Decompression matches every retained original. Full logs, smoke artifacts, profile,
  replay pins, scorecard and six complete partial-result artifacts are included.

## Verification Result
- STATUS: NOT_VERIFIED
- CLAIM_TYPE: FEATURE_GO
- CLAIM: Pre-push safety net ready for rollout.
- EVIDENCE: Passing integration controls coexist with measured runtime/completion failures and missing quality prerequisites.
- GAPS: Task 8.3 qualification; real independent positive/fixed/benign evidence; verified VAmPI transitive witness; useful changed-code latency and coverage.
- NOTES: Keep the feature in implementation and capabilities disabled. Follow owner remediation above, then rerun qualification and packaged validation.