From 5b7288aa58dee0c2707495d944d554569074deb5 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Mon, 21 Sep 2026 10:41:25 +0200 Subject: [PATCH] spec(pre-push-safety-net): amend the sequence and declare the shippable scope Four decisions, recorded after the family detection census and the PHP correction. They change sequence and declared scope; they change no acceptance threshold and no owner boundary. Close M3 at a scale whose questions complete, before anything else. The not-established PHP record came from two instrument defects rather than from the analyser, and bounded to 25 and 100 regions PMPro completes with no timeout and establishes a true-positive SQL injection. Tasks 13.3 and 13.4 carry the replay and the question-completion gap, the second because admission reads outcomes and M4 measures completion at 95 percent. Reserve the untouched M4 population now, in parallel. M4 already permits preparation before the frozen candidate exists and it is the longest lead-time item on the path. Task 15.1. M2 is the only engineering item left and cannot be scoped away: a changed-code transaction spends its whole budget in the build, and meeting the latency gate by analysing less is explicitly not the same result. Declare the v1 advisory scope as the families whose claims can be ADMITTED, which requires the entailed rung: injection, untrusted destination and configuration. Access control is excluded because on an application that splits route from data access the obligation is carried and a carried obligation corroborates by design, so the family would ship as silence. Task 15.2. The risk the amendment does not remove is stated with it: every number so far comes from subjects used to build the analyser, one week of examining two of them surfaced seven false positives, and the first independent evaluation should be expected to fail. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo --- .../pre-push-safety-net/release-milestones.md | 67 ++++++++++++++----- .kiro/specs/pre-push-safety-net/status.md | 15 +++++ .kiro/specs/pre-push-safety-net/tasks.md | 32 +++++++++ 3 files changed, 98 insertions(+), 16 deletions(-) diff --git a/.kiro/specs/pre-push-safety-net/release-milestones.md b/.kiro/specs/pre-push-safety-net/release-milestones.md index c44a34c..d4295d0 100644 --- a/.kiro/specs/pre-push-safety-net/release-milestones.md +++ b/.kiro/specs/pre-push-safety-net/release-milestones.md @@ -8,16 +8,51 @@ It does not approve an unreviewed implementation design or enable any capability ## Current position -**Next milestone: M1b. M1a is met; none of M1–M5 is verified yet.** The 28/29 completed -executable tasks measure implementation progress, not release readiness. Task 8.3 remains blocked. -The published `v1.2.0-alpha.1` is an experimental implementation release, not a qualified -advisory rollout. See [current status](status.md) for measured evidence and limitations. +**Next milestone: M3, with M4 population preparation running in parallel. M1a and M1b are met; +M2 is measured NO-GO; none of M1–M5 is verified.** The published `v1.2.0-alpha.1` is an +experimental implementation release, not a qualified advisory rollout. See +[current status](status.md) for measured evidence and limitations. -The immediate goal is one real changed-code defect, a defensible explanation, and a -fixed twin that stays quiet. M1a first shows the raw detection with every admission veto -recorded rather than applied; M1b repairs only the vetoes that fired. A longer, explicitly -named development budget is acceptable for M1; it cannot satisfy the hook's latency gate. -Runtime optimization is the next milestone after this useful result is demonstrated. +The immediate goal was one real changed-code defect with a defensible explanation and a quiet +fixed twin. That is met. Task 8.3 remains blocked. + +## Strategy amendment, 2026-09-21 + +Accepted after the family detection census and the PHP correction. Four decisions, in the order +they should be executed. They change sequence and declared scope; they change no acceptance +threshold and no owner boundary. + +1. **Close M3 at a scale that completes, before anything else.** PHP transfer is recorded as not + established, and that record came from two instrument defects rather than from the analyser: + a census verdict that checked question completion before findings, and a region budget whose + question count cannot fit any ceiling. Bounded to 25 and 100 regions, Paid Memberships Pro + completes with no timeout and establishes a true-positive SQL injection of the class its own + fact table predicted. Re-run the declared transfer replay at a completing scale and let the + result stand either way. + +2. **Reserve and review the untouched M4 population now, in parallel.** M4 already permits + population preparation to proceed before the frozen candidate exists, and it is the longest + lead-time item on this path: nothing can be qualified without it however good the analyser + gets. Starting it after M2 serialises the release for no reason. + +3. **M2 is the only engineering item that remains, and it cannot be scoped away.** A changed-code + transaction spends its whole budget in the build with zero query time. Graph reuse across + revisions is the accepted direction. Narrowing what a push analyses is explicitly not a way to + meet the latency gate, so the work is architectural and should have the effort that family + vocabulary work has been absorbing. + +4. **Declare the v1 advisory scope as the families whose claims can be ADMITTED.** Admission + requires the entailed rung. Injection, untrusted destination and configuration entail with + witnesses; access control does not, because on any application that splits its route from its + data access the obligation is carried and a carried obligation corroborates by design. Shipping + it would ship silence. Output encoding, path, prototype and deserialization have not earned a + place either. M4 explicitly does not require every language or family to become eligible. + +**The risk this amendment does not remove.** Every number so far comes from two repositories used +to build the analyser, and one week of examining them surfaced seven false positives. On untouched +code the precision estimate will move, against a bar of 95% with zero false alerts on the fixed and +benign sides. Expect the first M4 evaluation to fail and budget for a second. That is the strongest +argument for decision 2. ## M1 — One real security change works end to end @@ -234,10 +269,10 @@ remaining blocker and next concrete action. Report implementation task counts se Change a milestone to verified only with linked evidence; do not infer progress from elapsed effort or a completed experimental release tag. -Immediate next action (M1b): repair the five vetoes M1a recorded, smallest first, starting -with operation correspondence for an operation on a changed line, since that one survives -every other repair. Then rerun the same three revisions without the flag through the -production evidence and comparison rules. Use -[M1a evidence](m1a-recorded-vetoes.md) and -[remediation research](../contributor-scan/release-remediation-research.md) as the starting -record; do not begin another broad timing campaign before the M1 outcome exists. +Immediate next action (M3, task 13.3): re-run the declared PHP transfer replay bounded to a +region budget whose questions complete, using the corrected census verdict, and record the +outcome either way. Then task 13.4, PHP question completion, because admission reads outcomes +and M4 measures completion at 95%. Task 15.1 (M4 population reservation) runs in parallel and +blocks on nothing. Do not begin another family vocabulary campaign: the census has said which +families can speak, and the constraint has moved to runtime and to evidence from code this +project has not seen. diff --git a/.kiro/specs/pre-push-safety-net/status.md b/.kiro/specs/pre-push-safety-net/status.md index 7530bbb..7a648d7 100644 --- a/.kiro/specs/pre-push-safety-net/status.md +++ b/.kiro/specs/pre-push-safety-net/status.md @@ -223,3 +223,18 @@ Evidence: [family detection census](../../../benchmarks/measurements/2026-09-19- [every permissive setting](../../../benchmarks/measurements/2026-09-20-every-permissive-setting.json), [PHP runtime after the graph fix](../../../benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json), [PHP detection corrected](../../../benchmarks/measurements/2026-09-20-php-detection-corrected.json). + +## Strategy, 2026-09-21 + +Sequence amended after the family census and the PHP correction: close M3 at a completing scale +first (task 13.3), fix PHP question completion because admission reads outcomes (13.4), reserve +the untouched M4 population in parallel starting now (15.1), and give the remaining engineering to +M2 graph reuse, which cannot be scoped away because meeting the latency gate by analysing less is +explicitly not the same result. The v1 advisory scope is declared as the families whose claims can +be admitted at the entailed rung (15.2): injection, untrusted destination and configuration. +Access control is excluded because a carried obligation corroborates by design and would ship as +silence. + +No acceptance threshold and no owner boundary changes. The risk that remains is that every number +so far comes from subjects used to build the analyser, and one week of examining two of them +surfaced seven false positives; the first independent evaluation should be expected to fail. diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md index e65e0aa..7fceb14 100644 --- a/.kiro/specs/pre-push-safety-net/tasks.md +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -554,6 +554,38 @@ Preserve all completed task evidence below. - _Requirements: 6.2, 8.1_ +- [ ] 13.3 Re-run the declared PHP transfer at a scale whose questions complete + - **Named by the 2026-09-20 correction.** The not-established record came from two instrument defects, not from the analyser: the census verdict checked question completion before findings, and a 500-region budget asks 2,526 questions, which no ceiling this project sets can hold. Bounded to 25 and 100 regions, PMPro completes with no timeout and establishes a true-positive SQL injection at `adminpages/discountcodes.php:749`. + - Replay the three declared comparisons with the corrected harness at a region budget that completes, and record the declared target's evidence or its absence. State the budget as a limit of the run, not as a property of the repository: a bounded population establishes transfer, never coverage. + - Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding. + - _Boundary: Evaluation harness, full-run integration_ + - _Depends: 13.2_ + - _Requirements: 6.2, 6.3, 8.1, 8.2_ +- [ ] 13.4 Complete a PHP question, or say which scope it could not reach + - **Named by the 2026-09-20 correction.** `questions_completed` is zero in every PHP run, including the runs that establish findings, with `cross_partition_semantics_unresolved` and `partition_file_census_incomplete` present throughout. Findings and question outcomes therefore disagree in public, and the admission policy reads the outcomes rather than the findings. + - Group 12 raised completion from 205 to 298 of 436 for JavaScript and answered every VAmPI question; PHP was never measured. The partition and shard boundaries are the first place to look, because those are the degradations every run carries. + - Done when a PHP question whose scope the graph holds completes, a question whose scope it cannot reach stays unresolved with the reason named, and no question completes by widening its scope. + - _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_ + - _Depends: 13.3_ + - _Requirements: 2.1, 2.2, 6.3, 8.1_ + +- [ ] 15. Prepare the independent population and declare the shippable scope (M4 preparation) + - Runs in parallel with M2 and M3. M4 already permits population preparation before the frozen candidate exists, and it is the longest lead-time item on the path: nothing can be qualified without it however good the analyser gets. +- [ ] 15.1 Reserve and review an untouched evaluation population + - Freeze reviewed vulnerable, fixed and benign changes from projects not used to develop or tune this analyser. NodeGoat, VAmPI, PMPro, WP Statistics, MW WP Form and libpng are all development subjects and none of them may qualify anything. + - Reserve the cases untouched: do not examine their scan results before the population is frozen, and record the reservation so a later run cannot be mistaken for a blind one. + - Done when a frozen population exists with its provenance and review recorded, and nothing in it has been scanned. + - _Boundary: Evaluation harness, corpus provenance_ + - _Requirements: 8.1, 8.2_ +- [ ] 15.2 Declare the v1 advisory scope as the families whose claims can be admitted + - **Admission requires the entailed rung.** Injection, untrusted destination and configuration entail with witnesses on the measured subjects. Access control does not: on an application that splits its route from its data access the obligation is carried, and a carried obligation corroborates by design, so shipping the family would ship silence. Output encoding, path, prototype and deserialization have not earned a place. + - Record the scope, the reason each excluded family is excluded, and what a contributor is therefore NOT told. M4 does not require every language or family to become eligible, and a published limitation is the honest form of a narrow release. + - Done when one record states the declared scope and every exclusion's reason, and the packaged advisory output names the limits it ships with. + - _Boundary: pre-push-safety-net admission, release validation_ + - _Depends: 15.1_ + - _Requirements: 3.1, 5.1, 5.6, 7.1_ + + ## Implementation Notes - Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.