spec(pre-push-safety-net): amend the sequence and declare the shippable scope

Four decisions, recorded after the family detection census and the PHP correction. They change
sequence and declared scope; they change no acceptance threshold and no owner boundary.

Close M3 at a scale whose questions complete, before anything else. The not-established PHP record
came from two instrument defects rather than from the analyser, and bounded to 25 and 100 regions
PMPro completes with no timeout and establishes a true-positive SQL injection. Tasks 13.3 and 13.4
carry the replay and the question-completion gap, the second because admission reads outcomes and
M4 measures completion at 95 percent.

Reserve the untouched M4 population now, in parallel. M4 already permits preparation before the
frozen candidate exists and it is the longest lead-time item on the path. Task 15.1.

M2 is the only engineering item left and cannot be scoped away: a changed-code transaction spends
its whole budget in the build, and meeting the latency gate by analysing less is explicitly not
the same result.

Declare the v1 advisory scope as the families whose claims can be ADMITTED, which requires the
entailed rung: injection, untrusted destination and configuration. Access control is excluded
because on an application that splits route from data access the obligation is carried and a
carried obligation corroborates by design, so the family would ship as silence. Task 15.2.

The risk the amendment does not remove is stated with it: every number so far comes from subjects
used to build the analyser, one week of examining two of them surfaced seven false positives, and
the first independent evaluation should be expected to fail.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-21 10:41:25 +02:00
co-authored by Claude Opus 5
parent 1f51c82b51
commit 5b7288aa58
3 changed files with 98 additions and 16 deletions
@@ -8,16 +8,51 @@ It does not approve an unreviewed implementation design or enable any capability
## Current position
**Next milestone: M1b. M1a is met; none of M1–M5 is verified yet.** The 28/29 completed
executable tasks measure implementation progress, not release readiness. Task 8.3 remains blocked.
The published `v1.2.0-alpha.1` is an experimental implementation release, not a qualified
advisory rollout. See [current status](status.md) for measured evidence and limitations.
**Next milestone: M3, with M4 population preparation running in parallel. M1a and M1b are met;
M2 is measured NO-GO; none of M1–M5 is verified.** The published `v1.2.0-alpha.1` is an
experimental implementation release, not a qualified advisory rollout. See
[current status](status.md) for measured evidence and limitations.
The immediate goal is one real changed-code defect, a defensible explanation, and a
fixed twin that stays quiet. M1a first shows the raw detection with every admission veto
recorded rather than applied; M1b repairs only the vetoes that fired. A longer, explicitly
named development budget is acceptable for M1; it cannot satisfy the hook's latency gate.
Runtime optimization is the next milestone after this useful result is demonstrated.
The immediate goal was one real changed-code defect with a defensible explanation and a quiet
fixed twin. That is met. Task 8.3 remains blocked.
## Strategy amendment, 2026-09-21
Accepted after the family detection census and the PHP correction. Four decisions, in the order
they should be executed. They change sequence and declared scope; they change no acceptance
threshold and no owner boundary.
1. **Close M3 at a scale that completes, before anything else.** PHP transfer is recorded as not
established, and that record came from two instrument defects rather than from the analyser:
a census verdict that checked question completion before findings, and a region budget whose
question count cannot fit any ceiling. Bounded to 25 and 100 regions, Paid Memberships Pro
completes with no timeout and establishes a true-positive SQL injection of the class its own
fact table predicted. Re-run the declared transfer replay at a completing scale and let the
result stand either way.
2. **Reserve and review the untouched M4 population now, in parallel.** M4 already permits
population preparation to proceed before the frozen candidate exists, and it is the longest
lead-time item on this path: nothing can be qualified without it however good the analyser
gets. Starting it after M2 serialises the release for no reason.
3. **M2 is the only engineering item that remains, and it cannot be scoped away.** A changed-code
transaction spends its whole budget in the build with zero query time. Graph reuse across
revisions is the accepted direction. Narrowing what a push analyses is explicitly not a way to
meet the latency gate, so the work is architectural and should have the effort that family
vocabulary work has been absorbing.
4. **Declare the v1 advisory scope as the families whose claims can be ADMITTED.** Admission
requires the entailed rung. Injection, untrusted destination and configuration entail with
witnesses; access control does not, because on any application that splits its route from its
data access the obligation is carried and a carried obligation corroborates by design. Shipping
it would ship silence. Output encoding, path, prototype and deserialization have not earned a
place either. M4 explicitly does not require every language or family to become eligible.
**The risk this amendment does not remove.** Every number so far comes from two repositories used
to build the analyser, and one week of examining them surfaced seven false positives. On untouched
code the precision estimate will move, against a bar of 95% with zero false alerts on the fixed and
benign sides. Expect the first M4 evaluation to fail and budget for a second. That is the strongest
argument for decision 2.
## M1 — One real security change works end to end
@@ -234,10 +269,10 @@ remaining blocker and next concrete action. Report implementation task counts se
Change a milestone to verified only with linked evidence; do not infer progress from
elapsed effort or a completed experimental release tag.
Immediate next action (M1b): repair the five vetoes M1a recorded, smallest first, starting
with operation correspondence for an operation on a changed line, since that one survives
every other repair. Then rerun the same three revisions without the flag through the
production evidence and comparison rules. Use
[M1a evidence](m1a-recorded-vetoes.md) and
[remediation research](../contributor-scan/release-remediation-research.md) as the starting
record; do not begin another broad timing campaign before the M1 outcome exists.
Immediate next action (M3, task 13.3): re-run the declared PHP transfer replay bounded to a
region budget whose questions complete, using the corrected census verdict, and record the
outcome either way. Then task 13.4, PHP question completion, because admission reads outcomes
and M4 measures completion at 95%. Task 15.1 (M4 population reservation) runs in parallel and
blocks on nothing. Do not begin another family vocabulary campaign: the census has said which
families can speak, and the constraint has moved to runtime and to evidence from code this
project has not seen.
+15
View File
@@ -223,3 +223,18 @@ Evidence: [family detection census](../../../benchmarks/measurements/2026-09-19-
[every permissive setting](../../../benchmarks/measurements/2026-09-20-every-permissive-setting.json),
[PHP runtime after the graph fix](../../../benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json),
[PHP detection corrected](../../../benchmarks/measurements/2026-09-20-php-detection-corrected.json).
## Strategy, 2026-09-21
Sequence amended after the family census and the PHP correction: close M3 at a completing scale
first (task 13.3), fix PHP question completion because admission reads outcomes (13.4), reserve
the untouched M4 population in parallel starting now (15.1), and give the remaining engineering to
M2 graph reuse, which cannot be scoped away because meeting the latency gate by analysing less is
explicitly not the same result. The v1 advisory scope is declared as the families whose claims can
be admitted at the entailed rung (15.2): injection, untrusted destination and configuration.
Access control is excluded because a carried obligation corroborates by design and would ship as
silence.
No acceptance threshold and no owner boundary changes. The risk that remains is that every number
so far comes from subjects used to build the analyser, and one week of examining two of them
surfaced seven false positives; the first independent evaluation should be expected to fail.
+32
View File
@@ -554,6 +554,38 @@ Preserve all completed task evidence below.
- _Requirements: 6.2, 8.1_
- [ ] 13.3 Re-run the declared PHP transfer at a scale whose questions complete
- **Named by the 2026-09-20 correction.** The not-established record came from two instrument defects, not from the analyser: the census verdict checked question completion before findings, and a 500-region budget asks 2,526 questions, which no ceiling this project sets can hold. Bounded to 25 and 100 regions, PMPro completes with no timeout and establishes a true-positive SQL injection at `adminpages/discountcodes.php:749`.
- Replay the three declared comparisons with the corrected harness at a region budget that completes, and record the declared target's evidence or its absence. State the budget as a limit of the run, not as a property of the repository: a bounded population establishes transfer, never coverage.
- Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 13.2_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 13.4 Complete a PHP question, or say which scope it could not reach
- **Named by the 2026-09-20 correction.** `questions_completed` is zero in every PHP run, including the runs that establish findings, with `cross_partition_semantics_unresolved` and `partition_file_census_incomplete` present throughout. Findings and question outcomes therefore disagree in public, and the admission policy reads the outcomes rather than the findings.
- Group 12 raised completion from 205 to 298 of 436 for JavaScript and answered every VAmPI question; PHP was never measured. The partition and shard boundaries are the first place to look, because those are the degradations every run carries.
- Done when a PHP question whose scope the graph holds completes, a question whose scope it cannot reach stays unresolved with the reason named, and no question completes by widening its scope.
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
- _Depends: 13.3_
- _Requirements: 2.1, 2.2, 6.3, 8.1_
- [ ] 15. Prepare the independent population and declare the shippable scope (M4 preparation)
- Runs in parallel with M2 and M3. M4 already permits population preparation before the frozen candidate exists, and it is the longest lead-time item on the path: nothing can be qualified without it however good the analyser gets.
- [ ] 15.1 Reserve and review an untouched evaluation population
- Freeze reviewed vulnerable, fixed and benign changes from projects not used to develop or tune this analyser. NodeGoat, VAmPI, PMPro, WP Statistics, MW WP Form and libpng are all development subjects and none of them may qualify anything.
- Reserve the cases untouched: do not examine their scan results before the population is frozen, and record the reservation so a later run cannot be mistaken for a blind one.
- Done when a frozen population exists with its provenance and review recorded, and nothing in it has been scanned.
- _Boundary: Evaluation harness, corpus provenance_
- _Requirements: 8.1, 8.2_
- [ ] 15.2 Declare the v1 advisory scope as the families whose claims can be admitted
- **Admission requires the entailed rung.** Injection, untrusted destination and configuration entail with witnesses on the measured subjects. Access control does not: on an application that splits its route from its data access the obligation is carried, and a carried obligation corroborates by design, so shipping the family would ship silence. Output encoding, path, prototype and deserialization have not earned a place.
- Record the scope, the reason each excluded family is excluded, and what a contributor is therefore NOT told. M4 does not require every language or family to become eligible, and a published limitation is the honest form of a narrow release.
- Done when one record states the declared scope and every exclusion's reason, and the packaged advisory output names the limits it ships with.
- _Boundary: pre-push-safety-net admission, release validation_
- _Depends: 15.1_
- _Requirements: 3.1, 5.1, 5.6, 7.1_
## Implementation Notes
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.