propose.py: one change per round (instruction, demonstrations or one mask), examples built from
compile folds only with a provenance record of the change ids used, a registry for GEPA, MIPROv2
and BootstrapFewShot (MIPROv2 without the data-aware proposer so labels are never serialised),
compile-fold scoring with Wilson and the repository-paired bootstrap, and a compile-only history
the next round may read. accept.py: the rule as independent clauses (both folds non-negative with
one strictly positive, per-family regression budget on held-out recall, tie tolerance, no
acceptance without an interval, instruction cap, one change only), the comparison recorded in the
candidate artifact's configurations, rejections logged without held-out numbers. 48 fixture tests.
Known gap for live rounds: the optimisers and propose._score call the program directly, so a
tool-using program must be scored through the runner with its tools bound per checkout (6.1).
Implemented by Codex (gpt-6-astra), gated here.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
contamination.py reads the repositories named by the held-out pilot manifest and by every
benchmark report (accepted_repos.json and NOTES.md links under benchmarks/measurements), marks
each run directory clean or contaminated with the report digest and time, and refuses to count
an unchecked run toward a gain. ablation.analyse carries excluded_from_gain when its run is
contaminated. Implemented by Codex (gpt-6-astra), gated here; the benchmark import is lazy so
the package imports without the benchmarks tree.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
ablation.py registers the manifest (table digest, arms as masks, folds, metric, sample, budget,
rule, seed, program id) before any run and refuses a run or an analysis that does not match it.
Arms run over the same changes through runner.run, paired by change; effects on recall and on
suspicions per ordinary push are reported pooled and per family with the repository-clustered
bootstrap and exact McNemar, plus the recall-against-suspicions curve at every reported
confidence (never a tuned cut). The importance table and decisions read held-out effects only;
a neutral annotation survives only when a kept one needs it. 30 fixture tests with a scripted
model. Implemented by Codex (gpt-6-astra), gated here.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Six groups: table and producers; the hunter as a program with the search
metric and the run artifact; runner integration and a baseline on the 12
pilot positives; training set, folds, paired ablation and the importance
table; proposals and the acceptance rule; the pilot gain report.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
