Marius CiepluchandClaude Fable 5.1 fbc3220bbe feat(learn): A/B experiment runner -- registered manifests, paired arms, bootstrap and McNemar (decision-engine task 7) ...

learn/experiments.py: YAML manifests under plane/experiments/<id>.yaml,
digest-checked at register, run and analyse time; `experiment`,
`arm_outcome` and `experiment_result` rows; paired arms in a seeded
repository order with a per-unit coin; arm A replay-only at $0; canary
replicates under a fresh cache salt; repository-cluster paired bootstrap,
exact McNemar, two O'Brien-Fleming looks, adopt/reject/equivalent/
inconclusive. `ousast learn experiment register|units|run|analyse`.
Program: `ProgramSpec.sampling = retrieval_ensemble` (k samples at
temperature 0 over k disjoint retrieved example sets; default unchanged);
`Caller.salt`; `retrieve(exclude=)`. verify: OUSAST_VERIFY_CALLERS=off.
Manifests exp-001-known-callers (plane arms, units pending) and
exp-002-retrieval-ensemble (slice-2 incumbents, ceiling $4) -- not run.
18 scripted tests; module audit regenerated (151 modules).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-10-02 16:04:33 +02:00

..

archive

chore(bench): archive reference-only scripts

2026-10-02 08:07:35 +02:00

dev

chore(dev): token-lean workflow tools -- read guard, bulk reader, transcript report

2026-09-29 17:23:30 +02:00

fixtures

feat(ruleset): PHP quick-mode rules from the engine's vocabulary, measured on the development corpus

2026-09-30 15:05:47 +02:00

independent

bench(independent): OpenCVE case licence checked (BUSL-1.1); published by maintainer decision

2026-10-02 09:12:22 +02:00

learn

fix(learn): withhold label-tracking signals from the v1 classifier; leak audit

2026-10-01 19:40:58 +02:00

manifests

feat(ruleset): PHP quick-mode rules from the engine's vocabulary, measured on the development corpus

2026-09-30 15:05:47 +02:00

measurements

feat(learn): A/B experiment runner -- registered manifests, paired arms, bootstrap and McNemar (decision-engine task 7)

2026-10-02 16:04:33 +02:00

pairs

bench(pairs): real fix-commit pairs for the short families

2026-10-01 11:51:28 +02:00

push

perf(taint): ask the caller check only of a parameter that starts a flow into the field

2026-09-23 22:49:44 +02:00

ranking

feat(pre-push-safety-net): preserve uncertainty in benchmark scoring

2026-09-12 19:04:20 +02:00

repos

feat(pre-push-safety-net): pin labeled evaluation inputs

2026-09-12 18:45:54 +02:00