Marius CiepluchandClaude Opus 5.5 dd7ebc119b learn(exp-003): decision-rule experiments and the precision-bound BLOCK arm ...

kind: rule experiments score once from the response cache behind a
zero-ceiling meter (a miss is excluded and counted, never asked) and apply
each arm's BLOCK rule to the same held-out scores. Arm B chooses a per-family
raw-score threshold on the other outer folds only (lowest t with Wilson 95%
LB >= 0.95); a test fails if a fold's own labels move its threshold.
Manifest and frozen units (353 units, 146 groups, 244 paired) for exp-003.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-10-03 08:02:51 +02:00

..

exp-001-known-callers.yaml

feat(learn): A/B experiment runner -- registered manifests, paired arms, bootstrap and McNemar (decision-engine task 7)

2026-10-02 16:04:33 +02:00

exp-002-retrieval-ensemble.units.jsonl

bench(learn): exp-002 units frozen and registered

2026-10-02 17:27:26 +02:00

exp-002-retrieval-ensemble.yaml

feat(learn): A/B experiment runner -- registered manifests, paired arms, bootstrap and McNemar (decision-engine task 7)

2026-10-02 16:04:33 +02:00

exp-003-precision-bound-blocking.units.jsonl

learn(exp-003): decision-rule experiments and the precision-bound BLOCK arm

2026-10-03 08:02:51 +02:00

exp-003-precision-bound-blocking.yaml

learn(exp-003): decision-rule experiments and the precision-bound BLOCK arm

2026-10-03 08:02:51 +02:00