bench(learn): exp-002 result -- retrieval ensemble vs temperature sampling

Reject B. 160 paired units (injection 119, access_control 41; 56 repositories),
both arms, two replicates each, no early stop. Canary agreement A 0.875 -> B
0.906 (CI [-0.03, +0.10], spans; B meets 0.90). Within-pair AUC A 0.815 -> B
0.741 (CI [-0.13, -0.02], against B). AUC -0.02 (against), ADVISORY recall
-0.06 and precision +0.01 (span), usd per candidate A $0.0021 -> B $0.0051
(against). Priced usage A $0.68 / B $1.63; balance $16.72 -> $16.04 at the
run's end (lags). Counts only; task 7.4 marked done.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 19:05:34 +02:00
co-authored by Claude Fable 5.1
parent 780caaa558
commit 48510fd5c2
4 changed files with 631 additions and 3 deletions
@@ -19,5 +19,5 @@
"note": "frozen on the 937-example static memory (slice 2 evaluated 110 + 37 candidates on the 904-example memory of the same day; the recovered negatives of ef6945e moved the seeded compile split and the outer folds). Freezing needed two repairs first: example rows carried the plane-only features as null in x and the renderer refused them (1f525ff), and the RustFS store held none of the 788 embedding vectors the file store had (copied over with put_blob, no embedding call)."
},
"first_registration": {"registered": "2026-10-02", "pin": "fbc3220bbe1c8a4f8afa0a724fd41a51b390dd22", "units_status": "pending"},
"status": "registered, units frozen; run pending"
"status": "run and analysed 2026-10-02: reject (result.json; benchmarks/measurements/2026-10-02-exp-002/record.json)"
}