Files
Open_Ultra_SAST/benchmarks/measurements
Marius CiepluchandClaude Opus 5.5 cabdf81ece learn: trained classifier arm (logistic and gradient boosting on embeddings + signals, per-fold leak screening); fit-eval record
On the 501 exp-005 units, repo-held-out folds: logistic pooled within-pair AUC
0.721 vs the LLM programs 0.762 (diff -0.041, CI [-0.116, 0.032], a tie);
gradient boosting 0.597 (significantly worse). Per family all differences span
0. Implemented with Codex; fit run by the coordinator (4.2 h on the VM).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 07:47:42 +02:00
..