mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 12:09:27 +00:00
Marius CiepluchandClaude Opus 5
95ffd901e4
Task 2.4. Per family, and per slice when asked: outcome counts, recall, silence, Youden, a directional-bias index and recall at a fixed false-positive ceiling. Rows that cannot be scored are listed by reason and leave every denominator, and a family with nothing scorable reports no rate rather than a perfect one. Two numbers exist because Youden cannot tell apart a detector that flags everything from one that says everything is safe: the directional-bias index separates them, and recall at a fixed ceiling is forfeited entirely once a family leaks more than the budget allows. Change against a baseline is not a difference of means. Flips are counted per pair in both directions and a two-sided exact sign test decides whether the change is bigger than the noise, so one flip in ten reads as noise and six reads as a real regression. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
31 lines
985 B
JSON
31 lines
985 B
JSON
{
|
|
"feature_name": "learning-harness",
|
|
"created_at": "2026-09-06T06:32:41Z",
|
|
"updated_at": "2026-09-06T07:00:52Z",
|
|
"language": "en",
|
|
"phase": "implementation",
|
|
"approvals": {
|
|
"requirements": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"design": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"tasks": {
|
|
"generated": true,
|
|
"approved": true
|
|
}
|
|
},
|
|
"ready_for_implementation": true,
|
|
"depends_on": [
|
|
"pair-corpus-honesty",
|
|
"corpus-seeded-mechanisms",
|
|
"authorization-obligations",
|
|
"tree-sitter-overlay-extra",
|
|
"harnessx-self-improving-rulesets"
|
|
],
|
|
"approval_note": "2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase'); 2026-09-06 design approved by the maintainer in session ('approve the design and generate the tasks'); 2026-09-06 tasks approved by the maintainer in session ('approve the tasks and go on with the implementation')"
|
|
}
|