mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 20:19:29 +00:00
Marius CiepluchandClaude Opus 5
78f20df167
The learning harness built the measurement honestly and then hill-climbed on it by hand. Four review rounds' worth of defects in that loop were defects of its shape, not its code. Read from GEPA's Algorithm 1 and 2 rather than from a summary, three things bear directly on ours: the acceptance gate is deliberately cheap and noisy, because the Pareto frontier -- not the gate -- is what protects against a bad accept; selection is stochastic and diversity-preserving, which is what stops the search collapsing onto one lineage; and the expensive full evaluation is spent only on candidates that already passed the cheap gate. Two consequences for us are honesty findings, not performance ones. Our acceptance rule reads the holdout every round, so repeated rounds overfit the test set -- the same leak class this project already measured once, one level up. And the proposer is handed a score where the published work hands it textual feedback from the evaluation trace, which our evaluator produces and then discards. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
27 lines
756 B
JSON
27 lines
756 B
JSON
{
|
|
"feature_name": "detector-optimizer",
|
|
"created_at": "2026-09-06T14:30:00Z",
|
|
"updated_at": "2026-09-06T14:30:00Z",
|
|
"language": "en",
|
|
"phase": "brief",
|
|
"approvals": {
|
|
"requirements": {
|
|
"generated": false,
|
|
"approved": false
|
|
},
|
|
"design": {
|
|
"generated": false,
|
|
"approved": false
|
|
},
|
|
"tasks": {
|
|
"generated": false,
|
|
"approved": false
|
|
}
|
|
},
|
|
"ready_for_implementation": false,
|
|
"depends_on": [
|
|
"learning-harness"
|
|
],
|
|
"approval_note": "2026-09-06 brief drafted at the maintainer's direction ('I'd rather spend time to design the exact DSPy style interface and evolutionary / statistical prompt optimization than fixing piece by piece'). Not approved; requirements not generated."
|
|
}
|