mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(hunt): proposals and the acceptance rule (5.1, 5.2)
propose.py: one change per round (instruction, demonstrations or one mask), examples built from compile folds only with a provenance record of the change ids used, a registry for GEPA, MIPROv2 and BootstrapFewShot (MIPROv2 without the data-aware proposer so labels are never serialised), compile-fold scoring with Wilson and the repository-paired bootstrap, and a compile-only history the next round may read. accept.py: the rule as independent clauses (both folds non-negative with one strictly positive, per-family regression budget on held-out recall, tie tolerance, no acceptance without an interval, instruction cap, one change only), the comparison recorded in the candidate artifact's configurations, rejections logged without held-out numbers. 48 fixture tests. Known gap for live rounds: the optimisers and propose._score call the program directly, so a tool-using program must be scored through the runner with its tools bound per checkout (6.1). Implemented by Codex (gpt-6-astra), gated here. Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
47d0f0fac1
commit
bacd7a1e3f
@@ -77,12 +77,12 @@
|
||||
- Done when a fixture run is marked contaminated after its repository is added to a report list
|
||||
- _Requirements: 2.7_
|
||||
|
||||
- [ ] 5. Core: proposals and acceptance
|
||||
- [ ] 5.1 Propose candidates with the program runtime's optimisers
|
||||
- [x] 5. Core: proposals and acceptance
|
||||
- [x] 5.1 Propose candidates with the program runtime's optimisers
|
||||
- One change per round: an instruction proposal, a demonstration set, or a mask; proposals score on compile folds with the search metric; the proposer never sees held-out results; rejected candidates stay visible to the next round
|
||||
- Done when a round on a fixture produces a candidate with its compile-fold score and the proposer's inputs are shown to exclude held-out data
|
||||
- _Requirements: 5.4, 6.6, 6.8_
|
||||
- [ ] 5.2 Accept or reject by the rule
|
||||
- [x] 5.2 Accept or reject by the rule
|
||||
- Compile-fold and held-out results per family; accepted only when both are non-negative, one strictly positive, no family below its regression budget, instruction under the length cap; ties rejected; the comparison is recorded in the new artifact
|
||||
- Done when every clause of the rule has a synthetic fold-result test that flips the decision
|
||||
- _Requirements: 5.6, 6.1, 6.2, 6.3, 6.4, 6.5, 6.7_
|
||||
|
||||
Reference in New Issue
Block a user