mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-21 14:09:27 +00:00
feat(plane): tie-break pass on disputed candidates; 2-of-3 agreement
The design revisit after plane-increment-1.json (14/20 agreed, 19/20 in either
pass): a third verify pass on the candidates passes a and b dispute, and agree
decides 2-of-3 where pass c asked (the a/b rule elsewhere; c alone never decides).
- verify: optional OUSAST_INPUT_ONLY restricts the hunt to the named candidates
(agree's disputed.json); an empty list is done with 0 units and no model client.
- agree: optional OUSAST_INPUT_PASS_C; metrics keep every key and add tiebreak
{asked, flipped_to_agreed, flipped_to_rejected, rule}; usd_total, hunts and
declared-site metrics are on the final decision.
- generate: per case <case>-vc and <case>-final appended to validation-46; the
first 60 entries stay byte-identical, so a rerun keeps their done state.
- Offline replay over the recorded outputs with an empty pass c reproduces the
recorded agree summaries exactly (14/20 agreed, 23 agreed, 10 disputed).
- Spec: Requirement 6.4, design Measurement, task 8.1.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
c5e6ed3221
commit
6e6bcfc40c
@@ -132,6 +132,9 @@ before the remaining concerns are converted.
|
||||
current $0.022 for two passes; otherwise the increment is reported as not met and the design revisited.
|
||||
3. The development transcript measure (`benchmarks/dev/token_report.py`) is run at the end of the increment and
|
||||
recorded next to the result.
|
||||
4. After the first measurement (plane-increment-1.json: 14/20 agreed, 19/20 in either pass), agreement is decided
|
||||
2-of-3 with a third pass run only on candidates passes a and b dispute; the gate of 6.2 is re-evaluated on that
|
||||
decision, and the result is recorded as plane-increment-2.json next to the first.
|
||||
|
||||
### Requirement 7: Routing is a declared concern of the plane
|
||||
|
||||
|
||||
Reference in New Issue
Block a user