marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(learning-harness): record the offline dress rehearsal of round zero

Browse Source 
Round zero and one evolve round run end to end with the scripted client over
the whole vibe-py slice: per-family and per-slice blocks, twins out by
reason, a floor each, a proposal applied and reverted byte for byte, the
sweep reporting zero negative flips, the rejected version archived. The only
thing the live run adds is a detector that can see.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-06 14:06:46 +02:00
co-authored by Claude Opus 5
parent 32f16f78ff
commit 5fbb0494a7
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/learning-harness/tasks.md
+1
View File
@@ -165,6 +165,7 @@
- Round zero through the DeepSeek endpoint on the vibe-py and agent-vfc web families with K = 5, plus the memory-safety family over the vfc slice as a measured, non-gating run; artifacts, noise floors and cost per pair committed; the classifier report and review queue produced on the same run; a second baseline with the judge model on one slice for the model comparison table.
- Observable: the published table shows per-family outcomes, noise floors and cost for both models and for the vfc memory-safety family, and the roadmap section is regenerated from it.
- **Blocked 2026-09-06: no `DEEPSEEK_API_KEY` in `.env` or the environment.** Everything else is ready; the run is one command per baseline. The maintainer chose to add the key and have the full run go ahead (web families with `deepseek-v4-flash`, the vfc memory slice as a measured non-gating run, and a second baseline with `deepseek-v4-pro` on one slice). Scale, measured on the corpus: 58 scorable web pairs plus 176 vfc pairs, both sides, K=5 = 2,340 detector runs at up to four model calls each; the design estimated under $15 at peak flash pricing.
- Dress-rehearsed offline 2026-09-06 with the scripted client over the whole vibe-py slice at K=3: round zero produced six families with per-slice blocks, correct denominators (the twins out by reason), and a floor each; a round then read the recorded proposals, applied one, swept the other five families for negative flips (0 each), rejected on `tie`, reverted the family directory byte for byte and archived the rejected version's per-pair results. The only thing the live run adds is a detector that can actually see.
- _Depends: 3.9, 4.1_
- _Requirements: 8.3, 8.4, 11.1, 11.2_