docs(learning-harness): 5.3 targets injection, and 5.2 waits on the key

Measured on the corpus under the split and review-tier rules, access_control
has 1 train teacher and 4 holdout pairs: every agent-vfc row sits at tier
`title` and the vibe-py absence rows that could teach are trap pairs whose
two sides are different handlers, so the train split teaches zero obligation
shapes. injection has 12 teachers and 10 holdout pairs, the only family with
enough train evidence to tell a real change from noise.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-06 14:03:35 +02:00
co-authored by Claude Opus 5
parent 2dad25d312
commit 3acad97297
+3 -1
View File
@@ -164,11 +164,13 @@
- [ ] 5.2 Live round zero
- Round zero through the DeepSeek endpoint on the vibe-py and agent-vfc web families with K = 5, plus the memory-safety family over the vfc slice as a measured, non-gating run; artifacts, noise floors and cost per pair committed; the classifier report and review queue produced on the same run; a second baseline with the judge model on one slice for the model comparison table.
- Observable: the published table shows per-family outcomes, noise floors and cost for both models and for the vfc memory-safety family, and the roadmap section is regenerated from it.
- **Blocked 2026-09-06: no `DEEPSEEK_API_KEY` in `.env` or the environment.** Everything else is ready; the run is one command per baseline. The maintainer chose to add the key and have the full run go ahead (web families with `deepseek-v4-flash`, the vfc memory slice as a measured non-gating run, and a second baseline with `deepseek-v4-pro` on one slice). Scale, measured on the corpus: 58 scorable web pairs plus 176 vfc pairs, both sides, K=5 = 2,340 detector runs at up to four model calls each; the design estimated under $15 at peak flash pricing.
- _Depends: 3.9, 4.1_
- _Requirements: 8.3, 8.4, 11.1, 11.2_
- [ ] 5.3 First evolve rounds
- Three rounds with the scripted proposer on the access-control family using the obligation shapes as the first memory entries, then one round with the HarnessX proposer; every round journaled with attribution; the roadmap regenerated.
- **Amended 2026-09-06 (maintainer, in session): the target family is `injection`, not `access_control`.** Measured on the corpus under the split and review-tier rules: access_control has **1 train teacher and 4 holdout pairs**, six of its rows unusable because every agent-vfc row sits at tier `title`, and the vibe-py absence rows that could teach are Real-Vuln-Benchmark trap pairs whose two sides are different handlers — so the train split teaches **zero** obligation shapes and any accepted change would rest on one pair. `injection` has 12 train teachers and 10 holdout pairs, the only family in the corpus with enough train evidence to tell a real change from noise. The first memory entries are therefore the mechanism shapes the train split really teaches rather than obligation shapes, which do not exist yet. access_control follows when its corpus can teach: either a maintainer reviews agent-vfc rows up to `reviewed`, or the trap rows gain non-trap twins.
- Three rounds with the scripted proposer on the injection family using the shapes the train split teaches as the first memory entries, then one round with the HarnessX proposer; every round journaled with attribution; the roadmap regenerated.
- Observable: the journal holds four records with outcomes and attribution precision, at least one accepted change is versioned in its family directory, and no other family exceeded its budget in any accepted round.
- _Depends: 5.2_
- _Requirements: 9.9_