Task 3.9. Five entry points make the classified detector set reachable:
classify a corpus and report the classifier against itself, score a catalog by
family, run round zero, run one evolve round, and publish every number from its
artifacts. The pairs command gains a runs-per-pair flag, since the family scorer
needs more than one.
Every step degrades with a reason rather than failing: with no chat endpoint the
run says so and still records what it could measure without a detector, and a
missing configuration directory is written from the current hunter prompt rather
than being an error. Artifacts are keyed by model throughout, so comparing a
second detector is a flag rather than a rewrite.
Both matrices are green and all three gate outputs stay byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 3.8. A number in a roadmap that nobody can regenerate is a claim, not a
measurement. One entry point now writes them all from the committed artifacts
and puts the denominator beside each: how many pairs could be scored, why the
rest could not, which taxonomy version scored them, the noise the family shows
against itself, and what the run cost per pair it got right. Every table says
whether hierarchical partial credit could apply, which today it cannot.
The roadmap section is replaced in place between markers rather than appended,
so the file cannot accumulate a stale number beside a fresh one, and running the
command twice without new artifacts changes nothing. With no baseline recorded
it says so instead of printing an empty table.
The corrected mechanism-lever number travels with the rest: detected five of
seventeen at a negative Youden once only train-split pairs teach, against seven
when every pair taught, with the difference named as what holdout pairs stopped
teaching. Round zero now records what it spent so the cost column has a source.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 3.7. A round proposes one change for one family and then lets the evidence
decide whether it stays. The stages run cheapest first: a train minibatch, then
the family's held-out pairs, then every other family. Each stage scores several
runs per pair, and asking for fewer than three is refused outright, because a
single run misses most real changes.
The family directory is snapshotted before the change, so every outcome other
than acceptance leaves the tree exactly as it was found: a tie, a regression on
either side, another family below its floor, a proposal that reaches beyond its
directory, or a round that crosses its cost cap. Acceptance bumps only that
family's version and records which configuration won which pair.
Every outcome is journalled, including having no proposal at all, and the round
directory holds the proposal, the scores, the pair identities behind the
attribution counts and the detector trajectories, all redacted, so a round can
be replayed and its attribution audited rather than taken on trust. A round
number is never reused.
Recorded while writing the sweep: with per-family directories a change cannot
reach another family, so the sweep is a floor check over every other family at
the moment this round would land, not a causal claim about this change.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 3.3. Before anything learns, every family gets the same detector and is
measured against itself. Running a pair several times at the same settings does
not give the same answer, because the chat provider offers no seed, so the
spread between runs is real and unavoidable. That spread is the family's noise
floor and it becomes the budget a later round must stay inside before a
cross-family change counts as a regression. Choosing a budget instead of
measuring one would be a guess dressed as a threshold.
A pair the corpus already marked unscorable is never run: spending model calls
on a row that cannot be scored buys nothing and would put its noise into a floor
where it does not belong.
Artifacts are keyed by detector model, so a second model is baselined by
changing the model and running again, and the comparison table reads both side
by side. A model swap never touches a configuration, which is what makes the two
numbers comparable and reconsidering the model cheap. The web slices are the
default selection and the memory-safety family is measured with its floor marked
as never gating a round.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
