Marius CiepluchandClaude Opus 5 241af31a9a feat(learning-harness): per-pair family scoring ...

Task 2.3. A pair is scored once per family. Detection means a finding whose
family tag relates to the labeled family and whose line sits inside the labeled
function; the words in a finding are never read, because a text token is exactly
the proxy that made the old hunter number meaningless. Silence means no such
finding on the fixed side, and findings of other families or outside the labeled
function are counted separately and never called leaks. A fabricated family id
counts as neither, and is recorded as noise worth naming.

Every evaluation runs a pair several times, because a single run misses most
real changes. The recorded outcome is the majority of the runs and the runs that
disagree with it are the pair's own flakiness, which is what the regression
budget will be built from. A pair that cannot be scored keeps its reason and is
never run, and a labeled function no parsed range names is unscorable rather
than a silent miss.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

2026-09-06 09:31:04 +02:00

..

openultrasast

feat(learning-harness): per-pair family scoring

2026-09-06 09:31:04 +02:00