Design approved in session. Every one of the 36 criteria is claimed by a task.
Group 1 calls no model at all. The candidate enumerator, the ceiling report
and the stop-or-go reading are offline, so the condition that ends this
feature -- a deciding slice whose ceiling sits below its recorded detection
rate -- is reachable before a cent is spent. The ruleset variant of this same
design would have failed that condition at 12.1% against 64.9%.
Group 2 is the only work that touches scoring.py, and its observable is that
nothing changes: the recorded round-zero artifacts are the pin, and if they
cannot be reproduced exactly then the refactor is wrong, not the numbers.
Group 3 builds the detector, and task 3.3 is the test that would have caught
the defect this spec exists for -- two runs must make an identical sequence of
model calls. It lives in the suite and runs repeatedly, not in a one-off
measurement.
Group 4 is the maintainer's emphasis as code: deciding slices, development
slices, the cross-slice rejection by name, and the overfitting gap printed
beside every headline number.
Group 5 compares the two detectors per slice, re-measures the trajectory count
and the noise floor, and names -- without amending -- the learning-harness
requirements that a collapsed floor would retire.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Requirements approved in session. All 36 criteria trace to a component.
The shape: a deterministic pass enumerates candidate sites from the semantic
IR, shaped by family -- call sites for injection and path, operations with
their guards for access control, bindings and literals for configuration. The
model answers one bounded typed question per candidate with no tools and no
choice of what to look at. The predicates that decide whether a finding counts
are applied while it is being made, with one retry carrying the violation, and
they come from a single definition that the scorer also uses, so the two
cannot drift.
Determinism is a design property, not a hope: candidates are ordered by
(path, line, name) and batched at a fixed size, so two runs make an identical
sequence of model calls. That is the direct answer to 120 runs producing 120
distinct trajectories.
Phase 1 is the candidate ceiling, offline, no model call, measured on the
real-project slices as well as the teaching one. If the ceiling on a deciding
slice comes in below that slice's current detection rate, the design stops
there having spent nothing. Building that in as the first phase is deliberate:
the ruleset variant of this design would have capped recall at 12.1% against a
64.9% baseline, and only a measurement caught it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Renamed from detector-optimizer: the brief's own conclusion is that the
optimizer is not the next thing to build, and a spec should be named for what
it delivers.
Seven requirements, 37 criteria. The load-bearing ones:
Req 1 -- candidates come from the semantic IR, never the pattern ruleset,
and the enumerator's ceiling is published as the limit on recall. Measured
before building: ruleset 12.1%, IR 96.6%.
Req 2 -- one bounded typed question per candidate, no tools, no choice of
what to look at, and a test asserts two runs make an identical sequence of
model calls.
Req 3 is the maintainer's emphasis and it decides acceptance. The slices
harvested from real projects decide; hand-written vibe-py is a development
slice that cannot on its own justify a change; a change that improves vibe-py
and regresses a deciding slice is rejected by name; and the overfitting gap
-- 83.3% against 44.4% at baseline -- is published beside every headline
number. The candidate ceiling is measured separately on real-world slices,
because their functions are larger and the enumerator's behaviour there is
not implied by its behaviour on teaching code.
Req 5 -- the K-run and noise-floor machinery is re-measured against the
constrained detector and deleted if the floor collapses, naming the
learning-harness requirements that would be amended.
Not approved; design not started.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
