The maintainer settled both items the group 1 review escalated.
The family that covers server-side request forgery is named untrusted
destination and also covers open redirect, because both are untrusted input
choosing where a request goes and the corpus carries 34 open-redirect rows that
would otherwise be unclassifiable. The requirement, the brief and the design
example now say so, and the approval note records the amendment.
The shipped taxonomy is flat, so no answer can earn hierarchical partial credit.
Rather than leave that as prose, two new acceptance criteria require every
agreement figure and every family figure to say it, and both report types now
carry the flag, computed from the taxonomy rather than asserted. A taxonomy that
later declares a parent flips it without any other change.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.2. The report covers the whole corpus and scores the classifier only
where a maintainer label exists. Agreement uses the classifier's first answer,
the oracle number uses the best of its answers, and the random baseline is a
uniform pick over the nine real families, so a router error is visible on its
own rather than hidden inside a detector number. Credit is hierarchical: full
for the same family, half for a parent or child, none for a lateral or
fabricated id.
When it is being measured, the classifier no longer reads the declared label it
is scored against, otherwise agreement would be the classifier reading back its
own answer sheet. With no maintainer labels in the corpus yet the report says
the reference set is empty instead of inventing a score, and every disagreement
is queued once with both answers and the deciding tier for a human to settle.
Measured on the 277 vendored pairs: tier one places 275 of them without a single
model call, only two land in unknown, and the 176 memory-safety rows resolve to
one family instead of reading as an unclassified catch-all.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
