Three defects, two of them the same shape as ones already rejected once:
- The meta-agent proposer still could not propose. `MetaAgent(inner_model=)`
takes a ModelConfig; it was handed a model name, a `# type: ignore` hid
that from mypy, and a bare `except Exception` turned the resulting
AttributeError into `meta_agent_failed` -- which reads exactly like the
model declining. The composition is now a seam a test can pin, and a
construction error is re-raised rather than reported as a quiet round.
- `ousast pairs --hunter` crashed in its documented form: the K floor landed
with `--k-runs` still defaulting to 1. The covering test always passed
`--k-runs 4`, which is why nothing noticed.
- 28 rows (12% of the corpus) resolved a span on the vulnerable side and
none on the fixed side, so every fixed-side finding read as outside the
labeled function and the pair scored as silence whatever the detector
said. `fix_function` now reaches the catalog and the scorer, and the
generator derives it from the fixed excerpt for the 25 trap rows that
never declared one. Measured after: 0 rows.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
The catalog generator fills `fix_date` from the recipe at the precision the
recipe has -- a full ISO day when one was recorded, otherwise the year.
Every harvested recipe carries a year and almost none carries a day, and
writing `-01-01` would turn a year into a claim about a month. A row counts
as post-cutoff only when its earliest possible date is after the cutoff, so
a coarse date can under-claim the clean slice but never over-claim it.
Each family now reports the same numbers over the rows fixed after the
cutoff, beside the whole number and never instead of it, with the undated
count on its own line. Without a configured cutoff every scoreboard says so
rather than inventing one: guessing a model's training cutoff and then
publishing a contamination number from the guess would be worse than
reporting none.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 1.2, 2.1-2.5 and 6.2 (pair-side filters).
- pairs.py: coverage with a source counts as detection and as leak;
function-scoped matching by line-range containment from the vulnerable
side's parse (unresolvable labels are counted loss, never silent
misses); exact or alias sink matching; weak CWE-only labels never match;
side-by-side inventory/overlay/hunter scorers; per_profile,
per_mechanism, achievable, known_limit and loss payload; loader rejects
unknown mechanism/provenance/split and reserved or missing function
labels on harvested slices; candidate rulesets thread through for the
improve loop.
- overlay.py: OverlayRecord.function (additive) filled from the shared
naming helper in one parse; benchmark.py: ExpectedFinding.mechanism.
- cli pairs: --slice over every slice, --profile, --split, --hunter,
--hunter-model.
- catalogs migrated to the new schema (provenance, split, function,
mechanism, known_limit for owasp-java-hash); baseline recorded in the
spec notes.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 3.3 and 4.1.
- vfc curl and openssl-cve-2021-3449 excerpts re-harvested through the
shared library: whole functions with upstream_start headers, the
vendored key redacted code-safely, no truncated bodies.
- vfc/recipes.toml carries mode, fix_path and function anchors;
training/manifest.jsonl regenerated.
- Juliet CWE-121 strcpy pair gets its buffer declarations back so the
vulnerable side is a real overflow again.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 3.1, 3.2 (harvest library) and the Scorer-owned naming helper.
- semantic/functions.py: one source of function names for scorer, overlay
and catalog generator (declarator wins over parser name; JS/TS bare-param
arrows and undeclared methods are <anon>; C++ Class::method resolves to
its unqualified tail; Perl sub; statements never name).
- benchmarks/pairs/harvest.py: shared library with name / line_range /
hunk / enclosing modes, comment-blind masking (triple-quote aware),
upstream_start headers, code-safe quoted-only redaction.
- benchmarks/pairs/catalog_gen.py: derive function labels and sinks from
the declarator at the mapped sink location, CWE-family sink tokens,
--prune with refusal when nothing derived, fails loud without the
package; vfc/harvest.py and vfc/generate_catalog.py become shims.
- tests/test_pair_harvest.py replaces test_vfc_harvest.py.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Harvest 176 isolated vuln-vs-fix functions (109 CVEs, 352 training rows)
into benchmarks/pairs/vfc with provenance and a JSONL training manifest.
Score the slice as an honesty dashboard (ousast pairs --slice vfc);
keep the pair gate on the local fixtures only.
