Review of task 1.3 found the twin detector stripping every leading comment
line, which in C and C++ swallows include and define directives, and in any
language swallows an ordinary leading comment. Two files whose only difference
was a buffer size on a define hashed equal and were marked unscorable, which
removed a real pair from every denominator: the exact dishonesty this spec
exists to end, running backwards. Measured before the fix: twelve file sides
across six vendored pairs had directives inside the stripped prefix, including
the Juliet stack-overflow pair whose whole point is the buffer size.
The strip is now anchored to the grammar the harvester writes: a leading
comment line counts as header only when it carries one of the provenance keys
followed by a colon, and the scan stops at the first line that is code. Block
and hash comment forms both work. Three regression tests pin the boundary in
both directions, and a fourth asserts no vendored excerpt loses a directive.
The five identical twins, the ten duplicate groups and the nineteen straddling
repositories are unchanged, so the committed hygiene artifact still holds.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.3. An expected row may carry a family label, stored opaquely because the
classifier owns the taxonomy and the corpus module never imports the learning
package. Every pair carries a computed reason when it cannot be scored: a
declared known limit wins, otherwise identical twin when the two excerpts are
the same code once the provenance header is stripped. No row is ever dropped.
Two corpus defects the new accounting surfaces, committed as an artifact beside
the numbers they cap. Nineteen repositories sit on both sides of the split, so a
learning path could be taught by one file of a repository and evaluated on
another. Ten groups of pairs share a vulnerable excerpt byte for byte, mostly
the same fix harvested twice at different granularity.
The split helper keeps every declared split, lets rows that named none inherit
their repository's side, and otherwise assigns a whole repository at once,
oldest first, so nothing straddles by our doing. Pair signals can now be
restricted to one split, which is what lets a learning path teach from train
rows only.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 4.1. A new excerpt mode keeps the labeled handler with its decorators and
every statement outside it that registers the handler by name, so guards living
in decorators, routers and middleware travel with the pair. Comment and string
mentions never count.
The closed mechanism vocabulary gains the three obligation ids, catalog rows
accept an additive obligation field validated against the closed operation
kinds, and the pair scorer counts a finding tagged with that obligation inside
the labeled function as a detection. Rows without the label run the same code
as before.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 8.3 (Req 10.1-10.5).
- pairs.py: catalog rows with vendored = false carry their harvest recipe
and no excerpts; evaluate_catalog harvests them into
OPENULTRASAST_PAIR_CACHE (default ~/.cache/openultrasast/pairs) only
when `pairs --pointers` or OPENULTRASAST_PAIRS_NETWORK=1 allows the
network, otherwise records pointer_pair_skipped (a failed fetch is
pointer_pair_fetch_failed with detail); a warm cache never scores
without the switch. select_vendored for tests and gates.
- harvest.py: materialize_pointer harvests into the cache without the
license requirement (nothing is redistributed); redaction applies.
- catalog_gen.py: emits pointer rows with recipe fields; training
manifests skip them.
- pair_gate and improve/evolve.evaluate_profiles score vendored pairs
only; cli `pairs --pointers`; .gitignore covers the cache path.
- tests: loader, skip/degradation, cache hit, fetch failure, warm cache
with network off, profile gate exclusion, harvest and generator paths;
conftest keeps every test offline whatever the shell exports.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 8.1, 7.1 (tests) and the re-scoped 5.3.
- pairs.py: review_tier in {seeded, advisory, title, reviewed} with
per-slice defaults (local reviewed, vibe-py seeded, sast/vfc/vfc-js/
github advisory, agent-vfc title); reviewer = "pending" means title, a
named reviewer means reviewed, reviewed without a reviewer is rejected;
per_tier metrics in the payload; select_tier and GATING_TIERS for the
improve loop.
- catalog_gen.py: stamps review_tier (and reviewer) on every generated
row and training manifest row; title rows enter catalog.toml; the
catalog-candidates.toml queue is retired; precedence matches the loader.
- agent-vfc: 29 title-tier rows load and score (0/29, Youden 0.0, the
honest unreviewed baseline); local rows name their reviewer.
- Tests: tier defaults, loader rejections, per_tier payload, the Req 8.1
catalog-on-disk / stage-1 exclusion proof and per-slice CLI checks.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 1.2, 2.1-2.5 and 6.2 (pair-side filters).
- pairs.py: coverage with a source counts as detection and as leak;
function-scoped matching by line-range containment from the vulnerable
side's parse (unresolvable labels are counted loss, never silent
misses); exact or alias sink matching; weak CWE-only labels never match;
side-by-side inventory/overlay/hunter scorers; per_profile,
per_mechanism, achievable, known_limit and loss payload; loader rejects
unknown mechanism/provenance/split and reserved or missing function
labels on harvested slices; candidate rulesets thread through for the
improve loop.
- overlay.py: OverlayRecord.function (additive) filled from the shared
naming helper in one parse; benchmark.py: ExpectedFinding.mechanism.
- cli pairs: --slice over every slice, --profile, --split, --hunter,
--hunter-model.
- catalogs migrated to the new schema (provenance, split, function,
mechanism, known_limit for owasp-java-hash); baseline recorded in the
spec notes.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Harvest 176 isolated vuln-vs-fix functions (109 CVEs, 352 training rows)
into benchmarks/pairs/vfc with provenance and a JSONL training manifest.
Score the slice as an honesty dashboard (ousast pairs --slice vfc);
keep the pair gate on the local fixtures only.
Regex inventory stays the proposer. MAP attaches promote/demote/unadjudicated/coverage
from intra-file taint (tree-sitter when a grammar parses, Python ast as last resort).
Deep sandboxes promotions only, ordered by the existing heuristic ranker plus optional
OpenRouter mechanism recall. The SAST pair slice scores overlay outcomes; the stage-1
detection gate and local pair gate stay inventory-only.
