Commit Graph
3 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 5 3e71e9e88b feat(learning-harness): fix dates and the post-cutoff slice
The catalog generator fills `fix_date` from the recipe at the precision the
recipe has -- a full ISO day when one was recorded, otherwise the year.
Every harvested recipe carries a year and almost none carries a day, and
writing `-01-01` would turn a year into a claim about a month. A row counts
as post-cutoff only when its earliest possible date is after the cutoff, so
a coarse date can under-claim the clean slice but never over-claim it.

Each family now reports the same numbers over the rows fixed after the
cutoff, beside the whole number and never instead of it, with the undated
count on its own line. Without a configured cutoff every scoreboard says so
rather than inventing one: guessing a model's training cutoff and then
publishing a contamination number from the guess would be worse than
reporting none.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-06 13:15:20 +02:00
Marius CiepluchandClaude Fable 5.1 c3e841003c fix(pair-corpus-honesty): re-harvest curl/OpenSSL excerpts, restore Juliet fixtures
Tasks 3.3 and 4.1.

- vfc curl and openssl-cve-2021-3449 excerpts re-harvested through the
  shared library: whole functions with upstream_start headers, the
  vendored key redacted code-safely, no truncated bodies.
- vfc/recipes.toml carries mode, fix_path and function anchors;
  training/manifest.jsonl regenerated.
- Juliet CWE-121 strcpy pair gets its buffer declarations back so the
  vulnerable side is a real overflow again.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 10:45:42 +02:00
Marius Ciepluch 0116f9bea6 feat(real-world-vfc-slice): add labeled OpenSSL/Firefox/Chromium/curl pair corpus
Harvest 176 isolated vuln-vs-fix functions (109 CVEs, 352 training rows)
into benchmarks/pairs/vfc with provenance and a JSONL training manifest.
Score the slice as an honesty dashboard (ousast pairs --slice vfc);
keep the pair gate on the local fixtures only.
2026-09-04 16:45:38 +02:00