Files
Open_Ultra_SAST/benchmarks/pairs/agent-vfc
Marius CiepluchandClaude Fable 5.1 b1c5f36076 feat(pair-corpus-honesty): review tiers on every pair, agent-vfc loads at tier title
Tasks 8.1, 7.1 (tests) and the re-scoped 5.3.

- pairs.py: review_tier in {seeded, advisory, title, reviewed} with
  per-slice defaults (local reviewed, vibe-py seeded, sast/vfc/vfc-js/
  github advisory, agent-vfc title); reviewer = "pending" means title, a
  named reviewer means reviewed, reviewed without a reviewer is rejected;
  per_tier metrics in the payload; select_tier and GATING_TIERS for the
  improve loop.
- catalog_gen.py: stamps review_tier (and reviewer) on every generated
  row and training manifest row; title rows enter catalog.toml; the
  catalog-candidates.toml queue is retired; precedence matches the loader.
- agent-vfc: 29 title-tier rows load and score (0/29, Youden 0.0, the
  honest unreviewed baseline); local rows name their reviewer.
- Tests: tier defaults, loader rejections, per_tier payload, the Req 8.1
  catalog-on-disk / stage-1 exclusion proof and per-slice CLI checks.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 10:45:43 +02:00
..

agent-vfc slice

Security-fix commits carrying an AI coding agent co-author trailer or generation marker (Claude Code, Codex, Cursor, Copilot, Devin), in JavaScript, TypeScript, or Python. provenance = "agent". Only repositories with a stated license are vendored.

Review checklist (Req 5.3)

A recipe with reviewer = "pending" loads at review_tier = "title": scored and reported by pairs; the improve loop's profile gate uses only seeded and reviewed pairs (Req 9.3, task 8.2). Setting reviewer to your name promotes it to reviewed. Before doing so:

  1. The parent excerpt really contains the vulnerability the title claims (not a refactor).
  2. One mechanism from benchmarks/pairs/mechanisms.toml explains it; adjust mechanism and cwe.
  3. The fixed excerpt removes it and the change is inside the vendored function.
  4. No secret, token, or personal data is in either excerpt (run the redaction check).
  5. The license header is present and matches the repository license.

Rebuild: python benchmarks/pairs/agent-vfc/build_recipes.py --search /tmp/agents.jsonl then --from /tmp/agents.jsonl, harvest.py --slice agent-vfc --all --fetch, catalog_gen.py --slice agent-vfc.