Files
Open_Ultra_SAST/benchmarks/pairs/agent-vfc/README.md
T
Marius CiepluchandClaude Fable 5.1 3c36f13e03 feat(pair-corpus-honesty): LLM-generated Real-Vuln repos and unlicensed agent fixes as pointer pairs
Task 8.4 (Req 10.6, 5.1).

- harvest.merge_recipes appends recipes by name and never rewrites
  existing rows.
- vibe-py/build_recipes.py --pointers: the 40 LLM-generated repositories
  without a license become vendored = false rows (80 pairs, agent,
  seeded) appended to recipes.toml; unlicensed human repos stay skipped.
- agent-vfc/build_recipes.py --from ... --pointers [--cache]: unlicensed
  agent-authored fixes become title-tier pointer rows whose function
  label is derived from the cached excerpt (30 added from a fresh
  662-candidate search); TOML booleans rendered correctly.
- catalogs regenerated: vibe-py 115 rows (35 vendored + 80 pointer),
  agent-vfc 59 (29 + 30); no excerpt is vendored for any pointer row.
- `pairs --slice vibe-py --pointers` scores both provenances (human 35,
  agent 80); the default run skips 80 pairs with pointer_pair_skipped;
  suite and gates unchanged. README/docs name the nightly command.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 10:45:43 +02:00

1.4 KiB

agent-vfc slice

Security-fix commits carrying an AI coding agent co-author trailer or generation marker (Claude Code, Codex, Cursor, Copilot, Devin), in JavaScript, TypeScript, or Python. provenance = "agent". Only repositories with a stated license are vendored; fixes in unlicensed repositories are vendored = false pointer rows (build_recipes.py --from ... --pointers, function label derived from the cached excerpt) scored by ousast pairs --slice agent-vfc --pointers.

Review checklist (Req 5.3)

A recipe with reviewer = "pending" loads at review_tier = "title": scored and reported by pairs; the improve loop's profile gate uses only seeded and reviewed pairs (Req 9.3, task 8.2). Setting reviewer to your name promotes it to reviewed. Before doing so:

  1. The parent excerpt really contains the vulnerability the title claims (not a refactor).
  2. One mechanism from benchmarks/pairs/mechanisms.toml explains it; adjust mechanism and cwe.
  3. The fixed excerpt removes it and the change is inside the vendored function.
  4. No secret, token, or personal data is in either excerpt (run the redaction check).
  5. The license header is present and matches the repository license.

Rebuild: python benchmarks/pairs/agent-vfc/build_recipes.py --search /tmp/agents.jsonl then --from /tmp/agents.jsonl, harvest.py --slice agent-vfc --all --fetch, catalog_gen.py --slice agent-vfc.