marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-19 21:19:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(pair-corpus-honesty): LLM-generated Real-Vuln repos and unlicensed agent fixes as pointer pairs

Browse Source 
Task 8.4 (Req 10.6, 5.1).

- harvest.merge_recipes appends recipes by name and never rewrites
  existing rows.
- vibe-py/build_recipes.py --pointers: the 40 LLM-generated repositories
  without a license become vendored = false rows (80 pairs, agent,
  seeded) appended to recipes.toml; unlicensed human repos stay skipped.
- agent-vfc/build_recipes.py --from ... --pointers [--cache]: unlicensed
  agent-authored fixes become title-tier pointer rows whose function
  label is derived from the cached excerpt (30 added from a fresh
  662-candidate search); TOML booleans rendered correctly.
- catalogs regenerated: vibe-py 115 rows (35 vendored + 80 pointer),
  agent-vfc 59 (29 + 30); no excerpt is vendored for any pointer row.
- `pairs --slice vibe-py --pointers` scores both provenances (human 35,
  agent 80); the default run skips 80 pairs with pointer_pair_skipped;
  suite and gates unchanged. README/docs name the nightly command.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-05 10:45:43 +02:00
co-authored by Claude Fable 5.1
parent 4102404442
commit 3c36f13e03
15 changed files with 7380 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/pairs/harvest.py
+7
View File
@@ -553,6 +553,13 @@ def fetch_url(url: str, timeout: int = 60) -> str:
return response.read().decode("utf-8", "replace")
def merge_recipes(existing: list[dict[str, Any]], new: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], list[str]]:
"""Append ``new`` recipes whose name is not already present; existing rows are never rewritten (vendored labels stay put)."""
names = {str(r["name"]) for r in existing}
added = [r for r in new if str(r["name"]) not in names]
return [*existing, *added], [str(r["name"]) for r in added]
def load_recipes(path: Path) -> tuple[dict[str, Any], ...]:
payload = tomllib.loads(path.read_text())
return tuple(dict(item) for item in payload.get("recipe", []))