mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pair-corpus-honesty): LLM-generated Real-Vuln repos and unlicensed agent fixes as pointer pairs
Task 8.4 (Req 10.6, 5.1). - harvest.merge_recipes appends recipes by name and never rewrites existing rows. - vibe-py/build_recipes.py --pointers: the 40 LLM-generated repositories without a license become vendored = false rows (80 pairs, agent, seeded) appended to recipes.toml; unlicensed human repos stay skipped. - agent-vfc/build_recipes.py --from ... --pointers [--cache]: unlicensed agent-authored fixes become title-tier pointer rows whose function label is derived from the cached excerpt (30 added from a fresh 662-candidate search); TOML booleans rendered correctly. - catalogs regenerated: vibe-py 115 rows (35 vendored + 80 pointer), agent-vfc 59 (29 + 30); no excerpt is vendored for any pointer row. - `pairs --slice vibe-py --pointers` scores both provenances (human 35, agent 80); the default run skips 80 pairs with pointer_pair_skipped; suite and gates unchanged. README/docs name the nightly command. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
4102404442
commit
3c36f13e03
@@ -33,6 +33,7 @@ uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
|
||||
# Differential pair eval: fire on vuln, stay silent on the fix (local + real GitHub VFCs)
|
||||
uv run ousast pairs
|
||||
uv run ousast pairs --slice sast # OWASP Benchmark Java/Python + Juliet
|
||||
uv run ousast pairs --slice vibe-py --pointers # nightly: also harvest non-vendored pointer pairs into ~/.cache
|
||||
uv run python -m openultrasast.pair_gate
|
||||
|
||||
# Run the bounded self-improvement loop over the ruleset (benchmark-driven)
|
||||
|
||||
Reference in New Issue
Block a user