marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-18 12:39:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(learning-harness): the round-4 review findings

Browse Source 
Three defects, two of them the same shape as ones already rejected once:

- The meta-agent proposer still could not propose. `MetaAgent(inner_model=)`
  takes a ModelConfig; it was handed a model name, a `# type: ignore` hid
  that from mypy, and a bare `except Exception` turned the resulting
  AttributeError into `meta_agent_failed` -- which reads exactly like the
  model declining. The composition is now a seam a test can pin, and a
  construction error is re-raised rather than reported as a quiet round.
- `ousast pairs --hunter` crashed in its documented form: the K floor landed
  with `--k-runs` still defaulting to 1. The covering test always passed
  `--k-runs 4`, which is why nothing noticed.
- 28 rows (12% of the corpus) resolved a span on the vulnerable side and
  none on the fixed side, so every fixed-side finding read as outside the
  labeled function and the pair scored as silence whatever the detector
  said. `fix_function` now reaches the catalog and the scorer, and the
  generator derives it from the fixed excerpt for the 25 trap rows that
  never declared one. Measured after: 0 rows.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-06 14:03:10 +02:00
co-authored by Claude Opus 5
parent 75ae9ee47b
commit 2dad25d312
17 changed files with 427 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmarks/pairs/vibe-py/catalog.toml
+28
View File
@@ -2153,6 +2153,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "mutate"
fix_date = "2026"
[[pair.expected]]
@@ -2183,6 +2184,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "mutate"
fix_date = "2026"
[[pair.expected]]
@@ -2213,6 +2215,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "parse_field"
fix_date = "2026"
[[pair.expected]]
@@ -2243,6 +2246,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "forgot_password"
fix_date = "2026"
[[pair.expected]]
@@ -2273,6 +2277,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "parse_field"
fix_date = "2026"
[[pair.expected]]
@@ -2333,6 +2338,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "run"
fix_date = "2026"
[[pair.expected]]
@@ -2363,6 +2369,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "run"
fix_date = "2026"
[[pair.expected]]
@@ -2424,6 +2431,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "get_by_username"
fix_date = "2026"
[[pair.expected]]
@@ -2454,6 +2462,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "get_by_username"
fix_date = "2026"
[[pair.expected]]
@@ -2484,6 +2493,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "get_by_username"
fix_date = "2026"
[[pair.expected]]
@@ -2514,6 +2524,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "do_login"
fix_date = "2026"
[[pair.expected]]
@@ -2544,6 +2555,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "do_login"
fix_date = "2026"
[[pair.expected]]
@@ -2574,6 +2586,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "do_signup"
fix_date = "2026"
[[pair.expected]]
@@ -3366,6 +3379,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "post"
fix_date = "2026"
[[pair.expected]]
@@ -3397,6 +3411,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "post"
fix_date = "2026"
[[pair.expected]]
@@ -3428,6 +3443,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "login"
fix_date = "2026"
[[pair.expected]]
@@ -3495,6 +3511,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "register_user"
fix_date = "2026"
[[pair.expected]]
@@ -3525,6 +3542,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "register_user"
fix_date = "2026"
[[pair.context]]
@@ -3561,6 +3579,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "nosql_return_users_from_username"
fix_date = "2026"
[[pair.expected]]
@@ -3591,6 +3610,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "nosql_return_users_from_username"
fix_date = "2026"
[[pair.expected]]
@@ -3621,6 +3641,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "nosql_return_users_from_username"
fix_date = "2026"
[[pair.expected]]
@@ -3651,6 +3672,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "get_post"
fix_date = "2026"
[[pair.expected]]
@@ -3681,6 +3703,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "get_post"
fix_date = "2026"
[[pair.expected]]
@@ -3711,6 +3734,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "_execute"
fix_date = "2026"
[[pair.expected]]
@@ -3741,6 +3765,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "_execute"
fix_date = "2026"
[[pair.expected]]
@@ -3802,6 +3827,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "get_posts"
fix_date = "2026"
[[pair.expected]]
@@ -3832,6 +3858,7 @@ fix_policy = "silent"
provenance = "human"
split = "train"
review_tier = "seeded"
fix_function = "get_posts"
fix_date = "2026"
[[pair.expected]]
@@ -3862,6 +3889,7 @@ fix_policy = "silent"
provenance = "human"
split = "holdout"
review_tier = "seeded"
fix_function = "get_posts"
fix_date = "2026"
[[pair.expected]]