mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 04:29:27 +00:00
feat(learning-harness): absence rows travel with their registration
Eight of the ten vendored access-control rows had no named handler at all, so every one of them would have scored `missing_context` forever rather than as a detector miss. After the re-harvest nine are named and the tenth states its reason. The handler-context mode now carries the class its framework registers, multi-line decorators, and commented-out guards -- `#@token_required` above a ThreatByte handler is the entire bug. Where the registration lives in another document (connexion's `operationId`, an express router), a recipe names `context = [...]`, the harvester writes that stanza per side, and the pair materializes both files at their repository-relative paths so the entry-point mapper sees the layout the project has. `harvest_recipe` fetches every blob before the first write and the driver takes `--report`, so a failed fetch leaves a reviewed excerpt exactly as it was. Two defects the re-harvest exposed: `.mjs` was not a known suffix, so the one pair vendored as an ES module had no file target at all; and `_python_route_access` matched auth words inside decorator prose, reading `@ns.doc(description='... without proper authorization checks')` as a guard. Recorded in the hygiene artifact: nine of the ten duplicate groups sit on both sides of the split, which the repository-level split cannot see. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
7a33756a7e
commit
c2d9c4ad01
@@ -75,6 +75,23 @@ def test_duplicate_bodies_across_pairs_are_reported_before_any_split() -> None:
|
||||
payload = json.loads(report.read_text())
|
||||
assert payload["duplicate_groups"] == [list(names) for names in groups]
|
||||
assert set(payload["identical_twins"]) == IDENTICAL_TWINS
|
||||
assert groups == tuple(sorted(groups)), "the committed report must not reshuffle when one excerpt is re-harvested"
|
||||
|
||||
|
||||
def test_duplicate_groups_that_cross_the_split_are_named_as_the_leak_they_are() -> None:
|
||||
"""Two byte-identical excerpts on opposite sides of the split teach the holdout row exactly.
|
||||
|
||||
`split_by_repository` cannot see this: the duplicates are different repositories (a fork of a project, or one CVE
|
||||
harvested under two names), so the repository-level split reports no straddle and the leak stays invisible."""
|
||||
from openultrasast.pairs import duplicate_groups
|
||||
|
||||
cases = select_vendored(load_pair_catalog(DEFAULT_CATALOG))
|
||||
splits = {case.name: case.split for case in cases}
|
||||
crossing = [list(names) for names in duplicate_groups(cases) if len({splits[name] for name in names}) > 1]
|
||||
payload = json.loads(Path("benchmarks/measurements/2026-09-06-corpus-hygiene.json").read_text())
|
||||
assert payload["cross_split_duplicates"] == crossing
|
||||
assert len(crossing) == 9, "nine of the ten duplicate groups cross the split; the number is the finding"
|
||||
assert ["openniw-jobs-0fc947", "openniw-jobs-b74de3"] in crossing # two forks of one project, train and holdout
|
||||
|
||||
|
||||
def test_split_by_repository_keeps_declared_splits_and_reports_straddles() -> None:
|
||||
|
||||
Reference in New Issue
Block a user