mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-17 12:09:27 +00:00
feat(learning-harness): absence rows travel with their registration
Eight of the ten vendored access-control rows had no named handler at all, so every one of them would have scored `missing_context` forever rather than as a detector miss. After the re-harvest nine are named and the tenth states its reason. The handler-context mode now carries the class its framework registers, multi-line decorators, and commented-out guards -- `#@token_required` above a ThreatByte handler is the entire bug. Where the registration lives in another document (connexion's `operationId`, an express router), a recipe names `context = [...]`, the harvester writes that stanza per side, and the pair materializes both files at their repository-relative paths so the entry-point mapper sees the layout the project has. `harvest_recipe` fetches every blob before the first write and the driver takes `--report`, so a failed fetch leaves a reviewed excerpt exactly as it was. Two defects the re-harvest exposed: `.mjs` was not a known suffix, so the one pair vendored as an ES module had no file target at all; and `_python_route_access` matched auth words inside decorator prose, reading `@ns.doc(description='... without proper authorization checks')` as a guard. Recorded in the hygiene artifact: nine of the ten duplicate groups sit on both sides of the split, which the repository-level split cannot see. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
co-authored by
Claude Opus 5
parent
7a33756a7e
commit
c2d9c4ad01
@@ -44,7 +44,11 @@ def test_the_report_covers_the_whole_corpus_and_names_the_unknowns(tmp_path: Pat
|
||||
assert report.unknown == 2 and report.per_tier["static"] == len(cases) - 2 # tier one places all but two rows
|
||||
payload = report.to_dict()
|
||||
json.dumps(payload)
|
||||
assert payload["labeled"] == 0 # no maintainer family labels in the corpus yet, and the report says so
|
||||
# Task 4.1 declared `family` on the ten absence rows; only the four at a reference tier (`seeded`) count as labels
|
||||
# a maintainer stands behind, and the six `title`-tier agent rows do not. Agreement over four rows of one family is
|
||||
# not yet a measurement of the classifier, so the number is recorded, not celebrated.
|
||||
assert payload["labeled"] == 4 and payload["agreement"] == 1.0
|
||||
assert set(report.confusion) == {"access_control"}
|
||||
|
||||
|
||||
def test_agreement_uses_the_first_answer_and_the_oracle_uses_any_of_them(tmp_path: Path) -> None:
|
||||
|
||||
Reference in New Issue
Block a user