feat(learning-harness): absence rows travel with their registration

Eight of the ten vendored access-control rows had no named handler at all,
so every one of them would have scored `missing_context` forever rather
than as a detector miss. After the re-harvest nine are named and the tenth
states its reason.

The handler-context mode now carries the class its framework registers,
multi-line decorators, and commented-out guards -- `#@token_required`
above a ThreatByte handler is the entire bug. Where the registration lives
in another document (connexion's `operationId`, an express router), a
recipe names `context = [...]`, the harvester writes that stanza per side,
and the pair materializes both files at their repository-relative paths so
the entry-point mapper sees the layout the project has. `harvest_recipe`
fetches every blob before the first write and the driver takes `--report`,
so a failed fetch leaves a reviewed excerpt exactly as it was.

Two defects the re-harvest exposed: `.mjs` was not a known suffix, so the
one pair vendored as an ES module had no file target at all; and
`_python_route_access` matched auth words inside decorator prose, reading
`@ns.doc(description='... without proper authorization checks')` as a
guard. Recorded in the hygiene artifact: nine of the ten duplicate groups
sit on both sides of the split, which the repository-level split cannot
see.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-06 12:39:55 +02:00
co-authored by Claude Opus 5
parent 7a33756a7e
commit c2d9c4ad01
48 changed files with 1613 additions and 11776 deletions
+1
View File
@@ -158,6 +158,7 @@ tests/test_learning_families.py, test_learning_classify.py, test_learning_scorin
- `src/openultrasast/cli.py` — `ousast learning classify|score|baseline|round|publish`; `pairs --hunter` uses the family scorer and `--k-runs`; `[models].hunter` resolution through `learning.endpoint`.
- `pyproject.toml` — `tree-sitter-typescript>=0.23` in the semantic extra; `src/openultrasast/semantic/extra.py` maps `typescript`/`tsx`.
- `benchmarks/pairs/vibe-py/catalog.toml` and `recipes.toml` — the five identical twins gain `known_limit = "identical_twin"`; absence rows gain `family` and `mode = "handler_context"` (re-harvest is a maintainer step).
- **Cross-file registration context (amended during task 4.1).** Some frameworks keep the route table out of the handler module: connexion registers by `operationId` in an OpenAPI document, an Express application registers in a router file. A single-file excerpt of such a handler can never contain its registration, so Req 10.2 is unreachable for those rows without a second file. A recipe may therefore name `context = ["<repo-relative path>"]`; `benchmarks/pairs/harvest.py` fetches each document at the same two commits, extracts only the stanza that names the handler (`extract_registration_context`), and writes it under `<name>/context/<side>/<relpath>` with its own provenance header and the same redaction. The catalog carries `[[pair.context]]` rows, `PairCase.context_files` holds `(relpath, vuln, fixed)`, and `_materialize_side` copies them beside the excerpt at their repository-relative paths so the entry-point mapper sees the same layout the project has. `src/openultrasast/mapping.py` gains `_route_manifest_entry_points`, which reads `operationId` and the operation's `security` block out of a YAML or JSON document and attributes the record to the module the operation id points at. Trap pairs whose fixed side is a different, correctly guarded handler carry `fix_function` so the fixed side gets *its* registration rather than the vulnerable side's.
- `benchmarks/pairs/mechanisms.toml` — unchanged; families reference mechanism ids.
- `.kiro/steering/roadmap.md` — rewritten by `publish` from artifacts.
+13 -1
View File
@@ -137,7 +137,7 @@
- _Requirements: 11.1, 11.4_
- [ ] 4. Corpus repairs
- [ ] 4.1 Re-harvest absence rows with registration context
- [x] 4.1 Re-harvest absence rows with registration context
- Absence rows in vibe-py and agent-vfc gain their family and the handler-context mode in the recipes; the maintainer re-harvests them over the network; every re-harvested excerpt keeps its license line, passes redaction and keeps the decorators and registration statements; failed fetches keep the old excerpt with a recorded reason; catalogs regenerate idempotently.
- Observable: the vampi and threatbyte excerpts contain their registration lines, the entry-point mapper names their handlers, and the catalogs load with unchanged vendored counts.
- _Depends: 1.3_
@@ -207,3 +207,15 @@
- Task-plan updates from the review: 2.2 and 2.4 claim the two criteria the maintainer added (2.6 and 3.7); 3.6 claims Req 6.2 with an observable that every non-target family directory is byte-identical across a round; 3.7 claims Req 3.5's K floor, 9.6's pair identities and 11.3's trajectories.
- **Escalated to the maintainer**: Req 6.1 lists "a skills set" among a detector configuration's parts, and neither the approved design nor the code has one, although the repository ships a skill router. Either Req 6.1 drops the skills set or a task wires it; the implementation followed the approved design, so this is a spec-internal conflict rather than a deviation.
- Carry-forward: the nineteen straddling repositories still cross the split by declaration and cap every holdout number these tasks produce; `unknown` serves as both the classifier's abstention and the generalist detector, which deserves a sentence in the design before a round ever targets it.
- Task 4.1 (2026-09-06, RED first: 21 of 23 failing tests in `tests/test_pair_registration_context.py`; then green, 26 tests). Measured before the change: **8 of the 10 vendored access-control rows had no named handler at all**, so every one of them would have scored `missing_context` forever rather than as a detector miss. After the re-harvest 9 of 10 are named and the tenth states its reason. What the repair needed, in order of how much it cost:
- **Class registration.** A flask-restx handler is a method; `@ns.route('/profile')` sits on the class. `extract_handler_context` now carries the enclosing class declaration and its decorators, so the excerpt is both reachable and parsable — a bare indented `def get` is neither.
- **Multi-line decorators and commented-out guards.** `@ns.doc(...)` wraps over four lines, and stopping at a continuation dropped the `@ns.route(...)` above it — the registration itself. `#@token_required` above a handler is not noise; in the two ThreatByte rows it is the entire bug, and the evidence field says so. `_decorator_start` now walks both.
- **Cross-file registration (design amendment, recorded in `design.md`).** connexion registers VAmPI's handlers by `operationId` in `openapi_specs/openapi3.yml`; Overleaf registers `exportProject` in `router.mjs`. No single-file excerpt can contain that, so a recipe may name `context = [...]`, the harvester writes the stanza per side under `<name>/context/<side>/<relpath>`, the catalog carries `[[pair.context]]`, and `_materialize_side` lays the documents out beside the excerpt at their repository-relative paths. `mapping._route_manifest_entry_points` reads `operationId` plus the operation's `security` block; `_js_statements` joins a registration that spans lines, because reading `webRouter.post(` alone finds neither the handler nor the middleware that guards it.
- **Anchors.** Real-Vuln-Benchmark trap rows take their fixed side from a different, correctly guarded handler of the same snapshot, so `handler_context` honours `line`/`fix_line` and a new `fix_function` gives the fixed side *its* registration instead of the vulnerable side's.
- **`innermost_function_bounds`** for brace languages, used only by this mode: `function_bounds_at` returns the outermost top-level group, which for a handler declared inside a React component is the whole 6000-line component. `hunk` and `enclosing` rows keep the bounds they were harvested with.
- **Fetch-then-write.** `harvest_recipe` fetches every blob before the first write, and the driver takes `--report` so a failed fetch leaves the reviewed excerpt exactly as it was and says why. A half-written re-harvest is indistinguishable from a real upstream change.
- Bugs the re-harvest exposed and fixed on the way: `.mjs` and `.cjs` were not in `LANGUAGE_BY_EXTENSION`, so the one pair vendored as an ES module had **no file target at all** — no entry point, no finding, a permanent miss; and `_python_route_access` matched auth words anywhere in a decorator, so `@ns.doc(description='... without proper authorization checks')` read the unguarded ThreatByte endpoint as `authenticated`, silencing every obligation on it. The guard is now the decorator's name, never its prose, and `@token_required` — the commonest Flask guard — is recognised at all.
- Corpus corrections: `lensops-kubernetesmanagement-fe4910` was labeled `setErrorMessage`, a React state setter, because the hunk landed in the wrong bounds; it is now `loadNamespaces` and the pair diff is exactly the fix. Its bug is a browser-side call omitting the header its siblings carry, not a missing guard on an inbound handler, so it carries `known_limit = "client_side_caller"` and leaves the denominators with a stated reason.
- Three existing tests encoded the old corpus and now encode the new one: the TypeScript parse pin (the lensops excerpt parses now), the classifier report (`labeled` 0 -> 4, the four seeded rows; the six `title`-tier agent rows carry a family but are not labels a maintainer stands behind), and the hygiene report ordering.
- **New honesty finding, recorded in the hygiene artifact as `cross_split_duplicates`: nine of the ten duplicate groups sit on both sides of the split.** Two forks of one project (`openniw-jobs-0fc947` train, `openniw-jobs-b74de3` holdout) and eight CVEs harvested twice under different names are byte-identical, so the train row teaches the holdout row exactly. `split_by_repository` cannot see it — the duplicates are different repositories. `duplicate_groups` is now ordered by name so the committed report does not reshuffle when one excerpt is re-harvested.
- Pre-existing drift the regeneration corrected, unrelated to this task: `catalog.toml` in git disagreed with a fresh `catalog_gen` run at the group-3 tip on two derived labels (`origin` -> `createApplication`, `normalizeDomain` -> `getGoogleProvider`). Verified in a detached worktree at `7a33756` before touching anything.