From b95a371b7e6a9865d7ab7b1e005a545643b66a33 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Thu, 17 Sep 2026 13:28:40 +0200 Subject: [PATCH] feat(contributor-scan): name the function a question cannot locate (M1b 12.3) After per-family context, every remaining unresolved question reported context_scope_empty, all in three files. The cause is not a missing producer: app/routes/index.js requires seven handler modules and defines none of them, so a question naming handleContributionsUpdate there names a function that file genuinely does not hold, and the empty scope is the correct answer. Reporting that as "scope empty" hid which of two different things had happened. A file the frontend produced no methods for is one problem; a question naming a function defined in another module is another, and it is the common one. Taint and configuration now separate them, reporting context_function_unresolved with the name in the first case and context_scope_empty only when the file has no methods at all. The driver no longer stacks its generic gap on top of a query that already explained itself, and the new gap is question-owned so it cannot block another question's comparison. Completion is deliberately unchanged. The tempting fix is to widen such a question to its whole file, which would raise the number by answering a question nobody asked; a question whose function the graph does not hold stays unresolved, with the name now recoverable from the question that asked it. Group 12 is complete: admission no longer rejects genuine findings on a mis-extracted operation symbol, both families that could never complete their change context now can, and the remaining unresolved questions say precisely why. 1,406 tests passed, nine skipped. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo --- .kiro/specs/pre-push-safety-net/tasks.md | 5 +++-- src/openultrasast/cpg/queries/config.sc | 8 +++++++- src/openultrasast/cpg/queries/taint.sc | 16 +++++++++++++--- src/openultrasast/model/regions.py | 5 ++++- src/openultrasast/push/policy.py | 1 + tests/test_model_change_context.py | 19 +++++++++++++++++++ 6 files changed, 47 insertions(+), 7 deletions(-) diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md index 8f7c36f..9bbc788 100644 --- a/.kiro/specs/pre-push-safety-net/tasks.md +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -395,7 +395,7 @@ Preserve all completed task evidence below. - _Requirements: 4.1, 4.2, 6.2, 6.3, 6.5, 7.3_ - _Measured 2026-09-17, verdict NO-GO:_ the unchanged seven-class profile, three samples each, session enabled. Every changed-code workload still times out at the 30-second deadline, and identical-tip got worse, p50 rising from 18.5 s to 24.2 s. The cause is recorded in the artifacts rather than inferred: 13 transactions report `session_unavailable:session_startup_timeout`, so under the real deadline the session usually never starts and everything falls back. Startup is 9 to 13 seconds, about a third of the whole budget, spent before any analysis. Where a session did start, on cached-graph transactions with little remaining work, it cost more than it saved. Correctness is unchanged: the same three revisions through the production rules with the session on reproduce the M1 verdict. Evidence: `benchmarks/measurements/2026-09-17-session-runtime-profile.json`. **Dominant remaining cost: engine startup itself, which a transaction-owned session relocates but cannot remove.** -- [ ] 12. Security capability work found during M1a and M1b +- [x] 12. Security capability work found during M1a and M1b - Group 11 established that M2 needs graph reuse rather than more runtime tuning, and that decision is recorded there. This group returns to what the tool can actually establish about a codebase, which is where the remaining release risk sits: two of the three shipped families cannot complete their change context at all, and admission rejects genuine findings on a mis-extracted operation symbol. - [x] 12.1 Extract the operation symbol from the operation, not from the statement - **Found 2026-09-16 in the M1b explanation run.** Admission takes the text before the first parenthesis as the operation symbol, so `const preTax = eval(req.body.preTax)` yields `const preTax = eval`. The demonstration declaration had to enumerate each assignment spelling for its coverage check to pass. @@ -422,7 +422,7 @@ Preserve all completed task evidence below. - _Requirements: 4.1, 4.2, 6.5_ - _Decided 2026-09-17 by the maintainer: do not rebuild a whole graph per push._ Both startup attacks were measured and neither reaches the gate. The session helps a long transaction, 326 s to 243 s, and cannot help a 30-second one: startup is 9 to 13 seconds of the budget, so it usually declines. Removing JavaScript's second engine start is real, 11.0 s against 21.0 s on identical graphs, but its census gate was asymmetric and it was reverted after costing two engine starts on 12 profile transactions. What the profile names is not startup at all: a changed-code transaction reaches its deadline during the build, head build 25.2 s of 30 s with query time 0.0 s. The remaining cost is building a fresh whole-repository graph for every pushed revision. **Direction: reuse the graph across revisions instead of rebuilding it, rather than further optimising the rebuild.** Not scheduled here; M2 stays unmet and the 30-second and 2-second gates stay unchanged. -- [ ] 12.3 Resolve function-scoped questions whose function the query cannot locate +- [x] 12.3 Resolve function-scoped questions whose function the query cannot locate - **Named by the 12.2 measurement.** With per-family context in place, head completion rose from 205 to 298 of 436 and `context_projection_unavailable` fell to zero. Every remaining unresolved question reports `context_scope_empty`, and they sit in three files: `app/routes/index.js` (114), `app/routes/tutorial.js` (12) and `server.js` (12). None is access control. - These are function-scoped questions whose named function the query cannot locate in the graph, so the scope it describes is honestly empty. It is the same query-to-operation identity problem M1a recorded at the witness end, appearing at the scope end. - Carry verified provenance from the region's declared function to the method the engine actually holds, and leave an unresolvable name explicit. Do not widen a question to the whole file to make it complete, and do not inject a declared name into a region. @@ -430,6 +430,7 @@ Preserve all completed task evidence below. - _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_ - _Depends: 12.2_ - _Requirements: 2.1, 2.2, 6.3, 8.1_ + - _Verified 2026-09-17:_ 1,406 tests passed, nine skipped; Ruff/format/mypy passed. The cause is recorded rather than guessed: `app/routes/index.js` requires seven handler modules and defines none of them, so a question naming `handleContributionsUpdate` there names a function that file genuinely does not hold and the empty scope is correct. Taint and configuration now separate the two causes, reporting `context_function_unresolved` with the name when a question names a function the file does not define, and `context_scope_empty` only when the file has no methods at all. The driver no longer adds its generic empty-scope gap on top of a query that already explained itself, and the new gap is question-owned so it cannot block another question's comparison. Completion is unchanged by design: a question whose function the graph does not hold stays unresolved, and nothing completes by widening its scope to the whole file. ## Implementation Notes diff --git a/src/openultrasast/cpg/queries/config.sc b/src/openultrasast/cpg/queries/config.sc index 42560a4..dfb85b0 100644 --- a/src/openultrasast/cpg/queries/config.sc +++ b/src/openultrasast/cpg/queries/config.sc @@ -81,8 +81,14 @@ } val settingIds = calls.map(_.method.id).toSet val located = scoped.map(m => location(m, if (settingIds.contains(m.id)) "setting" else "sibling")) + // A question naming a function this file does not define is not the same as a file with no + // methods at all, and saying so is the difference between a reader knowing what to look at + // and reading "scope empty". val boundaries = - if (scoped.isEmpty) List(ujson.Obj("kind" -> "context_boundary", "reason" -> "context_scope_empty:contributor-scan")) + if (function.nonEmpty && scoped.isEmpty) + List(ujson.Obj("kind" -> "context_boundary", "reason" -> "context_function_unresolved:contributor-scan", "function" -> function)) + else if (scoped.isEmpty) + List(ujson.Obj("kind" -> "context_boundary", "reason" -> "context_scope_empty:contributor-scan")) else Nil ujson.Obj("kind" -> "context_summary") :: (located ++ boundaries) } diff --git a/src/openultrasast/cpg/queries/taint.sc b/src/openultrasast/cpg/queries/taint.sc index e4d56ea..8fea4ec 100644 --- a/src/openultrasast/cpg/queries/taint.sc +++ b/src/openultrasast/cpg/queries/taint.sc @@ -732,9 +732,19 @@ destinations.isEmpty || destinations.exists(d => d.isExternal || !inScope(d)) } }) - val boundaries = if (scoped.isEmpty) List(ujson.Obj("kind" -> "context_boundary", - "reason" -> "context_scope_empty:contributor-scan")) else if (missing) List(ujson.Obj("kind" -> "context_boundary", - "reason" -> "dynamic_external_or_depth_context_unresolved:contributor-scan")) else Nil + // An empty scope has two different causes and they are not interchangeable. A file the + // frontend produced no methods for is one thing; a question naming a function this file does + // not define is another, and it is the common one: a route-registration module names handlers + // that live in other files, so the graph rightly holds no such method here. Reporting both as + // "scope empty" hides which is which, and only the second names something a reader can act on. + val boundaries = + if (function.nonEmpty && scoped.isEmpty) + List(ujson.Obj("kind" -> "context_boundary", "reason" -> "context_function_unresolved:contributor-scan", "function" -> function)) + else if (scoped.isEmpty) + List(ujson.Obj("kind" -> "context_boundary", "reason" -> "context_scope_empty:contributor-scan")) + else if (missing) + List(ujson.Obj("kind" -> "context_boundary", "reason" -> "dynamic_external_or_depth_context_unresolved:contributor-scan")) + else Nil ujson.Obj("kind" -> "context_summary") :: (methods ++ fields ++ hooks ++ guards ++ boundaries) } return summary :: (perSink ++ contextRows) diff --git a/src/openultrasast/model/regions.py b/src/openultrasast/model/regions.py index 32db293..59bbbc0 100644 --- a/src/openultrasast/model/regions.py +++ b/src/openultrasast/model/regions.py @@ -226,7 +226,10 @@ def affected_context( local_gaps = [] if not any(r.get("kind") == "context_summary" for r in rows): local_gaps.append("context_projection_unavailable:contributor-scan") - elif not any(r.get("kind") == "context_method" for r in rows): + elif not any(r.get("kind") == "context_method" for r in rows) and not any(r.get("kind") == "context_boundary" for r in rows): + # Only when the query offered no account of its own. A query that already said why its + # scope is empty -- a named function this file does not define, most often -- has given + # the specific reason, and adding a generic one on top buries it. local_gaps.append("context_scope_empty:contributor-scan") recorded_only: list[str] = [] for row in rows: diff --git a/src/openultrasast/push/policy.py b/src/openultrasast/push/policy.py index 1923284..44892f7 100644 --- a/src/openultrasast/push/policy.py +++ b/src/openultrasast/push/policy.py @@ -320,6 +320,7 @@ def _context_boundaries(scan: ModelScanResult, *questions: QuestionIdentity) -> owned_prefixes = ( "context_projection_unavailable:contributor-scan:", "context_scope_empty:contributor-scan:", + "context_function_unresolved:contributor-scan:", "dynamic_external_or_depth_context_unresolved:contributor-scan:", ) retained = set(questions) diff --git a/tests/test_model_change_context.py b/tests/test_model_change_context.py index e7a6eba..509f8a0 100644 --- a/tests/test_model_change_context.py +++ b/tests/test_model_change_context.py @@ -328,3 +328,22 @@ def test_a_family_whose_context_query_answers_nothing_stays_unresolved(tmp_path) outcomes = {o.identity.family: o for o in result.question_outcomes} assert "access_control" in outcomes assert any("context_projection_unavailable" in gap for gap in result.scope.unresolved_boundaries) + + +def test_a_question_naming_a_function_the_file_does_not_define_says_so(tmp_path): + """M1b 12.3: a route-registration file names handlers defined elsewhere; that is not an empty file.""" + from dataclasses import replace as _replace + + (tmp_path / "source.js").write_text("readable\n") + boundary = {"kind": "context_boundary", "reason": "context_function_unresolved:contributor-scan", "function": "handleUpdate"} + unresolved = [{"kind": "context_summary"}, boundary] + backend = FamilyBackend({"source.js": vector() + unresolved}, {"dominance": [], "config": unresolved}) + target = _replace(region("source.js", "javascript"), function="handleUpdate", families=("injection",)) + result = scan_repository(tmp_path, [target], backend=backend, change_context=context("source.js"), population_complete=True) + outcome = result.question_outcomes[0] + assert outcome.status == "unresolved" and outcome.reason == "change_context_incomplete" + gaps = [g for g in result.scope.unresolved_boundaries if "context_function_unresolved" in g] + assert gaps, "the unresolvable function must be named, not reported as an empty scope" + # The gap is owned by the question that asked it, so it cannot block another question. + assert gaps[0].endswith(outcome.identity.question_id) + assert not any("context_scope_empty" in g for g in result.scope.unresolved_boundaries)