diff --git a/.kiro/specs/pre-push-safety-net/tasks.md b/.kiro/specs/pre-push-safety-net/tasks.md index f511beb..1440098 100644 --- a/.kiro/specs/pre-push-safety-net/tasks.md +++ b/.kiro/specs/pre-push-safety-net/tasks.md @@ -266,6 +266,48 @@ Preserve all completed task evidence below. - _Requirements: 1.1, 2.1, 2.4, 3.4, 6.3, 7.1, 7.3_ - _Verified 2026-09-16:_ all three pinned revisions replayed through the production CLI under a named 900-second development budget; expectations frozen on disk first. Raw `2` witnesses present at lines 32–34; fixed twin analyzed and quiet (target question answered with zero rows, no deadline reason, three base-only findings); unchanged revision analyzed and recorded `unchanged`. Five vetoes fired and were recorded; the remaining blocker is `operation_correspondence_unresolved` for an operation on a changed line. Zero admitted defects. Evidence: [M1a record](m1a-recorded-vetoes.md) and `benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json`. Diagnosis only; no capability enabled and no latency claim. +- [ ] 10. Repair only the vetoes M1a recorded (release milestone M1b) +- [ ] 10.1 Establish operation correspondence for an operation on a changed line + - M1a measured this as the blocker that survives every other repair: a newly introduced operation sits on a changed line, has no base line anchor, and therefore cannot be classified even when every completeness veto is lifted. + - Bind the required base/head correspondence to the specific operation identity rather than to a line anchor. A complete base answer for the counterpart question that contains no operation with the same operation and source establishes absence; a matching operation elsewhere in that answer is movement, not novelty. + - Keep the existing conservative rules: an incomplete or failed base stays unknown, benign movement and renames stay unchanged, and lexical alignment is never semantic proof. + - Add absent, renamed, removed-guard, moved-operation and repeated-identical-operation controls. Done when a newly introduced operation on a changed line is classified new, while movement, renames and an incomplete base remain unchanged or unknown. + - _Boundary: Delta and Actionability Policy_ + - _Requirements: 2.4, 3.1, 3.4_ +- [ ] 10.2 Scope correspondence ambiguity to the operations it actually affects + - `ambiguous_line_correspondence` is recorded file-wide and inherited by every question, so repeated unchanged lines invalidate distinct, uniquely mapped operations elsewhere. + - Retain ambiguous mappings as unavailable and record which regions are ambiguous, so only an operation whose required correspondence needs an ambiguous line is vetoed. Do not delete the ambiguity check. + - Done when a uniquely mapped operation is comparable in a file that also contains repeated unchanged lines, while an operation needing an ambiguous line stays unresolved. + - _Boundary: Snapshot Adapter, Delta and Actionability Policy_ + - _Depends: 10.1_ + - _Requirements: 2.4, 3.4_ +- [ ] 10.3 Make the external and dynamic context boundary question-owned + - M1a recorded `dynamic_external_or_depth_context_unresolved` on the target's own question. The flag is computed over every in-scope method, so one unresolved call anywhere vetoes every question, including those whose relevant methods are fully resolved. + - Report the boundary with the methods whose destinations are unresolved, and inherit it into a question only when such a method participates in that question's change relationships. Unknown ownership remains a blocker; absent evidence of independence is not proof of independence. + - Done when a question whose relevant methods resolve completely is not vetoed by an unrelated external call, while a question that depends on an unresolved destination stays unresolved. + - _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_ + - _Requirements: 2.1, 2.2, 8.1, 8.2_ +- [ ] 10.4 Stop vendor exclusion acting as a global completion veto + - `vendor_semantics_unresolved` currently degrades every otherwise complete outcome to `graph_incomplete`, including on a complete 44-file first-party census with successful query receipts. + - Keep the exclusion physical and keep the coverage note. Let the question-owned dependency evidence from 10.3 carry whether a specific answer needed excluded semantics; a recorded dependency on an excluded path still blocks that question. + - Done when an unrelated vendor exclusion no longer invalidates a first-party answer, while a question with evidence of dependency on excluded code remains unresolved and the exclusion stays out of targets and graphs. + - _Boundary: contributor-scan graph/scope evidence, Cache and Engine Boundary_ + - _Depends: 10.3_ + - _Requirements: 2.2, 2.3, 8.2, 8.4_ +- [ ] 10.5 Carry query-to-operation provenance into the reported evidence + - The target is selected through a file-fallback question with `function=None` while rows report method `2`. M1a showed this did not block the comparison once the question completed, so it is provenance quality rather than the blocker. + - Record verified query-to-operation provenance on the evidence; do not inject benchmark function names into regions or count coincident sites. + - Done when a reported operation names the question that produced it and the method the engine reported, with unresolved provenance explicit. + - _Boundary: Delta and Actionability Policy, contributor-scan normalization_ + - _Requirements: 3.1, 3.4, 6.3_ +- [ ] 10.6 Rerun the three revisions through the production rules + - Replay the same pinned vulnerable, fixed and unchanged revisions **without** the recorded-veto flag under the named development budget. Demonstrate a developer-readable evaluation finding with a supported consequence, exact location, witness, change attribution and repair direction. + - Verify the fixed twin is analyzed and produces no actionable defect and the unchanged revision is not reported as a new regression. Keep qualification eligibility disabled; experimental evaluation output cannot advertise an unqualified hook alert. + - Done when one replayable record states which vetoes were repaired and which remain, with completed required checks and readable source and census receipts. + - _Boundary: Evaluation harness, full-run integration_ + - _Depends: 10.1, 10.2, 10.3, 10.4, 10.5_ + - _Requirements: 1.1, 2.1, 2.4, 3.1, 3.4, 6.3, 7.1, 7.3_ + ## Implementation Notes - Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied. diff --git a/src/openultrasast/push/policy.py b/src/openultrasast/push/policy.py index af79a96..29d8460 100644 --- a/src/openultrasast/push/policy.py +++ b/src/openultrasast/push/policy.py @@ -136,18 +136,33 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) -> return tuple(result) -def _base_location(op: EvidenceOperation, context: ChangeContext) -> tuple[str, int] | None: +def _correspondence(op: EvidenceOperation, context: ChangeContext) -> tuple[tuple[str, int] | None, str]: + """Where this head operation maps in the base, and when it does not, why. + + `changed_line` means the operation lies inside a recorded head edit, so no base line + can correspond to it. That is a different fact from `ambiguous`, where several base + lines could correspond, and from `unmapped`, where no anchor was recorded at all. + Only the first supports reasoning about an operation the change introduced. + """ matches = [] for anchor in context.line_correspondences: size = anchor.base_end_line - anchor.base_start_line if context.decode_path(anchor.head_path) == op.path and anchor.head_start_line <= op.line <= anchor.head_start_line + size: matches.append((context.decode_path(anchor.base_path), anchor.base_start_line + op.line - anchor.head_start_line)) if matches: - return matches[0] if len(set(matches)) == 1 else None + return (matches[0], "anchored") if len(set(matches)) == 1 else (None, "ambiguous") changed = {context.decode_path(p) for p in context.changed_paths} if op.path not in changed: - return op.path, op.line - return None + return (op.path, op.line), "unchanged_path" + introduced = any( + span.side == "head" and context.decode_path(span.path) == op.path and span.start_line <= op.line <= span.end_line + for span in context.spans + ) + return None, ("changed_line" if introduced else "unmapped") + + +def _base_location(op: EvidenceOperation, context: ChangeContext) -> tuple[str, int] | None: + return _correspondence(op, context)[0] def _change_evidence(op: EvidenceOperation, context: ChangeContext) -> tuple[str, ...]: @@ -361,10 +376,15 @@ def _compare_evidence( old: tuple[EvidenceOperation, ...] = () change: tuple[str, ...] = () if op is not None: - location = _base_location(op, context) + location, correspondence = _correspondence(op, context) counterpart = _counterpart(op.question, context) contextual = tuple(b for b in base_ops if b.question == counterpart and b.mechanism == op.mechanism) old = tuple(b for b in contextual if (b.path, b.line) == location and b.operation == op.operation) + # Identity, not position. An operation the change introduced has no base line to + # map to, so absence must be established over the complete counterpart answer. + # Match on the operation spelling alone: a benign identifier rename changes the + # source text without introducing anything, and must read as movement. + identical = tuple(b for b in contextual if b.operation == op.operation) change = _change_evidence(op, context) reason = "base_incomplete" complete = bool( @@ -419,6 +439,14 @@ def _compare_evidence( and not op.discharged ): novelty, reason = "new", "source_connection_absent_from_comparable_base" + elif correspondence == "changed_line" and change and op.mechanism == "taint": + # The edit introduced these lines. A complete counterpart answer that + # contains this operation elsewhere is movement, never novelty; only its + # total absence establishes that the change introduced the flow. + if identical: + reason = "operation_moved_within_change" + elif finding.rung == Rung.ENTAILED and not op.discharged: + novelty, reason = "new", "operation_absent_from_comparable_base" results.append(_candidate(finding, op, old, novelty, reason, change, context)) return tuple( replace( diff --git a/tests/test_push_policy.py b/tests/test_push_policy.py index 81be52a..71430ab 100644 --- a/tests/test_push_policy.py +++ b/tests/test_push_policy.py @@ -151,8 +151,11 @@ def test_new_source_at_unchanged_operation_needs_complete_base_answer(): def test_changed_new_operation_in_comparable_existing_file(): + # M1b 10.1 changed the first case: an operation the edit introduced has no base line to + # map to, so a complete counterpart answer containing no such operation now establishes + # absence. Before that repair this stayed unknown and no introduced flow could be seen. ctx = replace(context(), spans=(ChangedSpan("a.js", 4, 4, "head"),), line_correspondences=()) - assert delta(scan(), scan(rows=[], findings=False), ctx).novelty == "unknown" + assert delta(scan(), scan(rows=[], findings=False), ctx).novelty == "new" assert delta(scan(), scan(line=6), ctx).novelty == "unknown" # movement is not new @@ -272,3 +275,70 @@ def test_comparison_cache_reuses_only_completed_compatible_evidence(tmp_path, mo hits = cache.hits policy.compare_evidence(scan(), scan(status="unanswered"), **args) assert cache.hits == hits + + +def changed_line_context(*, ambiguous=False, head_line=4): + """The head edit covers the operation's line, so no base line can correspond to it.""" + return ChangeContext( + "base", + "head", + ("a.js",), + (), + (ChangedSpan("a.js", 2, 2, "base"), ChangedSpan("a.js", head_line, head_line, "head")), + (), + (), + (), + (), + line_correspondences=( + (LineCorrespondence("a.js", "a.js", 9, 9, head_line), LineCorrespondence("a.js", "a.js", 11, 11, head_line)) + if ambiguous + else () + ), + ) + + +def test_operation_introduced_on_a_changed_line_is_new_against_a_complete_base(): + """M1b 10.1: absence is established over the counterpart answer, not over a line anchor.""" + candidate = delta(scan(), scan(rows=[], findings=False), changed_line_context()) + assert (candidate.novelty, candidate.reason) == ("new", "operation_absent_from_comparable_base") + + +def test_moved_identical_operation_on_a_changed_line_is_not_new(): + moved = scan(line=9) # same operation and source, different line in the base + candidate = delta(scan(), moved, changed_line_context()) + assert candidate.novelty == "unknown" and candidate.reason == "operation_moved_within_change" + + +def test_changed_line_novelty_still_requires_a_complete_base(): + empty = scan(rows=[], findings=False) + for base in (scan(rows=[], status="unanswered", findings=False), replace(empty, degradations=({"reason": "cpg_empty"},))): + assert delta(scan(), base, changed_line_context()).novelty == "unknown" + assert delta(scan(), None, changed_line_context()).novelty == "unknown" + + +def test_ambiguous_correspondence_on_a_changed_line_is_never_new(): + candidate = delta(scan(), scan(rows=[], findings=False), changed_line_context(ambiguous=True)) + assert candidate.novelty == "unknown" and candidate.reason != "operation_absent_from_comparable_base" + + +def test_operation_outside_any_recorded_head_span_is_not_new(): + """An unmapped operation in a changed file is unresolved, not introduced.""" + candidate = delta(scan(), scan(rows=[], findings=False), changed_line_context(head_line=99)) + assert candidate.novelty == "unknown" and candidate.reason == "operation_correspondence_unresolved" + + +def test_discharged_or_unentailed_operation_on_a_changed_line_is_not_new(): + from openultrasast.model.pipeline import ModelFinding + + assert delta(scan(sanitized=True), scan(rows=[], findings=False), changed_line_context()).novelty != "new" + weak = scan() + weak = replace(weak, findings=(replace(weak.findings[0], rung=Rung.CORROBORATED),)) + assert delta(weak, scan(rows=[], findings=False), changed_line_context()).novelty != "new" + assert isinstance(weak.findings[0], ModelFinding) + + +def test_renamed_file_keeps_operation_identity_across_a_changed_line(): + spans = (ChangedSpan("old.js", 2, 2, "base"), ChangedSpan("a.js", 4, 4, "head")) + ctx = replace(changed_line_context(), renames=(PathRename("old.js", "a.js"),), spans=spans) + # Identical operation carried across the rename is movement, never novelty. + assert delta(scan(), scan(path="old.js"), ctx).novelty == "unknown"