feat(pre-push-safety-net): carry query-to-operation provenance (M1b 10.8)

M1a recorded that the NodeGoat target is selected through a file-scoped question
naming no function while the engine reports method <lambda>2, and that this did
not block the comparison once the question completed. It was still only visible by
digging the method out of a serialized detail blob.

Evidence now carries the engine-reported method beside the question that produced
it, and the relationship between them is recorded: exact when the question named
that method, file_scope when the question named no function and the engine
supplied one, unresolved when the two disagree, and method_unreported when the
engine named none.

Recorded, not applied. No region is renamed, no benchmark function name is
injected into a region, and no coincident site is counted. The recorded-veto
report now reads the field instead of reparsing the detail blob.

1,371 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-16 18:55:31 +02:00
co-authored by Claude Opus 5
parent 9a67f54ea1
commit 541ee5813e
3 changed files with 41 additions and 7 deletions
+2 -1
View File
@@ -325,12 +325,13 @@ Preserve all completed task evidence below.
- _Depends: 10.2_
- _Requirements: 3.1, 3.3, 3.4_
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Gaps are now attributed: a boundary carrying a question id counts only when that question is in this candidate's required scope, an unattributed gap always counts because unknown ownership is not evidence of independence, and integrity degradations count. A declared exclusion, correspondence ambiguity and bounded reachability do not, because each was already weighed against this specific claim. The runner no longer hands every candidate the whole transaction's reason list, which had included other candidates' comparison outcomes.
- [ ] 10.8 Carry query-to-operation provenance into the reported evidence
- [x] 10.8 Carry query-to-operation provenance into the reported evidence
- The target is selected through a file-fallback question with `function=None` while rows report method `<lambda>2`. M1a showed this did not block the comparison once the question completed, so it is provenance quality rather than the blocker.
- Record verified query-to-operation provenance on the evidence; do not inject benchmark function names into regions or count coincident sites.
- Done when a reported operation names the question that produced it and the method the engine reported, with unresolved provenance explicit.
- _Boundary: Delta and Actionability Policy, contributor-scan normalization_
- _Requirements: 3.1, 3.4, 6.3_
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Evidence now carries the method the engine reported beside the question that produced it, so the link is inspectable rather than inferred or dug out of a detail blob. Provenance is recorded as `exact`, `file_scope` when the question named no function and the engine supplied one, `unresolved` when the two disagree, and `method_unreported` when the engine named none. Controls cover all four. No region is renamed, no benchmark function name is injected and no coincident site is counted; the state is recorded, not applied.
- [ ] 10.9 Rerun the three revisions through the production rules
- Replay the same pinned vulnerable, fixed and unchanged revisions **without** the recorded-veto flag under the named development budget. Demonstrate a developer-readable evaluation finding with a supported consequence, exact location, witness, change attribution and repair direction.
- Verify the fixed twin is analyzed and produces no actionable defect and the unchanged revision is not reported as a new regression. Keep qualification eligibility disabled; experimental evaluation output cannot advertise an unqualified hook alert.
+34 -1
View File
@@ -34,6 +34,26 @@ class EvidenceOperation(Contract):
source: str
discharged: bool
detail: str
# The method the engine reported this operation in. It need not equal the question's
# function: a file-scoped question names none, and the engine may report a synthetic
# name such as a lambda. Recorded so the link from question to operation is inspectable
# rather than inferred, and left None when the engine reported no method.
method: str | None = None
def operation_provenance(op: EvidenceOperation) -> str:
"""How the question that was asked relates to the operation that was reported.
`exact` when the question named this method, `file_scope` when the question named no
function and the engine supplied one, `method_unreported` when the engine named none, and
`unresolved` when the two disagree. Only a recorded state; it invents no correspondence
and never renames a region.
"""
if op.method is None:
return "method_unreported"
if op.question.function is None:
return "file_scope"
return "exact" if op.question.function == op.method else "unresolved"
@dataclass(frozen=True)
@@ -100,8 +120,10 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
kind: Literal["taint", "dominance", "config"]
if not isinstance(row, dict):
continue
method = None
if "sink" in row:
kind, path, line, op = "taint", row.get("sinkFile"), row.get("sinkLine"), row.get("sink")
method = row.get("sinkMethod")
if not isinstance(row.get("source"), str) or not row["source"].strip():
continue
if type(row.get("sanitized")) is not bool or type(row.get("bounded")) is not bool:
@@ -110,6 +132,7 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
discharged = row["sanitized"] or row["bounded"]
elif "operation" in row:
kind, path, line, op = "dominance", row.get("opFile"), row.get("opLine"), row.get("operation")
method = row.get("opMethod")
source = "obligation"
if not isinstance(row.get("dominatingGuards"), list) or not all(isinstance(g, str) for g in row["dominatingGuards"]):
continue
@@ -131,7 +154,17 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
):
continue
result.append(
EvidenceOperation(answer.identity, kind, path, int(str(line)), str(op), source, discharged, json.dumps(row, sort_keys=True))
EvidenceOperation(
answer.identity,
kind,
path,
int(str(line)),
str(op),
source,
discharged,
json.dumps(row, sort_keys=True),
str(method) if isinstance(method, str) and method.strip() else None,
)
)
return tuple(result)
+5 -5
View File
@@ -34,6 +34,7 @@ from openultrasast.push.policy import (
_context_boundaries,
_counterpart,
admit_candidates,
operation_provenance,
)
FLAG = "--experimental-record-vetoes"
@@ -227,9 +228,7 @@ def _summary(scan: ModelScanResult, lifted: ModelScanResult, counts: Counter[str
def _method(op: EvidenceOperation) -> str | None:
row = json.loads(op.detail)
value = row.get("sinkMethod") if op.mechanism == "taint" else row.get("opMethod") if op.mechanism == "dominance" else None
return value if isinstance(value, str) and value.strip() else None
return op.method
def _boundary_vetoes(stage: str, boundaries: Sequence[str]) -> list[RecordedVeto]:
@@ -277,12 +276,13 @@ def _finding_vetoes(
vetoes.extend(_boundary_vetoes("head_scan", _context_boundaries(head, question)))
vetoes.extend(_degradation_vetoes("head_scan", head))
method = _method(op)
if question.function is None or (method is not None and question.function != method):
provenance = operation_provenance(op)
if provenance != "exact":
vetoes.append(
RecordedVeto(
"identity",
"query_identity_differs_from_witness",
f"question function={question.function!r}; witness method={method!r}; recorded, not a policy veto",
f"provenance={provenance}; question function={question.function!r}; engine method={method!r}",
False,
)
)