mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 11:49:28 +00:00
feat(pre-push-safety-net): carry query-to-operation provenance (M1b 10.8)
M1a recorded that the NodeGoat target is selected through a file-scoped question naming no function while the engine reports method <lambda>2, and that this did not block the comparison once the question completed. It was still only visible by digging the method out of a serialized detail blob. Evidence now carries the engine-reported method beside the question that produced it, and the relationship between them is recorded: exact when the question named that method, file_scope when the question named no function and the engine supplied one, unresolved when the two disagree, and method_unreported when the engine named none. Recorded, not applied. No region is renamed, no benchmark function name is injected into a region, and no coincident site is counted. The recorded-veto report now reads the field instead of reparsing the detail blob. 1,371 tests passed, nine skipped. Rollout stays NO-GO. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5
parent
9a67f54ea1
commit
541ee5813e
@@ -325,12 +325,13 @@ Preserve all completed task evidence below.
|
||||
- _Depends: 10.2_
|
||||
- _Requirements: 3.1, 3.3, 3.4_
|
||||
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Gaps are now attributed: a boundary carrying a question id counts only when that question is in this candidate's required scope, an unattributed gap always counts because unknown ownership is not evidence of independence, and integrity degradations count. A declared exclusion, correspondence ambiguity and bounded reachability do not, because each was already weighed against this specific claim. The runner no longer hands every candidate the whole transaction's reason list, which had included other candidates' comparison outcomes.
|
||||
- [ ] 10.8 Carry query-to-operation provenance into the reported evidence
|
||||
- [x] 10.8 Carry query-to-operation provenance into the reported evidence
|
||||
- The target is selected through a file-fallback question with `function=None` while rows report method `<lambda>2`. M1a showed this did not block the comparison once the question completed, so it is provenance quality rather than the blocker.
|
||||
- Record verified query-to-operation provenance on the evidence; do not inject benchmark function names into regions or count coincident sites.
|
||||
- Done when a reported operation names the question that produced it and the method the engine reported, with unresolved provenance explicit.
|
||||
- _Boundary: Delta and Actionability Policy, contributor-scan normalization_
|
||||
- _Requirements: 3.1, 3.4, 6.3_
|
||||
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Evidence now carries the method the engine reported beside the question that produced it, so the link is inspectable rather than inferred or dug out of a detail blob. Provenance is recorded as `exact`, `file_scope` when the question named no function and the engine supplied one, `unresolved` when the two disagree, and `method_unreported` when the engine named none. Controls cover all four. No region is renamed, no benchmark function name is injected and no coincident site is counted; the state is recorded, not applied.
|
||||
- [ ] 10.9 Rerun the three revisions through the production rules
|
||||
- Replay the same pinned vulnerable, fixed and unchanged revisions **without** the recorded-veto flag under the named development budget. Demonstrate a developer-readable evaluation finding with a supported consequence, exact location, witness, change attribution and repair direction.
|
||||
- Verify the fixed twin is analyzed and produces no actionable defect and the unchanged revision is not reported as a new regression. Keep qualification eligibility disabled; experimental evaluation output cannot advertise an unqualified hook alert.
|
||||
|
||||
@@ -34,6 +34,26 @@ class EvidenceOperation(Contract):
|
||||
source: str
|
||||
discharged: bool
|
||||
detail: str
|
||||
# The method the engine reported this operation in. It need not equal the question's
|
||||
# function: a file-scoped question names none, and the engine may report a synthetic
|
||||
# name such as a lambda. Recorded so the link from question to operation is inspectable
|
||||
# rather than inferred, and left None when the engine reported no method.
|
||||
method: str | None = None
|
||||
|
||||
|
||||
def operation_provenance(op: EvidenceOperation) -> str:
|
||||
"""How the question that was asked relates to the operation that was reported.
|
||||
|
||||
`exact` when the question named this method, `file_scope` when the question named no
|
||||
function and the engine supplied one, `method_unreported` when the engine named none, and
|
||||
`unresolved` when the two disagree. Only a recorded state; it invents no correspondence
|
||||
and never renames a region.
|
||||
"""
|
||||
if op.method is None:
|
||||
return "method_unreported"
|
||||
if op.question.function is None:
|
||||
return "file_scope"
|
||||
return "exact" if op.question.function == op.method else "unresolved"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -100,8 +120,10 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
|
||||
kind: Literal["taint", "dominance", "config"]
|
||||
if not isinstance(row, dict):
|
||||
continue
|
||||
method = None
|
||||
if "sink" in row:
|
||||
kind, path, line, op = "taint", row.get("sinkFile"), row.get("sinkLine"), row.get("sink")
|
||||
method = row.get("sinkMethod")
|
||||
if not isinstance(row.get("source"), str) or not row["source"].strip():
|
||||
continue
|
||||
if type(row.get("sanitized")) is not bool or type(row.get("bounded")) is not bool:
|
||||
@@ -110,6 +132,7 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
|
||||
discharged = row["sanitized"] or row["bounded"]
|
||||
elif "operation" in row:
|
||||
kind, path, line, op = "dominance", row.get("opFile"), row.get("opLine"), row.get("operation")
|
||||
method = row.get("opMethod")
|
||||
source = "obligation"
|
||||
if not isinstance(row.get("dominatingGuards"), list) or not all(isinstance(g, str) for g in row["dominatingGuards"]):
|
||||
continue
|
||||
@@ -131,7 +154,17 @@ def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) ->
|
||||
):
|
||||
continue
|
||||
result.append(
|
||||
EvidenceOperation(answer.identity, kind, path, int(str(line)), str(op), source, discharged, json.dumps(row, sort_keys=True))
|
||||
EvidenceOperation(
|
||||
answer.identity,
|
||||
kind,
|
||||
path,
|
||||
int(str(line)),
|
||||
str(op),
|
||||
source,
|
||||
discharged,
|
||||
json.dumps(row, sort_keys=True),
|
||||
str(method) if isinstance(method, str) and method.strip() else None,
|
||||
)
|
||||
)
|
||||
return tuple(result)
|
||||
|
||||
|
||||
@@ -34,6 +34,7 @@ from openultrasast.push.policy import (
|
||||
_context_boundaries,
|
||||
_counterpart,
|
||||
admit_candidates,
|
||||
operation_provenance,
|
||||
)
|
||||
|
||||
FLAG = "--experimental-record-vetoes"
|
||||
@@ -227,9 +228,7 @@ def _summary(scan: ModelScanResult, lifted: ModelScanResult, counts: Counter[str
|
||||
|
||||
|
||||
def _method(op: EvidenceOperation) -> str | None:
|
||||
row = json.loads(op.detail)
|
||||
value = row.get("sinkMethod") if op.mechanism == "taint" else row.get("opMethod") if op.mechanism == "dominance" else None
|
||||
return value if isinstance(value, str) and value.strip() else None
|
||||
return op.method
|
||||
|
||||
|
||||
def _boundary_vetoes(stage: str, boundaries: Sequence[str]) -> list[RecordedVeto]:
|
||||
@@ -277,12 +276,13 @@ def _finding_vetoes(
|
||||
vetoes.extend(_boundary_vetoes("head_scan", _context_boundaries(head, question)))
|
||||
vetoes.extend(_degradation_vetoes("head_scan", head))
|
||||
method = _method(op)
|
||||
if question.function is None or (method is not None and question.function != method):
|
||||
provenance = operation_provenance(op)
|
||||
if provenance != "exact":
|
||||
vetoes.append(
|
||||
RecordedVeto(
|
||||
"identity",
|
||||
"query_identity_differs_from_witness",
|
||||
f"question function={question.function!r}; witness method={method!r}; recorded, not a policy veto",
|
||||
f"provenance={provenance}; question function={question.function!r}; engine method={method!r}",
|
||||
False,
|
||||
)
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user