mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
feat(pre-push-safety-net): compare targeted base and head evidence
This commit is contained in:
@@ -0,0 +1,41 @@
|
||||
# Group 4 implementation and verification
|
||||
|
||||
Date: 2026-09-13. The maintainer authorized group 4 as a whole. Scope is tasks
|
||||
4.1–4.3: targeted novelty comparison, actionability admission and compact reporting.
|
||||
Groups 1–3 are verified at `6ba2b06`; full transaction replay remains task 5.1.
|
||||
|
||||
Status: implementation in progress. Task completion requires independent review
|
||||
and fresh evidence. No production capability is enabled for normal hook alerts;
|
||||
capability admission still requires task 8.3 evaluation.
|
||||
|
||||
## Verification boundary
|
||||
|
||||
Exercise production policy functions with immutable Git snapshots and actual engine
|
||||
answers, plus deterministic error-path and equivalent-origin controls. Prove source
|
||||
bytes were read before interpreting empty answers. Synthetic evaluated capabilities
|
||||
exercise admission mechanics only and never establish real capability eligibility.
|
||||
The established gates protect detector regressions; these controls do not establish
|
||||
production hook latency, broad alert precision or feature-level readiness.
|
||||
|
||||
## Task 4.1 verification
|
||||
|
||||
VERIFIED (TASK scope), independent review APPROVED. Full suite: 1,167 passed,
|
||||
nine skipped in 122.44 seconds; global Ruff/format (233 files), mypy (110 source
|
||||
files), all three regression gates and diff checks passed. The first OFF control
|
||||
failed on novelty behavior. Review regressions reproduced false NEW classifications
|
||||
from wrong-mechanism base rows and non-string operations; both now stay unknown.
|
||||
|
||||
`benchmarks/measurements/2026-09-13-delta-comparison-smoke.json` records actual
|
||||
JavaScript guard removal as worsened, unchanged backlog moved by comments as
|
||||
unchanged, and a PHP source newly connected to a mapped unchanged sink as new.
|
||||
All inputs were read and both sides' questions completed. Current reviewed policy
|
||||
was independently rechecked against captured actual answers after the schema-only
|
||||
repair; the artifact preserves exact source hashes and that narrower provenance.
|
||||
Concurrent lab costs were 214.81, 214.52 and 291.01 seconds respectively under
|
||||
900-second budgets. These are not hook latency measurements.
|
||||
|
||||
Automatic dominance-only change-context projection remains an explicit upstream
|
||||
limitation for the later replay integration. These controls use declared questions
|
||||
and Git context directly; no missing-coverage result was erased. Ambiguous source
|
||||
and identifier changes remain unknown. Task 4.1 brings progress to 12/27 tasks;
|
||||
admission and rendering remain pending within this authorized group.
|
||||
@@ -178,3 +178,22 @@ incomplete coverage. A synthetic graph file must never compensate for an omitted
|
||||
real source, and a wrapper must not discard evidence supplied by a frontend.
|
||||
See `implementation-group-3.md` for final evidence. Eleven of 27 tasks are complete;
|
||||
change attribution, actionability admission and compact output are next.
|
||||
|
||||
## Group 4 comparison controls (2026-09-13)
|
||||
|
||||
Real JavaScript dominance rows distinguish a guarded base operation from an
|
||||
unguarded head operation while retaining a guarded sibling. The direct task 4.1
|
||||
comparator can consume these answers with declared questions and immutable Git
|
||||
span correspondence. This does not establish automatic context projection for
|
||||
that family: the existing driver's change-context evidence pass currently projects
|
||||
taint questions, and a dominance-only scan with `change_context` reports
|
||||
`context_projection_unavailable:contributor-scan`. Preserve that limitation in the
|
||||
full replay integration at 5.1; any extension belongs to the contributor-scan
|
||||
context contract, not a hook-specific graph or detector workaround.
|
||||
|
||||
Duplicate identical source lines also remain ambiguous to the snapshot adapter.
|
||||
Controls with unique lines prove the narrow supported correspondence path; they
|
||||
do not establish general identity across arbitrary refactoring. Changed source
|
||||
text alone is insufficient to distinguish a new source from a benign identifier
|
||||
rename. Those comparisons remain unknown. A completed comparable base with no flow
|
||||
to an unchanged mapped sink supplies a supported new-connection control instead.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-13T09:50:01.821464+00:00",
|
||||
"updated_at": "2026-09-13T10:26:47.307154+00:00",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -19,7 +19,7 @@
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true,
|
||||
"implementation_approval_note": "Groups 1\u20133 verified complete. Contributor-scan frontend-retention prerequisite 2.18 approved and verified. Remaining approved tasks require independent review and fresh verification.",
|
||||
"implementation_approval_note": "Groups 1\u20133 verified complete. Maintainer explicitly authorized group 4 as a whole; each task requires independent review and fresh verification.",
|
||||
"depends_on": [
|
||||
"contributor-scan"
|
||||
],
|
||||
@@ -55,6 +55,11 @@
|
||||
"at": "2026-09-13T09:17:01.788332+00:00",
|
||||
"source": "approved",
|
||||
"action": "Approved contributor-scan frontend retention prerequisite; implement it and resume task 3.4 review and verification."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T10:26:47.307154+00:00",
|
||||
"source": "move on to group 4 as a whole",
|
||||
"action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending."
|
||||
}
|
||||
],
|
||||
"task_review": {
|
||||
@@ -78,14 +83,16 @@
|
||||
"3.1",
|
||||
"3.2",
|
||||
"3.3",
|
||||
"3.4"
|
||||
"3.4",
|
||||
"4.1"
|
||||
],
|
||||
"remaining_tasks": 16,
|
||||
"verification_record": "implementation-group-3.md",
|
||||
"remaining_tasks": 15,
|
||||
"verification_record": "implementation-group-4.md",
|
||||
"verification_records": [
|
||||
"implementation-group-1.md",
|
||||
"implementation-group-2.md",
|
||||
"implementation-group-3.md"
|
||||
"implementation-group-3.md",
|
||||
"implementation-group-4.md"
|
||||
],
|
||||
"next_group": 4,
|
||||
"blocked_tasks": []
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Implementation Plan: pre-push-safety-net
|
||||
|
||||
Status: task plan approved by the maintainer's "approve." on 2026-09-12. Groups 1–3 (tasks 1.1–3.4)
|
||||
are complete after independent review and fresh verification (11/27 subtasks). Groups 4–8 remain pending;
|
||||
are complete after independent review and fresh verification (12/27 subtasks, including task 4.1). Tasks 4.2–4.3 and groups 5–8 remain pending;
|
||||
the hook and capability admission are not yet implemented. See `implementation-group-1.md` and `implementation-group-2.md`.
|
||||
|
||||
## Execution contract
|
||||
@@ -113,10 +113,11 @@ the hook and capability admission are not yet implemented. See `implementation-g
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; 1,151 passed, nine skipped; Ruff/format/mypy pass. Real PHP/JavaScript graphs retain tests/browser source, omit vendor nodes and share one ranker limit. Named census survives PHP aggregation; missing source names and unreadable discovery inputs remain explicit. Python exercises all three batch census schemas. Evidence: `benchmarks/measurements/2026-09-13-partition-scope-smoke.json`. No hook capability or latency claim.
|
||||
|
||||
- [ ] 4. Implement change attribution and actionable output
|
||||
- [ ] 4.1 Compare targeted base/head evidence for defect novelty
|
||||
- [x] 4.1 Compare targeted base/head evidence for defect novelty
|
||||
- Establish stable mechanism/operation identity with rename/span mapping and collect comparable base evidence under the same semantics and remaining deadline.
|
||||
- Distinguish new, worsened, unchanged and unknown; an absent finding from a failed/truncated base is unknown. Support removed guards and new source connections at unchanged sinks.
|
||||
- Done when benign movement and unchanged backlog do not become new defects, while a supported removed-guard case has a concrete change-to-witness relationship.
|
||||
- _Verified 2026-09-13:_ independent review APPROVED; 1,167 tests passed, nine skipped; real JavaScript guard/movement and PHP connection controls passed. Final policy rechecked against captured actual answers after malformed-schema repair. See `implementation-group-4.md`.
|
||||
- _Boundary: Delta and Actionability Policy, engine comparison integration_
|
||||
- _Depends: 2.3, 3.3, 3.4_
|
||||
- _Requirements: 2.4, 3.1, 3.4, 4.1_
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
{
|
||||
"counts": {
|
||||
"load_bearing": 99,
|
||||
"load_bearing": 100,
|
||||
"standalone_capability": 9
|
||||
},
|
||||
"modules": 108,
|
||||
"note": "Refreshed after pre-push-safety-net task 3.4: first-party frontend partitions share the authoritative ranker and execution budget.",
|
||||
"modules": 109,
|
||||
"note": "Refreshed after pre-push-safety-net task 4.1: targeted base comparison reuses existing arbiters and shared execution budget; replay integration remains task 5.1.",
|
||||
"orphaned": [],
|
||||
"rows": [
|
||||
{
|
||||
@@ -116,7 +116,8 @@
|
||||
"config",
|
||||
"cpg.artifact",
|
||||
"model.contracts",
|
||||
"push.contracts"
|
||||
"push.contracts",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "contracts",
|
||||
"reason": "pre-push-safety-net Req 7.3: strict shared manifest serialization, independent of push"
|
||||
@@ -132,7 +133,8 @@
|
||||
"model.partitions",
|
||||
"model.pipeline",
|
||||
"model.scan",
|
||||
"model.taint"
|
||||
"model.taint",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "cpg",
|
||||
"reason": "model-grounded-detection Req 4: the CPG seam package"
|
||||
@@ -153,7 +155,8 @@
|
||||
"model.partitions",
|
||||
"model.pipeline",
|
||||
"model.scan",
|
||||
"model.taint"
|
||||
"model.taint",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "cpg.backend",
|
||||
"reason": "model-grounded-detection Req 4.1/4.5: the single Joern subprocess boundary"
|
||||
@@ -356,6 +359,7 @@
|
||||
"model.taint",
|
||||
"preprocess",
|
||||
"push.contracts",
|
||||
"push.policy",
|
||||
"push.snapshot"
|
||||
],
|
||||
"module": "model",
|
||||
@@ -399,6 +403,7 @@
|
||||
"model.regions",
|
||||
"model.scan",
|
||||
"push.contracts",
|
||||
"push.policy",
|
||||
"push.snapshot"
|
||||
],
|
||||
"module": "model.contracts",
|
||||
@@ -450,7 +455,8 @@
|
||||
"model.judge",
|
||||
"model.pipeline",
|
||||
"model.scan",
|
||||
"model.taint"
|
||||
"model.taint",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "model.ladder",
|
||||
"reason": "model-grounded-detection Req 5: the evidence ladder every finding carries"
|
||||
@@ -476,7 +482,8 @@
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"model.scan"
|
||||
"model.scan",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "model.pipeline",
|
||||
"reason": "model-grounded-detection Req 8: the enumerator proposes, the LLM answers, the model disposes"
|
||||
@@ -486,7 +493,8 @@
|
||||
"importers": [
|
||||
"cli",
|
||||
"model.layout",
|
||||
"model.scan"
|
||||
"model.scan",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "model.regions",
|
||||
"reason": "contributor-scan Req 2: regions from entry points, with the families each admits"
|
||||
@@ -502,7 +510,8 @@
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [
|
||||
"cli"
|
||||
"cli",
|
||||
"push.policy"
|
||||
],
|
||||
"module": "model.scan",
|
||||
"reason": "contributor-scan Req 3: the repository driver -- one CPG, one budget, ranked spend"
|
||||
@@ -667,6 +676,12 @@
|
||||
"module": "push.contracts",
|
||||
"reason": "pre-push-safety-net Req 5.5/7.3: immutable comparisons and independent result statuses; runner pending"
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [],
|
||||
"module": "push.policy",
|
||||
"reason": "pre-push-safety-net Req 2.4/3.1/3.4/4.1: targeted same-semantics base comparison; replay integration pending"
|
||||
},
|
||||
{
|
||||
"classification": "load_bearing",
|
||||
"importers": [],
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -197,3 +197,20 @@ unshipped test-path records. Runtime origin remains unspecified without declarat
|
||||
readable Python graph and exercises the production taint, dominance and configuration
|
||||
batch queries. Each must return the actual source filename in its census. This
|
||||
checks query compilation and census transport, not security-property coverage.
|
||||
|
||||
## Targeted novelty comparison smoke
|
||||
|
||||
After rebuilding the image, run a declared JavaScript guard-removal comparison:
|
||||
|
||||
```bash
|
||||
docker run --rm -i --network none --entrypoint python openultrasast:dev - --case guard < ops/smoke_delta.py
|
||||
```
|
||||
|
||||
Use `--case movement` for unchanged JavaScript backlog shifted by comments, or
|
||||
`--case connection` for a PHP input newly reaching an unchanged operation. Every
|
||||
case verifies immutable base/head bytes, runs the existing driver and targeted
|
||||
comparison under one 900-second lab budget, and checks that dirty checkout/index
|
||||
state survives. Output contains an evidence JSON object followed by a completion
|
||||
JSON object; success requires exit zero and `verified: true` in that final object.
|
||||
These declared regions exercise comparison mechanics; full replay, normal alert
|
||||
admission and hook latency are separate evaluation work.
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Real Git/Joern novelty controls; no hook admission or latency claim."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from dataclasses import asdict
|
||||
from pathlib import Path
|
||||
|
||||
from openultrasast.cpg.backend import JoernBackend
|
||||
from openultrasast.model.contracts import ExecutionBudget
|
||||
from openultrasast.model.regions import ScanRegion
|
||||
from openultrasast.model.scan import ScanBudget, scan_repository
|
||||
from openultrasast.push.policy import compare_targeted_base, semantics_digest
|
||||
from openultrasast.push.snapshot import SnapshotAdapter
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--case", choices=("guard", "movement", "connection"), required=True)
|
||||
args = parser.parse_args()
|
||||
before = (
|
||||
b"export function handler(req) {\n"
|
||||
b" if (!authenticate(req)) return;\n"
|
||||
b" deleteUser(req.params.id);\n} // handler\n"
|
||||
b"export function sibling(other) {\n"
|
||||
b" if (!authenticate(other)) return;\n"
|
||||
b" deleteUser(other.params.id);\n} // sibling\n"
|
||||
)
|
||||
after = before.replace(b" if (!authenticate(req)) return;\n", b"", 1)
|
||||
language, family, filename = "javascript", "access_control", "api.js"
|
||||
expected = "worsened"
|
||||
if args.case == "movement":
|
||||
before = after
|
||||
after = b"// documentation moved this existing operation\n\n" + before
|
||||
expected = "unchanged"
|
||||
elif args.case == "connection":
|
||||
language, family, filename = "php", "injection", "api.php"
|
||||
before = b'<?php\nfunction handler($req) {\n $value = "fixed";\n eval($value);\n}\n'
|
||||
after = before.replace(b'$value = "fixed";', b"$value = $req;")
|
||||
expected = "new"
|
||||
started = time.monotonic()
|
||||
budget = ExecutionBudget(started + 900, 2.0)
|
||||
report: dict[str, object] = {
|
||||
"case": args.case,
|
||||
"purpose": (
|
||||
"Real targeted comparison with declared regions and Git context; no automatic context projection, full replay or admission"
|
||||
),
|
||||
"shared_budget_seconds": 900,
|
||||
}
|
||||
with tempfile.TemporaryDirectory(prefix="ousast-delta-smoke-") as temporary:
|
||||
repo = Path(temporary)
|
||||
|
||||
def git(*parts: str) -> bytes:
|
||||
return subprocess.run(["git", "-C", str(repo), *parts], check=True, capture_output=True, timeout=10).stdout
|
||||
|
||||
git("init")
|
||||
git("config", "user.name", "Delta smoke")
|
||||
git("config", "user.email", "[email protected]")
|
||||
git("config", "core.hooksPath", os.devnull)
|
||||
source = repo / filename
|
||||
source.write_bytes(before)
|
||||
git("add", filename)
|
||||
git("commit", "-m", "base")
|
||||
base_oid = git("rev-parse", "HEAD").decode().strip()
|
||||
source.write_bytes(after)
|
||||
git("add", filename)
|
||||
git("commit", "-m", args.case)
|
||||
head_oid = git("rev-parse", "HEAD").decode().strip()
|
||||
source.write_bytes(b"uncommitted content\n")
|
||||
index = (repo / ".git/index").read_bytes()
|
||||
status = git("status", "--porcelain=v1", "-z")
|
||||
adapter = SnapshotAdapter(repo)
|
||||
(comparison,) = adapter.resolve_updates(f"refs/heads/topic {head_oid} refs/heads/topic {base_oid}\n").comparisons
|
||||
context = adapter.compare(comparison, declaration_paths=(), budget=budget)
|
||||
regions = (ScanRegion(filename, "handler", language, (family,), 1.0, "entry_point"),)
|
||||
backend = JoernBackend(heap_mb=1024)
|
||||
semantics = semantics_digest(engine_identity="joern-4.0.625/frontend-retention-v1", options={"max_regions": 1})
|
||||
with adapter.materialize(base_oid, budget=budget) as base_snapshot, adapter.materialize(head_oid, budget=budget) as head_snapshot:
|
||||
assert (base_snapshot.root / filename).read_bytes() == before and before
|
||||
assert (head_snapshot.root / filename).read_bytes() == after and after
|
||||
head = scan_repository(
|
||||
head_snapshot.root,
|
||||
regions,
|
||||
backend=backend,
|
||||
budget=ScanBudget(max_regions=1),
|
||||
execution_budget=budget,
|
||||
ranking_mode="evidence",
|
||||
unit="delta-smoke",
|
||||
population_complete=True,
|
||||
)
|
||||
assert head.findings and head.scope and head.question_outcomes, head
|
||||
assert not head.degradations and all(q.status == "completed" for q in head.question_outcomes), head
|
||||
delta = compare_targeted_base(
|
||||
base_snapshot.root,
|
||||
head=head,
|
||||
head_regions=regions,
|
||||
base_regions=regions,
|
||||
context=context,
|
||||
backend=backend,
|
||||
execution_budget=budget,
|
||||
scan_budget=ScanBudget(max_regions=1),
|
||||
head_semantics=semantics,
|
||||
base_semantics=semantics,
|
||||
)
|
||||
report.update(
|
||||
{
|
||||
"comparison": comparison.to_payload(),
|
||||
"context": context.to_payload(),
|
||||
"base_input": {"bytes": len(before), "sha256": hashlib.sha256(before).hexdigest()},
|
||||
"head_input": {"bytes": len(after), "sha256": hashlib.sha256(after).hexdigest()},
|
||||
"head": asdict(head),
|
||||
"delta": asdict(delta),
|
||||
"expected_novelty": expected,
|
||||
}
|
||||
)
|
||||
# Emit evidence before assertions so a failed instrument cannot look like a clean zero.
|
||||
print(json.dumps(report, indent=2), flush=True)
|
||||
assert delta.base_scan and delta.base_scan.question_outcomes, delta
|
||||
assert not delta.base_scan.degradations, delta
|
||||
assert all(q.status == "completed" for q in delta.base_scan.question_outcomes), delta
|
||||
assert delta.candidates and all(c.novelty == expected for c in delta.candidates), delta
|
||||
assert not delta.coverage_reasons, delta
|
||||
assert not base_snapshot.root.exists() and not head_snapshot.root.exists()
|
||||
assert source.read_bytes() == b"uncommitted content\n"
|
||||
assert (repo / ".git/index").read_bytes() == index
|
||||
assert git("status", "--porcelain=v1", "-z") == status
|
||||
# A separate completion record distinguishes all assertions passing from partial JSON.
|
||||
print(json.dumps({"verified": True, "case": args.case, "elapsed_seconds": round(time.monotonic() - started, 2)}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -28,6 +28,7 @@ ROOTS = ("gate", "map_gate", "pair_gate", "cli", "__main__")
|
||||
|
||||
# Modules that stand alone: their own CLI subcommand or external protocol, deliberately off the scan path.
|
||||
SPEC_OWNED = {
|
||||
"push.policy": "pre-push-safety-net Req 2.4/3.1/3.4/4.1: targeted same-semantics base comparison; replay integration pending",
|
||||
"push.snapshot": "pre-push-safety-net Req 1.1/1.3/1.4/7.1: local immutable ref resolution; runner integration pending",
|
||||
"contracts": "pre-push-safety-net Req 7.3: strict shared manifest serialization, independent of push",
|
||||
"model.contracts": "pre-push-safety-net Req 4.1/7.3: generic deadline, change and exact scope contracts; integration pending",
|
||||
|
||||
@@ -0,0 +1,374 @@
|
||||
"""Compare arbiter-backed evidence from immutable revisions; this does not admit alerts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import time
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
from openultrasast.contracts import Contract
|
||||
from openultrasast.cpg.backend import CpgResult
|
||||
from openultrasast.model.contracts import ChangeContext, ExecutionBudget, QuestionIdentity
|
||||
from openultrasast.model.ladder import Rung
|
||||
from openultrasast.model.pipeline import ModelFinding, _arbitrate_all
|
||||
from openultrasast.model.regions import ScanRegion
|
||||
from openultrasast.model.scan import ModelScanResult, ScanBudget, _prefetched, _spec_for, scan_repository
|
||||
from openultrasast.model.specs import ConfigSpec, DominanceSpec, TaintSpec
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EvidenceOperation(Contract):
|
||||
question: QuestionIdentity
|
||||
mechanism: Literal["taint", "dominance", "config"]
|
||||
path: str
|
||||
line: int
|
||||
operation: str
|
||||
source: str
|
||||
discharged: bool
|
||||
detail: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CandidateDelta(Contract):
|
||||
family: str
|
||||
site: str
|
||||
rung: str
|
||||
witness: str | None
|
||||
defect_id: str
|
||||
novelty: Literal["new", "worsened", "unchanged", "unknown"]
|
||||
reason: str
|
||||
head_operation: EvidenceOperation | None
|
||||
base_operations: tuple[EvidenceOperation, ...]
|
||||
change_evidence: tuple[str, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DeltaComparison:
|
||||
candidates: tuple[CandidateDelta, ...]
|
||||
base_scan: ModelScanResult | None
|
||||
semantics: str
|
||||
coverage_reasons: tuple[str, ...]
|
||||
|
||||
|
||||
def semantics_digest(*, engine_identity: str, options: dict[str, object]) -> str:
|
||||
"""Fingerprint shipped semantics plus caller-owned engine/options provenance.
|
||||
|
||||
Include frontend adaptation, declaration/exclusion policy and scan options in options.
|
||||
This is an identity, never evidence of engine liveness or input readability.
|
||||
"""
|
||||
root = Path(__file__).resolve().parents[1]
|
||||
files = sorted([*(root / "cpg" / "queries").glob("*.sc"), *(root / "ruleset").rglob("*.toml"), *(root / "model").glob("*.py")])
|
||||
payload = {
|
||||
"engine": engine_identity,
|
||||
"options": options,
|
||||
"files": [(str(p.relative_to(root)), hashlib.sha256(p.read_bytes()).hexdigest()) for p in files],
|
||||
}
|
||||
return hashlib.sha256(json.dumps(payload, sort_keys=True).encode()).hexdigest()
|
||||
|
||||
|
||||
def _operations(scan: ModelScanResult, budget: ExecutionBudget | None = None) -> tuple[EvidenceOperation, ...]:
|
||||
result = []
|
||||
for answer in scan.question_outcomes:
|
||||
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
|
||||
break
|
||||
if answer.status != "completed" or answer.raw_rows_json is None:
|
||||
continue
|
||||
spec = _spec_for(answer.identity.family, answer.identity.language)
|
||||
expected = (
|
||||
"taint"
|
||||
if isinstance(spec, TaintSpec)
|
||||
else "dominance"
|
||||
if isinstance(spec, DominanceSpec)
|
||||
else "config"
|
||||
if isinstance(spec, ConfigSpec)
|
||||
else None
|
||||
)
|
||||
for row in json.loads(answer.raw_rows_json):
|
||||
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
|
||||
break
|
||||
kind: Literal["taint", "dominance", "config"]
|
||||
if not isinstance(row, dict):
|
||||
continue
|
||||
if "sink" in row:
|
||||
kind, path, line, op = "taint", row.get("sinkFile"), row.get("sinkLine"), row.get("sink")
|
||||
if not isinstance(row.get("source"), str) or not row["source"].strip():
|
||||
continue
|
||||
if type(row.get("sanitized")) is not bool or type(row.get("bounded")) is not bool:
|
||||
continue
|
||||
source = json.dumps([row.get("sourceKind", "unknown"), row.get("source")], separators=(",", ":"))
|
||||
discharged = row["sanitized"] or row["bounded"]
|
||||
elif "operation" in row:
|
||||
kind, path, line, op = "dominance", row.get("opFile"), row.get("opLine"), row.get("operation")
|
||||
source = "obligation"
|
||||
if not isinstance(row.get("dominatingGuards"), list) or not all(isinstance(g, str) for g in row["dominatingGuards"]):
|
||||
continue
|
||||
discharged = bool(row["dominatingGuards"])
|
||||
elif "setting" in row:
|
||||
kind, path, line, op = "config", row.get("file"), row.get("line"), row.get("setting")
|
||||
source = "configuration"
|
||||
discharged = False
|
||||
else:
|
||||
continue
|
||||
if (
|
||||
kind != expected
|
||||
or not isinstance(path, str)
|
||||
or not path
|
||||
or not isinstance(op, str)
|
||||
or not op.strip()
|
||||
or not str(line).isdigit()
|
||||
or int(str(line)) < 1
|
||||
):
|
||||
continue
|
||||
result.append(
|
||||
EvidenceOperation(answer.identity, kind, path, int(str(line)), str(op), source, discharged, json.dumps(row, sort_keys=True))
|
||||
)
|
||||
return tuple(result)
|
||||
|
||||
|
||||
def _base_location(op: EvidenceOperation, context: ChangeContext) -> tuple[str, int] | None:
|
||||
matches = []
|
||||
for anchor in context.line_correspondences:
|
||||
size = anchor.base_end_line - anchor.base_start_line
|
||||
if context.decode_path(anchor.head_path) == op.path and anchor.head_start_line <= op.line <= anchor.head_start_line + size:
|
||||
matches.append((context.decode_path(anchor.base_path), anchor.base_start_line + op.line - anchor.head_start_line))
|
||||
if matches:
|
||||
return matches[0] if len(set(matches)) == 1 else None
|
||||
changed = {context.decode_path(p) for p in context.changed_paths}
|
||||
if op.path not in changed:
|
||||
return op.path, op.line
|
||||
return None
|
||||
|
||||
|
||||
def _change_evidence(op: EvidenceOperation, context: ChangeContext) -> tuple[str, ...]:
|
||||
paths = {op.path, op.question.path}
|
||||
for relation in context.relationships:
|
||||
if relation.target == op.question and relation.evidence:
|
||||
paths.add(relation.source.path)
|
||||
paths.update(context.decode_path(r.base_path) for r in context.renames if context.decode_path(r.head_path) in paths)
|
||||
return tuple(
|
||||
f"{span.side}:{context.decode_path(span.path)}:{span.start_line}-{span.end_line}"
|
||||
for span in context.spans
|
||||
if context.decode_path(span.path) in paths
|
||||
)
|
||||
|
||||
|
||||
def _candidate(
|
||||
finding: ModelFinding,
|
||||
op: EvidenceOperation | None,
|
||||
bases: tuple[EvidenceOperation, ...],
|
||||
novelty: Literal["new", "worsened", "unchanged", "unknown"],
|
||||
reason: str,
|
||||
change: tuple[str, ...],
|
||||
context: ChangeContext,
|
||||
) -> CandidateDelta:
|
||||
location = _base_location(op, context) if op else None
|
||||
identity = [
|
||||
finding.family,
|
||||
op.mechanism if op else "unknown",
|
||||
location or ((op.path, op.line) if op else finding.site),
|
||||
op.operation if op else finding.site,
|
||||
op.source if op else "unknown",
|
||||
op.question.function if op else None,
|
||||
]
|
||||
key = hashlib.sha256(json.dumps(identity, sort_keys=True).encode()).hexdigest()
|
||||
return CandidateDelta(
|
||||
finding.family, finding.site, finding.rung.value, finding.witness or None, key, novelty, reason, op, bases, change
|
||||
)
|
||||
|
||||
|
||||
def _valid_answer(scan: ModelScanResult, question: QuestionIdentity, operations: tuple[EvidenceOperation, ...]) -> bool:
|
||||
answer = next((q for q in scan.question_outcomes if q.identity == question), None)
|
||||
if answer is None or answer.raw_rows_json is None:
|
||||
return False
|
||||
rows = json.loads(answer.raw_rows_json)
|
||||
# Query bodies are operation rows. An unrecognized/malformed row is not an empty answer.
|
||||
return len(rows) == sum(op.question == question for op in operations)
|
||||
|
||||
|
||||
def _context_boundaries(scan: ModelScanResult) -> tuple[str, ...]:
|
||||
# A missing scheduling vector is not missing arbiter evidence after its question completed.
|
||||
return tuple(b for b in scan.scope.unresolved_boundaries if not b.startswith("evidence_unknown:")) if scan.scope else ("scope_missing",)
|
||||
|
||||
|
||||
def _counterpart(question: QuestionIdentity, context: ChangeContext) -> QuestionIdentity:
|
||||
renamed = {context.decode_path(r.head_path): context.decode_path(r.base_path) for r in context.renames}
|
||||
return replace(question, path=renamed.get(question.path, question.path))
|
||||
|
||||
|
||||
def _proven_finding(finding: ModelFinding, op: EvidenceOperation, head: ModelScanResult) -> bool:
|
||||
"""Match the existing arbiter's exact witness, not just a coincident line number."""
|
||||
answer = next((q for q in head.question_outcomes if q.identity == op.question), None)
|
||||
spec = _spec_for(op.question.family, op.question.language)
|
||||
if answer is None or answer.raw_rows_json is None or spec is None:
|
||||
return False
|
||||
rows = json.loads(answer.raw_rows_json)
|
||||
verdicts = _arbitrate_all(
|
||||
CpgResult(Path("prefetched"), _prefetched(rows)),
|
||||
spec,
|
||||
path=op.question.path,
|
||||
function=op.question.function or "",
|
||||
parameter_sources=False,
|
||||
call_depth=0,
|
||||
)
|
||||
return any(v.witness == finding.witness and v.rung == finding.rung for v in verdicts)
|
||||
|
||||
|
||||
def compare_evidence(
|
||||
head: ModelScanResult,
|
||||
base: ModelScanResult | None,
|
||||
*,
|
||||
context: ChangeContext,
|
||||
head_semantics: str,
|
||||
base_semantics: str,
|
||||
execution_budget: ExecutionBudget | None = None,
|
||||
) -> tuple[CandidateDelta, ...]:
|
||||
"""Absence counts only in completed comparable scope, never from scan silence."""
|
||||
head_ops, base_ops = _operations(head, execution_budget), _operations(base, execution_budget) if base else ()
|
||||
results = []
|
||||
for finding in head.findings:
|
||||
if execution_budget is not None and time.monotonic() >= execution_budget.deadline_monotonic:
|
||||
results.append(_candidate(finding, None, (), "unknown", "deadline_exhausted", (), context))
|
||||
continue
|
||||
matching = [
|
||||
op
|
||||
for op in head_ops
|
||||
if op.question.family == finding.family
|
||||
and finding.site.startswith(f"{op.path}:{op.line}:")
|
||||
and _proven_finding(finding, op, head)
|
||||
]
|
||||
if len(matching) > 1:
|
||||
matching = [
|
||||
op
|
||||
for op in matching
|
||||
if op.mechanism != "taint" or finding.witness.startswith(str(json.loads(op.detail).get("source")) + " -> ")
|
||||
]
|
||||
op = matching[0] if len(matching) == 1 else None
|
||||
novelty: Literal["new", "worsened", "unchanged", "unknown"] = "unknown"
|
||||
reason = "witness_identity_unresolved"
|
||||
old: tuple[EvidenceOperation, ...] = ()
|
||||
change: tuple[str, ...] = ()
|
||||
if op is not None:
|
||||
location = _base_location(op, context)
|
||||
counterpart = _counterpart(op.question, context)
|
||||
contextual = tuple(b for b in base_ops if b.question == counterpart and b.mechanism == op.mechanism)
|
||||
old = tuple(b for b in contextual if (b.path, b.line) == location and b.operation == op.operation)
|
||||
change = _change_evidence(op, context)
|
||||
reason = "base_incomplete"
|
||||
complete = bool(
|
||||
base
|
||||
and base.scope
|
||||
and base.scope.population_complete
|
||||
and not base.scope.deferred
|
||||
and not _context_boundaries(base)
|
||||
and not base.degradations
|
||||
and _valid_answer(base, counterpart, base_ops)
|
||||
and counterpart in {q.identity for q in base.scope.selected}
|
||||
and counterpart in {q.identity for q in base.question_outcomes if q.status == "completed"}
|
||||
and {q.identity for q in base.scope.selected} == {q.identity for q in base.question_outcomes}
|
||||
and all(q.status == "completed" for q in base.question_outcomes)
|
||||
)
|
||||
if head_semantics != base_semantics:
|
||||
reason = "semantics_mismatch"
|
||||
elif context.base_revision is None:
|
||||
reason = "base_unavailable"
|
||||
elif context.unresolved_boundaries:
|
||||
reason = "change_context_unresolved"
|
||||
elif (
|
||||
not head.scope
|
||||
or _context_boundaries(head)
|
||||
or head.degradations
|
||||
or not _valid_answer(head, op.question, head_ops)
|
||||
or op.question not in {q.identity for q in head.scope.selected}
|
||||
):
|
||||
reason = "head_context_incomplete"
|
||||
elif execution_budget is not None and time.monotonic() >= execution_budget.deadline_monotonic:
|
||||
reason = "deadline_exhausted"
|
||||
elif complete:
|
||||
same = tuple(b for b in old if b.source == op.source)
|
||||
reason = "operation_correspondence_unresolved"
|
||||
if any(
|
||||
b.discharged == op.discharged
|
||||
and (op.mechanism != "config" or json.loads(b.detail).get("literalArgs") == json.loads(op.detail).get("literalArgs"))
|
||||
for b in same
|
||||
):
|
||||
novelty, reason = "unchanged", "same_supported_mechanism"
|
||||
elif old and change and finding.rung == Rung.ENTAILED and not op.discharged:
|
||||
if same and all(b.discharged for b in same):
|
||||
novelty, reason = "worsened", "discharge_removed"
|
||||
elif not same and op.mechanism == "taint":
|
||||
reason = "source_correspondence_unresolved"
|
||||
elif (
|
||||
location is not None
|
||||
and not old
|
||||
and change
|
||||
and op.mechanism == "taint"
|
||||
and finding.rung == Rung.ENTAILED
|
||||
and not op.discharged
|
||||
):
|
||||
novelty, reason = "new", "source_connection_absent_from_comparable_base"
|
||||
results.append(_candidate(finding, op, old, novelty, reason, change, context))
|
||||
return tuple(results)
|
||||
|
||||
|
||||
def compare_targeted_base(
|
||||
base_root: Path,
|
||||
*,
|
||||
head: ModelScanResult,
|
||||
head_regions: Sequence[ScanRegion],
|
||||
base_regions: Sequence[ScanRegion],
|
||||
context: ChangeContext,
|
||||
backend: Any,
|
||||
execution_budget: ExecutionBudget,
|
||||
scan_budget: ScanBudget,
|
||||
head_semantics: str,
|
||||
base_semantics: str,
|
||||
) -> DeltaComparison:
|
||||
"""Re-run counterparts of the ranker's selected head questions through the existing driver.
|
||||
|
||||
The caller supplies immutable base regions, not a different selector. Unknown function
|
||||
correspondence is left unknown rather than widening to every function in the file.
|
||||
"""
|
||||
selected = {q.identity for q in head.scope.selected} if head.scope else set()
|
||||
renames = {context.decode_path(r.head_path): context.decode_path(r.base_path) for r in context.renames}
|
||||
wanted = {(renames.get(q.path, q.path), q.function, q.family) for q in selected}
|
||||
targets = tuple(
|
||||
replace(r, families=tuple(f for f in r.families if (r.path, r.function, f) in wanted))
|
||||
for r in base_regions
|
||||
if any((r.path, r.function, family) in wanted for family in r.families)
|
||||
)
|
||||
# Preserve source/entry-point contracts and family depth; mismatches are not comparable.
|
||||
source_contract = {(renames.get(r.path, r.path), r.function): r.source for r in head_regions}
|
||||
compatible = all(source_contract.get((r.path, r.function)) == r.source for r in targets)
|
||||
reasons = []
|
||||
base = None
|
||||
if context.base_revision is None:
|
||||
reasons.append("base_unavailable")
|
||||
elif head_semantics != base_semantics or not compatible:
|
||||
reasons.append("semantics_mismatch")
|
||||
elif time.monotonic() >= execution_budget.deadline_monotonic:
|
||||
reasons.append("deadline_exhausted")
|
||||
elif not targets:
|
||||
reasons.append("base_counterpart_unresolved")
|
||||
else:
|
||||
try:
|
||||
base = scan_repository(
|
||||
base_root,
|
||||
targets,
|
||||
backend=backend,
|
||||
budget=scan_budget,
|
||||
execution_budget=execution_budget,
|
||||
ranking_mode=head.scope.ranking_mode if head.scope else "static",
|
||||
unit=next(iter(selected)).unit,
|
||||
population_complete=True,
|
||||
)
|
||||
except Exception as error: # noqa: BLE001 -- comparison failure is coverage, not a new defect
|
||||
reasons.append(f"base_scan_failed:{type(error).__name__}")
|
||||
candidates = compare_evidence(
|
||||
head, base, context=context, head_semantics=head_semantics, base_semantics=base_semantics, execution_budget=execution_budget
|
||||
)
|
||||
reasons.extend(c.reason for c in candidates if c.novelty == "unknown")
|
||||
return DeltaComparison(candidates, base, head_semantics, tuple(dict.fromkeys(reasons)))
|
||||
@@ -0,0 +1,208 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import time
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
|
||||
from openultrasast.cpg.backend import CpgResult
|
||||
from openultrasast.model.contracts import (
|
||||
ChangeContext,
|
||||
ChangedSpan,
|
||||
ExecutionBudget,
|
||||
LineCorrespondence,
|
||||
PathRename,
|
||||
QuestionIdentity,
|
||||
QuestionOutcome,
|
||||
RankedQuestion,
|
||||
ScopeDecision,
|
||||
)
|
||||
from openultrasast.model.ladder import Rung
|
||||
from openultrasast.model.pipeline import ModelFinding
|
||||
from openultrasast.model.regions import ScanRegion
|
||||
from openultrasast.model.scan import ModelScanResult, ScanBudget
|
||||
from openultrasast.push.policy import CandidateDelta, compare_evidence, compare_targeted_base
|
||||
|
||||
|
||||
def scan(*, path="a.js", line=4, source="req.body", sanitized=False, findings=True, status="completed", rows=None):
|
||||
q = QuestionIdentity("repository", "javascript", path, "handler", "injection")
|
||||
row = {
|
||||
"sink": "exec(cmd)",
|
||||
"sinkFile": path,
|
||||
"sinkLine": line,
|
||||
"sinkMethod": "handler",
|
||||
"source": source,
|
||||
"sourceKind": "field",
|
||||
"sanitized": sanitized,
|
||||
"bounded": False,
|
||||
"length": 2,
|
||||
}
|
||||
return ModelScanResult(
|
||||
findings=(
|
||||
ModelFinding(f"{path}:{line}:handler", "injection", Rung.ENTAILED, f"{source} -> exec(cmd) in {path} (line {line}, 2 steps)"),
|
||||
)
|
||||
if findings
|
||||
else (),
|
||||
scope=ScopeDecision("v1", "static", True, (RankedQuestion(q, 1.0, ()),), (), ()),
|
||||
question_outcomes=(
|
||||
QuestionOutcome(q, status, "test_answer", json.dumps([row] if rows is None else rows) if status != "unanswered" else None),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def context(*, rename=False):
|
||||
return ChangeContext(
|
||||
"base",
|
||||
"head",
|
||||
("a.js",),
|
||||
(),
|
||||
(ChangedSpan("old.js" if rename else "a.js", 2, 2, "base"),),
|
||||
(PathRename("old.js", "a.js"),) if rename else (),
|
||||
(),
|
||||
(),
|
||||
(),
|
||||
line_correspondences=(LineCorrespondence("old.js" if rename else "a.js", "a.js", 5, 5, 4),),
|
||||
)
|
||||
|
||||
|
||||
def delta(head, base, ctx=None, **kwargs):
|
||||
return compare_evidence(head, base, context=ctx or context(), head_semantics="same", base_semantics=kwargs.get("semantics", "same"))[0]
|
||||
|
||||
|
||||
def test_removed_discharge_is_worsened_at_unchanged_sink():
|
||||
result = delta(scan(), scan(line=5, sanitized=True, findings=False))
|
||||
assert result.novelty == "worsened"
|
||||
assert result.reason == "discharge_removed"
|
||||
assert result.change_evidence == ("base:a.js:2-2",)
|
||||
assert CandidateDelta.from_payload(result.to_payload()) == result
|
||||
|
||||
|
||||
def test_unchanged_backlog_and_renamed_movement_stay_unchanged():
|
||||
assert delta(scan(), scan(line=5)).novelty == "unchanged"
|
||||
renamed = delta(scan(), scan(path="old.js", line=5), context(rename=True))
|
||||
assert renamed.novelty == "unchanged"
|
||||
baseline = ChangeContext("base", "head", (), (), (), (), (), (), ())
|
||||
assert renamed.defect_id == delta(scan(path="old.js", line=5), scan(path="old.js", line=5), baseline).defect_id
|
||||
|
||||
|
||||
def test_changed_source_text_needs_identity_proof_not_a_rename_guess():
|
||||
assert delta(scan(), scan(line=5, source="req.query")).reason == "source_correspondence_unresolved"
|
||||
assert delta(scan(source="request.body"), scan(line=5)).novelty == "unknown"
|
||||
|
||||
|
||||
def test_failed_truncated_incompatible_base_remains_unknown():
|
||||
assert delta(scan(), scan(status="unanswered")).novelty == "unknown"
|
||||
assert delta(scan(), replace(scan(), degradations=({"reason": "files_unparsed"},))).novelty == "unknown"
|
||||
assert delta(scan(), scan(line=5), semantics="different").reason == "semantics_mismatch"
|
||||
assert delta(scan(), None).novelty == "unknown"
|
||||
|
||||
|
||||
def test_unrelated_lexical_guard_removal_does_not_override_same_flow():
|
||||
assert delta(scan(), scan(line=5)).reason == "same_supported_mechanism"
|
||||
|
||||
|
||||
def test_missing_anchor_and_ambiguous_witness_do_not_invent_novelty():
|
||||
assert delta(scan(), scan(line=5), replace(context(), line_correspondences=())).novelty == "unknown"
|
||||
answer = scan().question_outcomes[0]
|
||||
ambiguous = replace(scan(), question_outcomes=(replace(answer, raw_rows_json=json.dumps(json.loads(answer.raw_rows_json) * 2)),))
|
||||
assert delta(ambiguous, scan()).novelty == "unknown"
|
||||
|
||||
|
||||
def test_targeted_base_uses_real_driver_same_deadline(tmp_path):
|
||||
(tmp_path / "a.js").write_text("function handler(req) { exec(req.body); }\n")
|
||||
seen = []
|
||||
|
||||
class Backend:
|
||||
def build(self, root, *, language, exclude, execution_budget):
|
||||
assert (root / "a.js").read_bytes()
|
||||
seen.append(execution_budget)
|
||||
rows = json.loads(scan(line=5, sanitized=True).question_outcomes[0].raw_rows_json)
|
||||
return CpgResult(Path("graph"), lambda kind, params: rows)
|
||||
|
||||
region = ScanRegion("a.js", "handler", "javascript", ("injection",), 1.0, "entry_point")
|
||||
budget = ExecutionBudget(time.monotonic() + 30, 1.0)
|
||||
result = compare_targeted_base(
|
||||
tmp_path,
|
||||
head=scan(),
|
||||
head_regions=(region,),
|
||||
base_regions=(region,),
|
||||
context=context(),
|
||||
backend=Backend(),
|
||||
execution_budget=budget,
|
||||
scan_budget=ScanBudget(tiering=False),
|
||||
head_semantics="same",
|
||||
base_semantics="same",
|
||||
)
|
||||
assert seen == [budget]
|
||||
assert result.base_scan.question_outcomes[0].status == "completed"
|
||||
assert result.candidates[0].novelty == "worsened"
|
||||
|
||||
|
||||
def test_new_source_at_unchanged_operation_needs_complete_base_answer():
|
||||
assert delta(scan(), scan(rows=[], findings=False)).novelty == "new"
|
||||
assert delta(scan(), scan(rows=[], status="unanswered", findings=False)).novelty == "unknown"
|
||||
|
||||
|
||||
def test_changed_new_operation_in_comparable_existing_file():
|
||||
ctx = replace(context(), spans=(ChangedSpan("a.js", 4, 4, "head"),), line_correspondences=())
|
||||
assert delta(scan(), scan(rows=[], findings=False), ctx).novelty == "unknown"
|
||||
assert delta(scan(), scan(line=6), ctx).novelty == "unknown" # movement is not new
|
||||
|
||||
|
||||
def test_other_completed_question_cannot_prove_absence_for_missing_counterpart():
|
||||
assert delta(scan(), scan(path="other.js", findings=False)).novelty == "unknown"
|
||||
assert delta(scan(), replace(scan(), scope=replace(scan().scope, selected=()))).novelty == "unknown"
|
||||
|
||||
|
||||
def test_head_failure_or_unproven_witness_stays_unknown():
|
||||
assert delta(replace(scan(), degradations=({"reason": "cpg_sharded"},)), scan(line=5)).novelty == "unknown"
|
||||
finding = replace(scan().findings[0], witness="invented witness")
|
||||
assert delta(replace(scan(), findings=(finding,)), scan(line=5)).novelty == "unknown"
|
||||
|
||||
|
||||
def test_string_false_is_not_supported_discharge_evidence():
|
||||
base = scan(line=5, sanitized=True)
|
||||
row = json.loads(base.question_outcomes[0].raw_rows_json)[0]
|
||||
row["sanitized"] = "false"
|
||||
result = delta(scan(), scan(line=5, rows=[row]))
|
||||
# A malformed returned row must not be reinterpreted as valid negative evidence.
|
||||
assert result.novelty == "unknown"
|
||||
|
||||
|
||||
def test_deadline_exhaustion_prevents_any_new_base_build(tmp_path):
|
||||
region = ScanRegion("a.js", "handler", "javascript", ("injection",), 1.0, "entry_point")
|
||||
|
||||
class Backend:
|
||||
def build(self, *args, **kwargs):
|
||||
raise AssertionError("expired budget must prevent build")
|
||||
|
||||
result = compare_targeted_base(
|
||||
tmp_path,
|
||||
head=scan(),
|
||||
head_regions=(region,),
|
||||
base_regions=(region,),
|
||||
context=context(),
|
||||
backend=Backend(),
|
||||
execution_budget=ExecutionBudget(0.0, 1.0),
|
||||
scan_budget=ScanBudget(),
|
||||
head_semantics="same",
|
||||
base_semantics="same",
|
||||
)
|
||||
assert "deadline_exhausted" in result.coverage_reasons
|
||||
assert result.candidates[0].novelty == "unknown"
|
||||
|
||||
|
||||
def test_benign_identifier_rename_is_unknown_not_new():
|
||||
ctx = replace(context(), spans=(ChangedSpan("a.js", 4, 4, "head"),), line_correspondences=())
|
||||
assert delta(scan(source="request.body"), scan(line=4), ctx).novelty == "unknown"
|
||||
|
||||
|
||||
def test_wrong_query_mechanism_cannot_prove_base_absence():
|
||||
row = {"operation": "exec(cmd)", "opFile": "a.js", "opLine": 5, "dominatingGuards": []}
|
||||
assert delta(scan(), scan(line=5, rows=[row])).novelty == "unknown"
|
||||
|
||||
|
||||
def test_non_string_operation_cannot_prove_base_absence():
|
||||
row = json.loads(scan(line=5).question_outcomes[0].raw_rows_json)[0]
|
||||
row["sink"] = 42
|
||||
assert delta(scan(), scan(line=5, rows=[row])).novelty == "unknown"
|
||||
Reference in New Issue
Block a user