mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
pre-push(17.4): a file new in head is new, not comparison unknown
ChangeContext records the paths Git reports as added. An entailed, undischarged operation in an added file, with complete head evidence and no identical operation in any base answer, is classified new (file_added_in_head) instead of base_incomplete; admission accepts that reason as change evidence. Unpaired add/delete pushes and every other unknown comparison keep their vetoes. An engine finding whose novelty is established and whose only vetoes are capability vetoes is shown as an advisory engine finding, not an alert. Requirement 9.4. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
15999a09e1
commit
511de4b75f
@@ -642,10 +642,11 @@ Preserve all completed task evidence below.
|
|||||||
- Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
|
- Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
|
||||||
- _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
|
- _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
|
||||||
- _Requirements: 9.3_
|
- _Requirements: 9.3_
|
||||||
- [ ] 17.4 Classify a file new in head as new, not unknown
|
- [x] 17.4 Classify a file new in head as new, not unknown
|
||||||
- `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
|
- `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
|
||||||
- Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
|
- Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
|
||||||
- Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
|
- Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
|
||||||
|
- _Done 2026-10-02:_ `ChangeContext.added_paths` records Git status-A paths (rename targets excluded). In `_compare_evidence`, after every existing context/head/deadline check, an operation in an added file is `new`/`file_added_in_head` when the claim is entailed, undischarged and has change evidence, and no base answer holds the same operation (else `operation_moved_within_change`); an unpaired add/delete still blocks as `change_context_unresolved`. Admission accepts `(new, file_added_in_head)`; with the empty registry such a candidate's only veto is `capability_unavailable`, and the terminal now shows it under "Engine: N finding(s) this push introduced. Advisory ... not an alert." Any comparison, evidence or location veto keeps a candidate out of that line. Tests: `tests/test_push_new_file.py` (9). Live engine evidence depends on host memory (17.7). Gates: 2007 passed, 42 skipped; ruff, format, mypy clean.
|
||||||
- _Requirements: 9.4, 3.4_
|
- _Requirements: 9.4, 3.4_
|
||||||
- [ ] 17.5 Say when the engine cannot finish, and offer it in the background
|
- [ ] 17.5 Say when the engine cannot finish, and offer it in the background
|
||||||
- One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
|
- One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
|
||||||
|
|||||||
@@ -111,6 +111,9 @@ class ChangeContext(Contract):
|
|||||||
# are not encoded or decoded by this field.
|
# are not encoded or decoded by this field.
|
||||||
path_encoding: Literal["text", "filesystem-bytes-hex"] = "text"
|
path_encoding: Literal["text", "filesystem-bytes-hex"] = "text"
|
||||||
line_correspondences: tuple[LineCorrespondence, ...] = ()
|
line_correspondences: tuple[LineCorrespondence, ...] = ()
|
||||||
|
# Paths Git reports as added (status A) in head, in this context's path encoding. A rename
|
||||||
|
# target is not added. Only a lexical fact: novelty still needs complete head evidence.
|
||||||
|
added_paths: tuple[str, ...] = ()
|
||||||
|
|
||||||
def decode_path(self, path: str) -> str:
|
def decode_path(self, path: str) -> str:
|
||||||
"""Return the filesystem spelling for matching ordinary question paths.
|
"""Return the filesystem spelling for matching ordinary question paths.
|
||||||
@@ -130,6 +133,7 @@ class ChangeContext(Contract):
|
|||||||
*(s.path for s in self.spans),
|
*(s.path for s in self.spans),
|
||||||
*(p for r in self.renames for p in (r.base_path, r.head_path)),
|
*(p for r in self.renames for p in (r.base_path, r.head_path)),
|
||||||
*(p for m in self.line_correspondences for p in (m.base_path, m.head_path)),
|
*(p for m in self.line_correspondences for p in (m.base_path, m.head_path)),
|
||||||
|
*self.added_paths,
|
||||||
]
|
]
|
||||||
for path in paths:
|
for path in paths:
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -658,6 +658,16 @@ def _compare_evidence(
|
|||||||
reason = "head_context_incomplete"
|
reason = "head_context_incomplete"
|
||||||
elif execution_budget is not None and time.monotonic() >= execution_budget.deadline_monotonic:
|
elif execution_budget is not None and time.monotonic() >= execution_budget.deadline_monotonic:
|
||||||
reason = "deadline_exhausted"
|
reason = "deadline_exhausted"
|
||||||
|
elif op.path in {context.decode_path(p) for p in context.added_paths}:
|
||||||
|
# The file did not exist in the base, so no base line or counterpart question can hold
|
||||||
|
# this operation (task 17.4, Requirement 9.4). Code moved here from elsewhere is still
|
||||||
|
# movement: an identical operation in any base answer keeps it out of `new`, and an
|
||||||
|
# unpaired add/delete in the same push already blocked above as context.
|
||||||
|
reason = "operation_moved_within_change"
|
||||||
|
if not any(b.operation == op.operation and b.mechanism == op.mechanism for b in base_ops):
|
||||||
|
reason = "file_added_unsupported_claim"
|
||||||
|
if change and finding.rung == Rung.ENTAILED and not op.discharged:
|
||||||
|
novelty, reason = "new", "file_added_in_head"
|
||||||
elif enumerated:
|
elif enumerated:
|
||||||
same = tuple(b for b in old if b.source == op.source)
|
same = tuple(b for b in old if b.source == op.source)
|
||||||
reason = "operation_correspondence_unresolved"
|
reason = "operation_correspondence_unresolved"
|
||||||
@@ -926,6 +936,7 @@ def admit_candidates(candidates: Sequence[AdmissionCandidate], *, capabilities:
|
|||||||
elif not delta.change_evidence or (delta.novelty, delta.reason) not in (
|
elif not delta.change_evidence or (delta.novelty, delta.reason) not in (
|
||||||
("new", "source_connection_absent_from_comparable_base"),
|
("new", "source_connection_absent_from_comparable_base"),
|
||||||
("new", "operation_absent_from_comparable_base"),
|
("new", "operation_absent_from_comparable_base"),
|
||||||
|
("new", "file_added_in_head"),
|
||||||
("worsened", "discharge_removed"),
|
("worsened", "discharge_removed"),
|
||||||
):
|
):
|
||||||
reasons.append("change_unsupported")
|
reasons.append("change_unsupported")
|
||||||
|
|||||||
@@ -219,6 +219,50 @@ def _experimental_line(experimental: Mapping[str, object]) -> str:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# Vetoes that say only "this capability is not qualified yet". A candidate whose novelty is established
|
||||||
|
# and whose only vetoes are these is shown as an advisory engine finding (task 17.4, Requirement 9.4);
|
||||||
|
# any comparison, evidence or location veto keeps it out of the terminal.
|
||||||
|
_CAPABILITY_VETOES = frozenset(
|
||||||
|
{
|
||||||
|
"capability_unavailable",
|
||||||
|
"capability_disabled",
|
||||||
|
"capability_unevaluated",
|
||||||
|
"capability_ambiguous",
|
||||||
|
"operation_semantics_mismatch",
|
||||||
|
"consequence_unsupported",
|
||||||
|
"repair_unsupported",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _advisory_engine_lines(report: PushReport) -> list[str]:
|
||||||
|
shown: dict[str, list[str]] = {}
|
||||||
|
for disposition in report.admission.dispositions:
|
||||||
|
delta = disposition.candidate.delta
|
||||||
|
op = delta.head_operation
|
||||||
|
if (
|
||||||
|
disposition.admitted
|
||||||
|
or delta.novelty not in ("new", "worsened")
|
||||||
|
or not set(disposition.reasons) <= _CAPABILITY_VETOES
|
||||||
|
or op is None
|
||||||
|
or delta.witness is None
|
||||||
|
or delta.defect_id in shown
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
how = "new" if delta.novelty == "new" else "worsened"
|
||||||
|
shown[delta.defect_id] = [
|
||||||
|
f"- {_short(delta.family)} at {_short(op.path, 4096)}:{op.line} ({how}): {_short(redact_secrets(delta.witness))}"
|
||||||
|
]
|
||||||
|
if not shown:
|
||||||
|
return []
|
||||||
|
lines = [f"Engine: {len(shown)} finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert."]
|
||||||
|
for entry in list(shown.values())[:3]:
|
||||||
|
lines.extend(entry)
|
||||||
|
if len(shown) > 3:
|
||||||
|
lines.append(f"{len(shown) - 3} more advisory engine findings retained in the detailed result.")
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
# Quick-rule matches shown per comparison; the artifact keeps every one.
|
# Quick-rule matches shown per comparison; the artifact keeps every one.
|
||||||
_QUICK_LINES = 5
|
_QUICK_LINES = 5
|
||||||
|
|
||||||
@@ -285,6 +329,7 @@ def render_report(report: PushReport, *, artifact: Path | None, error: str | Non
|
|||||||
)
|
)
|
||||||
if len(defects) > 3:
|
if len(defects) > 3:
|
||||||
lines.append(f"{len(defects) - 3} more actionable defects retained in the detailed result.")
|
lines.append(f"{len(defects) - 3} more actionable defects retained in the detailed result.")
|
||||||
|
lines.extend(_advisory_engine_lines(report))
|
||||||
for payload in report.quick:
|
for payload in report.quick:
|
||||||
lines.extend(_quick_lines(payload))
|
lines.extend(_quick_lines(payload))
|
||||||
if not defects:
|
if not defects:
|
||||||
|
|||||||
@@ -249,6 +249,7 @@ class SnapshotAdapter:
|
|||||||
boundaries = ["declaration_paths_unavailable"] if declaration_paths is None else []
|
boundaries = ["declaration_paths_unavailable"] if declaration_paths is None else []
|
||||||
changed: list[str] = []
|
changed: list[str] = []
|
||||||
deleted: list[str] = []
|
deleted: list[str] = []
|
||||||
|
added: list[str] = []
|
||||||
spans: list[ChangedSpan] = []
|
spans: list[ChangedSpan] = []
|
||||||
renames: list[PathRename] = []
|
renames: list[PathRename] = []
|
||||||
mappings: list[LineCorrespondence] = []
|
mappings: list[LineCorrespondence] = []
|
||||||
@@ -305,6 +306,9 @@ class SnapshotAdapter:
|
|||||||
if status == "D":
|
if status == "D":
|
||||||
assert path_base is not None
|
assert path_base is not None
|
||||||
deleted.append(path_base.hex())
|
deleted.append(path_base.hex())
|
||||||
|
if status == "A":
|
||||||
|
assert path_head is not None
|
||||||
|
added.append(path_head.hex())
|
||||||
if status == "R":
|
if status == "R":
|
||||||
assert path_base is not None and path_head is not None
|
assert path_base is not None and path_head is not None
|
||||||
renames.append(PathRename(path_base.hex(), path_head.hex()))
|
renames.append(PathRename(path_base.hex(), path_head.hex()))
|
||||||
@@ -432,6 +436,7 @@ class SnapshotAdapter:
|
|||||||
tuple(boundaries),
|
tuple(boundaries),
|
||||||
"filesystem-bytes-hex",
|
"filesystem-bytes-hex",
|
||||||
tuple(mappings),
|
tuple(mappings),
|
||||||
|
tuple(added),
|
||||||
)
|
)
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
|
|||||||
@@ -0,0 +1,96 @@
|
|||||||
|
"""A file new in head is `new`, not `comparison unknown` (task 17.4, Requirement 9.4)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import replace
|
||||||
|
|
||||||
|
from test_push_admission import control
|
||||||
|
from test_push_policy import delta, scan
|
||||||
|
from test_push_report import budget, report_with
|
||||||
|
|
||||||
|
from openultrasast.model.contracts import ChangeContext, ChangedSpan
|
||||||
|
from openultrasast.push.policy import admit_candidates
|
||||||
|
from openultrasast.push.report import deliver_report
|
||||||
|
|
||||||
|
|
||||||
|
def added(*, boundaries=(), listed=True):
|
||||||
|
return ChangeContext(
|
||||||
|
"base",
|
||||||
|
"head",
|
||||||
|
("a.js",),
|
||||||
|
(),
|
||||||
|
(ChangedSpan("a.js", 1, 6, "head"),),
|
||||||
|
(),
|
||||||
|
(),
|
||||||
|
(),
|
||||||
|
tuple(boundaries),
|
||||||
|
added_paths=("a.js",) if listed else (),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_snapshot_records_added_paths_and_not_modified_ones(tmp_path):
|
||||||
|
import json
|
||||||
|
|
||||||
|
from test_push_runner import history
|
||||||
|
|
||||||
|
from openultrasast.cpg.backend import NullBackend
|
||||||
|
from openultrasast.push.runner import replay
|
||||||
|
|
||||||
|
root, _, base, git = history(tmp_path)
|
||||||
|
(root / "debug.js").write_text("eval(x)\n")
|
||||||
|
(root / "api.js").write_text("changed\n")
|
||||||
|
git("add", "debug.js", "api.js")
|
||||||
|
git("commit", "-m", "add and modify")
|
||||||
|
head = git("rev-parse", "HEAD")
|
||||||
|
replay(root, base=base, head=head, artifact=tmp_path / "r.json", backend=NullBackend())
|
||||||
|
context = json.loads((tmp_path / "r.json").read_text())["change_context"]
|
||||||
|
assert context["added_paths"] == [b"debug.js".hex()]
|
||||||
|
|
||||||
|
|
||||||
|
def test_operation_in_an_added_file_is_new_without_any_base_answer():
|
||||||
|
result = delta(scan(), None, added())
|
||||||
|
assert (result.novelty, result.reason) == ("new", "file_added_in_head")
|
||||||
|
assert result.change_evidence == ("head:a.js:1-6",)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_same_shape_without_the_added_fact_keeps_its_veto():
|
||||||
|
assert delta(scan(), None, added(listed=False)).novelty == "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
def test_unpaired_add_and_delete_stays_unknown():
|
||||||
|
result = delta(scan(), None, added(boundaries=("unmatched_added_deleted_path_correspondence",)))
|
||||||
|
assert (result.novelty, result.reason) == ("unknown", "change_context_unresolved")
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_operation_already_in_the_base_elsewhere_is_movement_not_new():
|
||||||
|
result = delta(scan(), scan(path="b.js", line=9), added())
|
||||||
|
assert (result.novelty, result.reason) == ("unknown", "operation_moved_within_change")
|
||||||
|
|
||||||
|
|
||||||
|
def test_unentailed_or_discharged_claims_in_added_files_are_not_new():
|
||||||
|
assert delta(scan(sanitized=True), None, added()).novelty == "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
def test_admission_accepts_the_added_file_reason():
|
||||||
|
candidate, capability, _ = control()
|
||||||
|
new_file = replace(candidate, delta=replace(candidate.delta, reason="file_added_in_head"))
|
||||||
|
assert len(admit_candidates((new_file,), capabilities=(capability,)).defects) == 1
|
||||||
|
unqualified = admit_candidates((new_file,))
|
||||||
|
assert unqualified.dispositions[0].reasons == ("capability_unavailable",)
|
||||||
|
|
||||||
|
|
||||||
|
def test_unqualified_new_engine_finding_is_shown_as_advisory_not_alert(tmp_path):
|
||||||
|
report = report_with(1, admitted=False)
|
||||||
|
delivered = deliver_report(report, tmp_path / "r.json", execution_budget=budget())
|
||||||
|
assert delivered.exit_code == 0 and delivered.result.actionable_defect_ids == ()
|
||||||
|
assert "Engine: 1 finding(s) this push introduced. Advisory" in delivered.text
|
||||||
|
assert "- injection at handler.src:4 (new): request.input -> exec(command)" in delivered.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_comparison_veto_keeps_an_engine_finding_out_of_the_terminal(tmp_path):
|
||||||
|
report = report_with(1, admitted=False)
|
||||||
|
disposition = report.admission.dispositions[0]
|
||||||
|
vetoed = replace(disposition, reasons=(*disposition.reasons, "comparison_unknown"))
|
||||||
|
report = replace(report, admission=replace(report.admission, dispositions=(vetoed,)))
|
||||||
|
delivered = deliver_report(report, tmp_path / "r.json", execution_budget=budget())
|
||||||
|
assert "Engine:" not in delivered.text
|
||||||
Reference in New Issue
Block a user