feat(prover): access-control prover, delivery A: path records, Stapler and Jetty entries, Java operation rows, the prover and its scoring arm

Codex delivery under spec access-control-prover. `semantic/obligations/paths.py` builds entry-to-function path
records from the cached caller index; `mapping.py` names Stapler web methods and Jetty handlers as entries;
`check_obligations` gains the opt-in premise `assume_entries_protected` with the label `unguarded_path`;
`hunt/prover.py` proves a change with no model call and states its coverage; `benchmarks/valid/prover_arm.py`
scores it with the valid-benchmark metrics. Two fixes after the first smoke run: a product file git holds empty
is read, not unread (four changes failed on `__init__.py` and empty assets), and absent-input reasons are
counted by name.

Smoke on the 23 fix changes of the training pool: 7 pushes caught (30 percent, Wilson 16 to 51), 28 of 93
locations, precision 7 of 8 flagged, no instrument failure; three changes over the 120 second bound. Twelve of
the 17 Jenkins fixes reach the fixed method but check no operation because the method is the operation;
the spec now carries handler-named operations for delivery B.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-14 16:50:04 +02:00
co-authored by Claude Fable 5.1
parent e514551cb6
commit beb272a5ec
15 changed files with 1352 additions and 17 deletions
+13 -2
View File
@@ -30,8 +30,9 @@ runner that scores it with the valid-benchmark metrics.
- Operation and discharger rows added to `ruleset/obligations/{java,php,python,typescript}.toml`.
### Out of Boundary
- The facts loader and schema, `OrderDominance`, and the checker's existing decision rules (owned by
`authorization-obligations`). One opt-in extension of the checker is allowed and owned here: the
- The facts loader's existing fields, `OrderDominance`, and the checker's existing decision rules (owned by
`authorization-obligations`). Two extensions are allowed and owned here: the handler-named operation fields
(`handlers`, `framework`) in the facts, and the
`assume_entries_protected` keyword and the `unguarded_path` label (see the checker block below); default off,
every existing checker test unchanged.
- The feature table, the DSPy and hunter arms, the ablation (`hunter-learning-loop`).
@@ -185,6 +186,16 @@ access is not `public` and the route is not declared public; evidence carries th
strings and the sentence "no declared policy clause and no sibling norm; the prover assumes a recognised entry is
protected". `findings_to_static` explains the label like the others. `LABELS` gains the value.
### facts extension: handler-named operations (owned here, schema change)
`[[operation]]` rows gain two optional fields: `handlers` (a list of regular expressions over the function name)
and `framework` (a recogniser id the mapper's evidence names: `stapler`, `tornado`, `remix`, `jetty`,
`symfony`, `laravel`). A row with `handlers` and no `calls` is a handler-named operation: `find_operations`
yields it at the function's declaration line for every function whose name matches and that the caller lists as
a recognised entry of that framework (new keyword `handlers_of: Mapping[str, str] | None`, function name to
framework id; default None keeps today's behaviour). The smoke run of delivery A showed why: 15 of the 17
Jenkins fixes are form-validation and list-filling web methods whose body calls no listed operation; the
method is the operation. Jenkins documents the rule (form validation methods must check a permission).
### obligations.enforcement
```python
def witnesses(*, root: Path, texts: Mapping[str, str], entries: Sequence[EntryPointRecord], language: str) -> list[Discharge]
@@ -82,6 +82,11 @@ so that a guarded route is not reported because its handler body shows no check.
overrides `get_current_user` and checks it before the operation, the prover shall treat that as a path guard.
6. Where a declared policy marks a route public, the prover shall suppress `path_guard` findings for it and
keep `ownership_check` findings, as the checker already does.
8. Where a framework's security model makes the web method itself the protected operation (Jenkins form
validation and list-filling methods `doCheck*`, `doFill*Items`, `doTest*`, `doValidate*`, `doVerify*`, which
Jenkins requires to check a permission; Tornado handler methods; a Remix action), the facts shall be able to
name the obligation by handler name pattern and framework, and the prover shall apply such an operation only
to functions the mapper recognised as entries of that framework, at the method's declaration line.
7. The prover shall write every enforcement fact as closed data in the obligation facts files, with the header
naming the clones and references checked, and shall never add a name that does not occur in the checked code
or the framework's documentation.
+32 -5
View File
@@ -1,7 +1,7 @@
# Implementation Plan
- [ ] 1. Paths from the caller index
- [ ] 1.1 Add `PathRecord` and `compose_paths` in `src/openultrasast/semantic/obligations/paths.py` (P)
- [x] 1. Paths from the caller index
- [x] 1.1 Add `PathRecord` and `compose_paths` in `src/openultrasast/semantic/obligations/paths.py` (P)
- Upward walk through a `callers(path, function)` callable, depth and fan-out bounds, stop at the first entry,
chains kept per file, deduplicated; unreached targets returned
- Done when `tests/test_obligation_paths.py` covers depth 1 to 3, unreached, fan-out cut and same-name files,
@@ -9,7 +9,7 @@
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6_
- _Boundary: obligations.paths_
- [ ] 1.2 Checker opt-in `assume_entries_protected` and label `unguarded_path` in `semantic/obligations/check.py`
- [x] 1.2 Checker opt-in `assume_entries_protected` and label `unguarded_path` in `semantic/obligations/check.py`
- Default off; `_finding` receives the entry records' access by handler; evidence as the design states;
`findings_to_static` explains the label; `LABELS` updated
- Done when the existing checker tests pass unchanged and a new test shows: with paths, no policy, no siblings,
@@ -19,7 +19,7 @@
- _Boundary: obligations.check (opt-in only)_
- [ ] 2. Entry recognisers
- [ ] 2.1 Stapler and Jetty recognisers in `mapping.py` (P)
- [x] 2.1 Stapler and Jetty recognisers in `mapping.py` (P)
- Public `do[A-Z]...` methods on exposed types or with Stapler-typed or `@QueryParameter` parameters; Jetty
`AbstractHandler`, `HttpServlet`, JAX-RS resources; evidence strings; `public` only for `UnprotectedRootAction`
- Done when `tests/test_mapping_frameworks.py` names the entries of a fixture Descriptor, rejects a private
@@ -50,6 +50,13 @@
- _Requirements: 2.3, 2.4, 2.7_
- _Boundary: ruleset/obligations_
- [ ] 3.3 Handler-named operations: `handlers` and `framework` fields in the facts loader, `find_operations(..., handlers_of=)`, Stapler form-validation rows in `java.toml`, Tornado handler rows in `python.toml`, Remix action rows in `typescript.toml`
- Done when `tests/test_obligation_facts.py` loads a handler-named row, `tests/test_obligation_operations.py`
(or the existing operations test file) yields the operation at the declaration line only for listed
handlers, and the prover smoke on the 23 fixes reaches and checks the Jenkins form-validation methods
- _Requirements: 2.8_
- _Boundary: obligations.facts, obligations.operations_
- [ ] 4. The prover
- [ ] 4.1 `hunt/prover.py`: `prove_change` over the hunter `Context`, `Coverage`, findings as hunt defects
- Touched functions from head lines, build output left out, targets include direct callees holding operations,
@@ -62,7 +69,7 @@
- _Depends: 1.1, 2.1, 3.1, 3.2_
- [ ] 5. The arm
- [ ] 5.1 `benchmarks/valid/prover_arm.py`: run over a pool slice with checkpoints, summarise with the valid metrics
- [x] 5.1 `benchmarks/valid/prover_arm.py`: run over a pool slice with checkpoints, summarise with the valid metrics
- Record envelope copied from `hunter_arm`; instrument fields; resume; `summary.json` with recall (pushes and
locations, Wilson), false alarms per 100, precision, instrument failures, seconds per change and the list
over 120 s, per-repository and per-framework coverage, `uncovered` repositories; refuse on zero-byte reads
@@ -76,3 +83,23 @@
`benchmarks/measurements/<date>-access-control-prover/` with NOTES, summary, coverage table; compare with the
DSPy and hunter arms of the same pool
- _Requirements: 5.2, 5.3, 7.2_
## Implementation Notes
- Delivery A (2026-10-14) implements paths, the opt-in checker premise, Stapler/Jetty/Servlet/JAX-RS
entries, the Java facts, the deterministic prover and the checkpointed arm. No commit was made.
- Task 3.2 stays open for the PHP, Python and TypeScript facts. Task 4.1's delivery A core is delivered;
its full-spec checkbox stays open for delivery B's enforcement witnesses and guarded Laravel acceptance
case. This delivery intentionally passes `extra_witnesses=()` as required by the maintainer's brief.
- Default checker behavior is preserved. Only opt-in checking includes a caller's statement witness,
using the existing dominance contract at the outgoing call site. The prover checks each complete path
independently so a guarded chain cannot hide an unguarded chain to the same operation.
- Java additions were checked against actual Git HEAD source objects in the sparse training clones;
checked call sites are named in the Java TOML header. No network or model call was made.
- Mapping accepts cached source texts and an opt-out from external manifest scanning. Default callers
retain their previous behavior; the delivery A prover uses source entries and its Context caches.
- Delivery A review was performed in the main context because the brief forbids model calls. The exact
requested static gate passed (format, Ruff and mypy); the exact pytest gate passed with 112 passed,
zero failed, exit 0. UV_NO_SYNC=1 was set; UV_CACHE_DIR was placed under /tmp because the default
cache is outside the writable sandbox. The unchanged unjustified-path checker test remains intact.
+282
View File
@@ -0,0 +1,282 @@
"""Run deterministic access-control proofs on local frozen changes and score the arm."""
from __future__ import annotations
import argparse
import hashlib
import json
import statistics
import tempfile
import time
from collections import Counter
from dataclasses import asdict
from pathlib import Path
from benchmarks.unseen import score
from benchmarks.unseen.extract import SHA, InstrumentFailure
from benchmarks.unseen.replay import canonical, load_slice
from benchmarks.valid import metrics
from benchmarks.valid.catalog import local_git
from openultrasast.hunt.producers import Context, _Absent, _safe_path
from openultrasast.hunt.prover import prove_change
def _write_json(path, value):
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(".json.tmp")
temporary.write_text(json.dumps(value, sort_keys=True, indent=2) + "\n")
temporary.replace(path)
def _record_path(out, identity):
return out / (hashlib.sha256(identity.encode()).hexdigest() + ".json")
def _prove(change, repos, *, max_depth):
started = time.monotonic()
instrument = dict(
hook_exit=0,
hook_bytes_read=0,
changed_bytes={},
file_bytes={},
head_verified=False,
base_verified=False,
suspect_fast=False,
failure=None,
)
record = {
**change,
"done": True,
"status": "prover_covered",
"instrument": instrument,
"findings": [],
"coverage": {},
"degradations": [],
"push": {"admission": {"defects": [], "dispositions": []}, "quick_tier": []},
}
try:
if not all(SHA.fullmatch(change[side]) for side in ("base", "head")):
raise InstrumentFailure("invalid revision")
git = local_git(Path(repos[change["repo"]]).expanduser().resolve())
git.run("cat-file", "-e", change["base"] + "^{commit}")
instrument["base_verified"] = True
with tempfile.TemporaryDirectory(prefix="ousast-prover-") as temporary:
root = Path(temporary) / "head"
added = False
try:
git.run("-c", "core.hooksPath=/dev/null", "worktree", "add", "--detach", str(root), change["head"])
added = True
checked_git = local_git(root)
# Use the local-only Git runner even when expanding an inherited sparse checkout.
if (
checked_git.runner(
["git", "-C", str(root), "config", "--get", "core.sparseCheckout"], capture_output=True, text=True
).stdout.strip()
== "true"
):
checked_git.run("sparse-checkout", "disable")
if checked_git.run("status", "--porcelain", "--untracked-files=no").strip():
raise InstrumentFailure("incomplete checkout")
ctx = Context(root, change)
ctx.verify_head()
instrument["head_verified"] = True
changed = {file["path"] for file in change.get("files", [])}
# Verify every source read, including hop files, before trusting a quiet verdict.
for relative in sorted(changed | set(ctx.product_files)):
path = _safe_path(root, relative)
if not path.is_file():
held = checked_git.runner(["git", "-C", str(root), "cat-file", "-e", f"HEAD:{relative}"], capture_output=True)
if held.returncode == 0:
raise InstrumentFailure("unread tracked file")
continue # a file deleted by this change has no head input
instrument["file_bytes"][relative] = 0
if relative in changed:
instrument["changed_bytes"][relative] = 0
try:
text = ctx.read(relative)
except _Absent as error:
# A file git holds empty is read, not unread: the first smoke run failed four changes on
# legitimately empty files (`__init__.py`, an empty asset). Anything else absent is unread.
held = checked_git.runner(
["git", "-C", str(root), "cat-file", "-s", f"HEAD:{relative}"], capture_output=True, text=True
)
if error.reason == "empty_source" and held.returncode == 0 and int(held.stdout.strip() or 0) == 0:
instrument["empty_files"] = instrument.get("empty_files", 0) + 1
del instrument["file_bytes"][relative]
instrument["changed_bytes"].pop(relative, None)
continue
raise InstrumentFailure("unread tracked file") from error
size = len(text.encode("utf-8"))
instrument["file_bytes"][relative] = size
if relative in changed:
instrument["changed_bytes"][relative] = size
print(f"{change['id']} {relative}: {size} bytes read", flush=True)
if not size:
raise InstrumentFailure("zero-byte read")
result = prove_change(root, change, ctx, max_depth=max_depth)
instrument["hook_bytes_read"] = sum(instrument["file_bytes"].values())
record.update(findings=list(result.findings), coverage=asdict(result.coverage), degradations=list(result.degradations))
record["push"]["admission"]["defects"] = [
{**finding, "defect_id": finding["id"], "source": "prover", "locations": [[finding["path"], finding["line"]]]}
for finding in result.findings
]
finally:
if added:
git.run("worktree", "remove", "--force", str(root))
except Exception as error:
instrument.update(hook_exit=2, failure=type(error).__name__)
record.update(status="instrument_failure", findings=[])
record["push"]["admission"]["defects"] = []
print(f"{change['id']}: instrument_failure ({type(error).__name__})", flush=True)
record["seconds"] = time.monotonic() - started
return record
def run(manifest, repos_path, out, *, slice_id="1", only="all", max_depth=3):
if max_depth < 0 or only not in {"all", "ordinary", "vulnerability"}:
raise ValueError("invalid prover arguments")
manifest, repos_path, out = Path(manifest), Path(repos_path), Path(out)
changes = load_slice(manifest, slice_id)
if len({change["id"] for change in changes}) != len(changes):
raise ValueError("duplicate change ids")
changes = [change for change in changes if only == "all" or change["kind"] == only]
index = out / "inputs.json"
entries = json.loads(index.read_text()) if index.exists() else []
if len({entry["item_id"] for entry in entries}) != len(entries):
raise ValueError("duplicate inputs")
indexed = {entry["item_id"]: entry for entry in entries}
pending = []
for change in changes:
digest = hashlib.sha256(canonical({"change": change, "max_depth": max_depth, "arm": "prover-a"})).hexdigest()
previous_entry = indexed.get(change["id"])
if previous_entry and (previous_entry["change"] != change or previous_entry["input_digest"] != digest):
raise ValueError("existing prover inputs do not match")
entry = dict(item_id=change["id"], change=change, input_digest=digest)
if not previous_entry:
entries.append(entry)
path = _record_path(out, change["id"])
previous = json.loads(path.read_text()) if path.exists() else {}
if previous and previous.get("input_digest") != digest:
raise ValueError("existing prover inputs do not match")
if previous.get("done"):
print(f"{change['id']}: skipped (done)", flush=True)
else:
pending.append(entry)
_write_json(index, entries)
repos = json.loads(repos_path.read_text())
failed = False
for entry in pending:
record = _prove(entry["change"], repos, max_depth=max_depth)
record["input_digest"] = entry["input_digest"]
_write_json(_record_path(out, entry["item_id"]), record)
failed |= record["status"] == "instrument_failure"
return 2 if failed else 0
def summarize(out, *, seed=0):
out = Path(out)
records = score.load_records(out)
if not records or len({row["id"] for row in records}) != len(records):
raise ValueError("missing or duplicate prover records")
for record in records:
instrument = record.get("instrument", {})
if record.get("status") == "instrument_failure":
continue # counted as a failure already; its byte table stops where the failure happened
if any(size == 0 for field in ("changed_bytes", "file_bytes") for size in instrument.get(field, {}).values()):
raise ValueError("zero-byte read of a tracked file; summary refused")
result = metrics.summarize(records, seed=seed)
usable = [row for row in records if score.coverage(row) != "instrument_failure"]
locations = [
bool(score.catches({**row, **target, "files": [{"path": target["path"], "head_lines": [target["function_lines"]]}]}))
for row in usable
if row["kind"] == "vulnerability"
for target in row.get("targets") or [row]
]
repositories = {}
framework_rows = {}
fields = ("entry_points", "reached", "unreached", "operations_checked", "operations_discharged")
for row in records:
cov = row.get("coverage", {})
repository = repositories.setdefault(
row["repo"],
{
**dict.fromkeys(fields, 0),
"files_by_language": Counter(),
"frameworks_recognised": Counter(),
"unchecked_by_reason": Counter(),
"changes": 0,
},
)
repository["changes"] += 1
for field in fields:
repository[field] += cov.get(field, 0)
for field in ("files_by_language", "frameworks_recognised", "unchecked_by_reason"):
repository[field].update(cov.get(field, {}))
for framework, count in cov.get("frameworks_recognised", {}).items():
table = framework_rows.setdefault(
framework, {"files": 0, "changes": 0, **dict.fromkeys(fields, 0), "unchecked_by_reason": Counter()}
)
table["files"] += count
table["changes"] += 1
for field in fields:
table[field] += cov.get(field, 0)
table["unchecked_by_reason"].update(cov.get("unchecked_by_reason", {}))
uncovered = sorted(repo for repo, cov in repositories.items() if not cov["frameworks_recognised"])
for repo, cov in repositories.items():
cov["status"] = "uncovered" if repo in uncovered else "covered"
positives = [row for row in usable if row["repo"] == repo and row["kind"] == "vulnerability"]
cov["recall"] = score.rate([bool(score.catches(row)) for row in metrics.scoring_records(positives)])
seconds = [row["seconds"] for row in records]
result.update(
location_recall=score.rate(locations),
per_repository=repositories,
per_framework=framework_rows,
uncovered_repositories=uncovered,
covered_recall=score.rate(
[
bool(score.catches(row))
for row in metrics.scoring_records(usable)
if row["kind"] == "vulnerability" and row["repo"] not in uncovered
]
),
uncovered_recall=score.rate(
[
bool(score.catches(row))
for row in metrics.scoring_records(usable)
if row["kind"] == "vulnerability" and row["repo"] in uncovered
]
),
instrument_failures=sum(score.coverage(row) == "instrument_failure" for row in records),
seconds_per_change=dict(n=len(seconds), mean=statistics.mean(seconds), median=statistics.median(seconds), maximum=max(seconds)),
changes_over_120_seconds=[{"id": row["id"], "seconds": row["seconds"]} for row in records if row["seconds"] > 120],
)
# This is a training pool; the generic metric headline refers to unseen repositories.
result["headline"] = result["headline"].replace("On unseen app repos:", "On the training pool:")
_write_json(out / "summary.json", result)
return result
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
commands = parser.add_subparsers(dest="command", required=True)
runner = commands.add_parser("run")
runner.add_argument("pool", type=Path)
runner.add_argument("--repos", type=Path, required=True)
runner.add_argument("--out", type=Path, required=True)
runner.add_argument("--slice", default="1")
runner.add_argument("--only", choices=("all", "ordinary", "vulnerability"), default="all")
runner.add_argument("--max-depth", type=int, default=3)
summarizer = commands.add_parser("summarize")
summarizer.add_argument("out", type=Path)
args = parser.parse_args(argv)
if args.command == "run":
status = run(args.pool, args.repos, args.out, slice_id=args.slice, only=args.only, max_depth=args.max_depth)
if status:
return status
result = summarize(args.out)
print(result["headline"])
return 0
if __name__ == "__main__":
raise SystemExit(main())
+267
View File
@@ -0,0 +1,267 @@
"""Deterministic access-control verdicts from recognised entries and bounded paths."""
from __future__ import annotations
from collections import Counter
from collections.abc import Mapping
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any
from .. import mapping, source
from ..preprocess import FileTarget, count_loc, detect_language, detect_tags
from ..semantic.facts import load_facts
from ..semantic.ir import FileIR, FunctionIR, trailing_name
from ..semantic.obligations.check import _on_caller_hop, _relevant, check_obligations
from ..semantic.obligations.dominance import OrderDominance
from ..semantic.obligations.facts import load_obligation_facts
from ..semantic.obligations.operations import Operation, find_operations, valid_witness
from ..semantic.obligations.paths import compose_paths
from ..semantic.obligations.policy import DEFAULT_POLICY_RELPATH, load_declared_policy
from .producers import Context, _safe_path
@dataclass(frozen=True)
class Coverage:
files_by_language: dict[str, int]
frameworks_recognised: dict[str, int]
entry_points: int
reached: int
unreached: int
operations_checked: int
operations_discharged: int
unchecked_by_reason: dict[str, int]
@dataclass(frozen=True)
class ProverResult:
findings: tuple[dict[str, Any], ...]
coverage: Coverage
degradations: tuple[dict[str, Any], ...]
def prove_change(root: Path, change: Mapping[str, Any], ctx: Context | None = None, *, max_depth: int = 3) -> ProverResult:
"""Check touched head functions and their direct operation-bearing callees, with no model."""
if max_depth < 0:
raise ValueError("invalid path depth")
root = Path(root)
ctx = ctx if ctx is not None else Context(root, change)
ctx.verify_head()
facts, flow = load_obligation_facts(), load_facts()
reasons: Counter[str] = Counter()
languages: Counter[str] = Counter()
frameworks: Counter[str] = Counter()
degradations: list[dict[str, Any]] = []
touched: set[tuple[str, str]] = set()
functions: dict[tuple[str, str], FunctionIR] = {}
operations: dict[tuple[str, str], tuple[Operation, ...]] = {}
irs: dict[str, tuple[FileIR, str]] = {}
failed: set[str] = set()
readable: list[str] = []
targets: list[FileTarget] = []
# Read each product file once through the shared cache. Build mapping targets from that text.
for path in ctx.product_files:
try:
text = ctx.read(path)
language = source.canonical_language(detect_language(root / path))
languages[language] += 1
targets.append(FileTarget(path, str(root / path), language, count_loc(text), detect_tags(path, language, text), False, [], []))
readable.append(path)
except Exception as error:
failed.add(path)
reasons[str(getattr(error, "reason", None) or type(error).__name__)] += 1
degradations.append({"path": path, "reason": str(getattr(error, "reason", None) or type(error).__name__)})
if "entry_points" not in ctx.__dict__:
# External manifest enforcement is supplied by delivery B; this delivery uses source entries.
mapped: list[mapping.EntryPointRecord] = []
for target in targets:
try:
mapped.extend(mapping.analyze_entry_points(root, [target], texts=ctx.texts, include_manifests=False))
except Exception as error:
failed.add(target.path)
reasons[str(getattr(error, "reason", None) or type(error).__name__)] += 1
degradations.append({"path": target.path, "reason": str(getattr(error, "reason", None) or type(error).__name__)})
ctx.__dict__["entry_points"] = mapped
entries = [entry for entry in ctx.entry_points if entry.function_name and entry.defines_handler and entry.line is not None]
recognised = {entry.path for entry in entries}
for path in recognised:
evidence = " ".join(item for entry in entries if entry.path == path for item in entry.access_evidence)
framework = (
"stapler"
if "Stapler" in evidence
else "jetty"
if "Jetty" in evidence
else "servlet"
if "Servlet" in evidence
else "jaxrs"
if "JAX-RS" in evidence
else source.canonical_language(detect_language(root / path))
)
frameworks[framework] += 1
def read_ir(path: str) -> FileIR | None:
if path in failed:
return None
if path in irs:
return irs[path][0]
try:
ir, text = ctx.ir(path), ctx.read(path)
if not ir.parse_ok:
reasons["language_unsupported" if ir.reason == "language_unsupported" else "parse_failed"] += 1
failed.add(path)
return None
if not facts.for_language(ir.language).operations:
reasons["no_facts"] += 1
failed.add(path)
return None
irs[path] = ir, text
ctx.spans.setdefault(path, source.ranges_from_ir(ir, text))
found = find_operations(ir, facts.for_language(ir.language), text=text)
for fn in ir.functions:
if fn.name != "<module>":
functions[path, fn.name] = fn
operations[path, fn.name] = tuple(op for op in found if op.function == fn.name)
return ir
except Exception as error:
reasons[str(getattr(error, "reason", None) or type(error).__name__)] += 1
failed.add(path)
degradations.append({"path": path, "reason": str(getattr(error, "reason", None) or type(error).__name__)})
return None
for file in change.get("files", ()):
path = str(file["path"])
_safe_path(root, path)
reason = source.generated_reason(root, path)
if reason:
reasons[reason] += 1
continue
ir = read_ir(path)
if ir is None:
continue
selected = [
fn
for fn in ir.functions
if fn.name != "<module>" and any(fn.start_line <= end and start <= fn.end_line for start, end in file.get("head_lines", ()))
]
touched.update((path, fn.name) for fn in selected)
if file.get("head_lines") and not selected:
reasons["function_unavailable"] += 1
# The existing name-based index supplies call sites; retain file-qualified enclosing functions.
def callers(path: str, function: str) -> list[tuple[str, int]]:
del path
out = []
for caller_path, line in source.callers_of(root, function, files=readable, texts=ctx.texts, index=ctx.call_index):
ir = read_ir(caller_path)
if ir is None:
continue
spans = [fn for fn in ir.functions if fn.name != "<module>" and fn.start_line <= line <= fn.end_line]
if spans:
fn = min(spans, key=lambda item: item.end_line - item.start_line)
out.append((f"{caller_path}::{fn.name}", line))
return out
target_functions = set(touched)
call_names = {trailing_name(call.name) for key in touched for call in functions[key].calls}
if call_names:
for path in readable:
# Ranges are cached and avoid parsing unrelated files just to look for direct callees.
try:
if any(span.name in call_names for span in ctx.ranges(path)):
read_ir(path)
except Exception as error:
reasons[str(getattr(error, "reason", None) or type(error).__name__)] += 1
degradations.append({"path": path, "reason": str(getattr(error, "reason", None) or type(error).__name__)})
target_functions.update(key for key, ops in operations.items() if key[1] in call_names and ops)
paths, unreached = compose_paths(
callers=callers, entries={f"{e.path}::{e.function_name}" for e in entries}, targets=sorted(target_functions), max_depth=max_depth
)
path_nodes = {node for record in paths for node, _ in record.hops}
for node in path_nodes:
read_ir(node.rpartition("::")[0])
for node in unreached:
path = node.rpartition("::")[0]
reasons["unreached" if path in recognised else "framework_unrecognised"] += 1
entry_index = {f"{e.path}::{e.function_name}": e for e in entries}
policy = load_declared_policy(_safe_path(root, DEFAULT_POLICY_RELPATH.as_posix()))
dominance = OrderDominance(ctx.texts)
findings: dict[str, dict[str, Any]] = {}
checked: set[Operation] = set()
undischarged: set[Operation] = set()
for record in paths:
needed_files = {path for path, _ in touched} | {node.rpartition("::")[0] for node, _ in record.hops}
path_irs = {path: irs[path] for path in sorted(needed_files) if path in irs}
if len(path_irs) != len(needed_files):
continue
result = check_obligations(
irs=path_irs,
entries=entries,
facts=facts,
flow_facts=flow,
policy=policy,
paths=[record],
dominance=dominance,
store_shapes=(),
min_siblings=3,
extra_witnesses=(),
assume_entries_protected=True,
)
degradations.extend(result.degradations)
target_path, _, target_name = record.hops[-1][0].rpartition("::")
for op in operations.get((target_path, target_name), ()):
checked.add(op)
# Silence from a public premise is not evidence that a check dominated the operation.
if not any(
w.kind in op.requires
and valid_witness(w)
and dominance.dominates(w, op, record)
and (_relevant(w, op) or _on_caller_hop(w, record, op, dominance))
for w in result.discharges
):
undischarged.add(op)
for finding in result.findings:
op = finding.operation
identity = f"obligation:{op.kind}:{op.path}:{op.line}:{finding.missing}"
op_fact = next(fact for fact in facts.operations if fact.id == op.fact_id)
fn = functions[op.path, op.function]
calls = sorted(
{
trailing_name(call.name)
for call in fn.calls
if call.line == op.line and any(call.name == name or call.name.endswith("." + name) for name in op_fact.calls)
}
)
entry = asdict(entry_index[record.entry])
hops = [list(hop) for hop in record.hops]
findings.setdefault(
identity,
dict(
path=op.path,
line=op.line,
family="access_control",
function=op.function,
evidence=[
*finding.evidence,
f"hops: {' -> '.join(node for node, _ in record.hops)}",
f"operation {op.fact_id} ({op.kind}) at {op.path}:{op.line}; missing {finding.missing}",
],
id=identity,
entry=entry,
hops=hops,
operation=dict(id=op.fact_id, kind=op.kind, calls=calls, line=op.line),
missing=finding.missing,
label=finding.label,
),
)
coverage = Coverage(
dict(languages),
dict(frameworks),
len(entries),
len({record.hops[-1][0] for record in paths} & {f"{path}::{name}" for path, name in touched}),
len(set(unreached) & {f"{path}::{name}" for path, name in touched}),
len(checked),
len(checked - undischarged),
dict(reasons),
)
return ProverResult(tuple(findings.values()), coverage, tuple(degradations))
+108 -3
View File
@@ -13,6 +13,7 @@ from .preprocess import FileTarget
MappingKind = Literal["semgrep", "codeql", "differential", "sharp_edge"]
AccessLevel = Literal[
"public",
"unknown",
"authenticated",
"role-restricted",
"contract-only/callback",
@@ -95,14 +96,17 @@ def attach_static_hints(targets: list[FileTarget], hints: list[StaticHint]) -> l
return [replace(target, static_hints=hints_by_path.get(target.path, target.static_hints)) for target in targets]
def analyze_entry_points(root: Path, targets: list[FileTarget]) -> list[EntryPointRecord]:
def analyze_entry_points(
root: Path, targets: list[FileTarget], *, texts: Mapping[str, str] | None = None, include_manifests: bool = True
) -> list[EntryPointRecord]:
records: list[EntryPointRecord] = []
for target in targets:
path = root / target.path
text = path.read_text(errors="ignore")
text = texts[target.path] if texts is not None else path.read_text(errors="ignore")
records.extend(_language_entry_points(target, text))
records.extend(_tag_entry_points(target))
records.extend(_route_manifest_entry_points(root, targets))
if include_manifests:
records.extend(_route_manifest_entry_points(root, targets))
return sorted(records, key=lambda item: (item.path, item.line is None, item.line or 0, item.name))
@@ -171,6 +175,8 @@ def write_entry_points(entry_points: list[EntryPointRecord], path: Path) -> None
def _language_entry_points(target: FileTarget, text: str) -> list[EntryPointRecord]:
if target.language == "java":
return _stapler_entries(target, text) + _jetty_entries(target, text)
if target.language == "python":
return _python_entry_points(target, text)
if target.language in {"javascript", "typescript"}:
@@ -184,6 +190,105 @@ def _language_entry_points(target: FileTarget, text: str) -> list[EntryPointReco
return []
def _java_methods(target: FileTarget, text: str) -> list[tuple[object, str, str]]:
"""Use the CST to keep class declarations, parameters and method bodies separate."""
from .semantic.extra import grammar_for
grammar = grammar_for("java")
if grammar is None:
return []
from tree_sitter import Parser
data = text.encode("utf-8")
tree = Parser(grammar).parse(data)
if tree.root_node.has_error:
return []
methods: list[tuple[object, str, str]] = []
def walk(node: Any, owner: str = "") -> None:
if node.type in {"class_declaration", "interface_declaration"}:
body = node.child_by_field_name("body")
owner = data[node.start_byte : body.start_byte if body else node.end_byte].decode("utf-8")
if node.type == "method_declaration":
body = node.child_by_field_name("body")
if body is not None:
header = data[node.start_byte : body.start_byte].decode("utf-8")
methods.append((node, owner, header))
return
for child in node.named_children:
walk(child, owner)
walk(tree.root_node)
return methods
def _stapler_entries(target: FileTarget, text: str) -> list[EntryPointRecord]:
exposed = re.compile(
r"\b(?:extends|implements)\b[^\{]*\b(?:Descriptor|Action|RootAction|Plugin|UnprotectedRootAction|AbstractDescribableImpl)\b"
)
parameters = re.compile(r"\b(?:StaplerRequest2?|StaplerResponse2?)\b|@(?:QueryParameter|AncestorInPath)\b")
records = []
for raw_node, owner, header in _java_methods(target, text):
node = cast(Any, raw_node)
name_node = node.child_by_field_name("name")
name = name_node.text.decode("utf-8")
params = node.child_by_field_name("parameters").text.decode("utf-8")
class_match = exposed.search(owner)
param_match = parameters.search(params)
if not re.search(r"\bpublic\b", header) or not re.match(r"^do[A-Z]", name) or not (class_match or param_match):
continue
public = bool(re.search(r"\bimplements\b[^\{]*\bUnprotectedRootAction\b", owner))
evidence = ["Stapler public do method"]
if class_match:
evidence.append(class_match.group())
if param_match:
evidence.append("Stapler parameter: " + param_match.group())
records.append(
_entry(
target,
node.start_point[0] + 1,
node.end_point[0] + 1,
name,
name,
"route",
"public" if public else "unknown",
"http_request",
evidence,
[],
access_declared=public,
)
)
return records
def _jetty_entries(target: FileTarget, text: str) -> list[EntryPointRecord]:
records = []
for raw_node, owner, header in _java_methods(target, text):
node = cast(Any, raw_node)
name = node.child_by_field_name("name").text.decode("utf-8")
evidence = []
if re.search(r"\bextends\s+(?:[\w.]+\.)?AbstractHandler\b", owner) and name == "handle":
evidence.append("Jetty AbstractHandler.handle")
if re.search(r"\bextends\s+(?:[\w.]+\.)?(?:HttpServlet|GenericServlet|Servlet)\b", owner) and name in {
"service",
"doGet",
"doPost",
"doPut",
"doDelete",
"doPatch",
"doHead",
"doOptions",
}:
evidence.append("Servlet request method: " + name)
if re.search(r"@(?:[\w.]+\.)?(?:GET|POST|PUT|DELETE|PATCH|HEAD|OPTIONS|Path)\b", header):
evidence.append("JAX-RS request annotation")
if evidence and not re.search(r"\bprivate\b", header):
records.append(
_entry(target, node.start_point[0] + 1, node.end_point[0] + 1, name, name, "route", "unknown", "http_request", evidence, [])
)
return records
# WordPress registers handlers rather than decorating them, and the hook name states the access level:
# `wp_ajax_nopriv_*` is admin-ajax for logged-OUT users and is therefore declared public, while `wp_ajax_*`
# fires only for logged-in ones. That is a DECLARED access level, not a guess from an absent decorator, which
@@ -1,3 +1,13 @@
# Delivery A additionally checked actual calls in Git HEAD objects (2026-10-14):
# p4-plugin: CredentialsHelper.lookupCredentialsInItem, P4PasswordImpl and P4TicketImpl
# call ConnectionFactory.testConnection, ConnectionFactory calls connect, ReviewNotifier
# calls openConnection; zephyr-enterprise-test-management-plugin: ZephyrInstance calls
# getCredentialsFromId and UserService.verifyCredentials; azure-keyvault-plugin:
# AzureKeyVaultGlobalConfiguration.doTestConnection calls listPropertiesOfSecrets;
# rocketchatnotifier-plugin: RocketChatNotifier.doTestConnection calls publish.
# Each listed clone is under /home/mc/ousast-results/valid/train/hist/jenkinsci__*.
# No network documentation lookup was performed for this delivery. The existing
# documented checkAnyPermission row below is retained; it was not found in these HEADs.
# Closed authorization facts for Jenkins/Stapler, Spring Security and Servlet code.
# Checked Git HEAD source objects in /home/mc/ousast-results/valid/train/hist/:
# jenkinsci__analysis-core-plugin: DefaultGraphConfigurationView.checkPermission,
@@ -19,7 +29,7 @@ language = "java"
[[operation]]
id = "jenkins-credentials-read"
kind = "protected_read"
calls = ["lookupCredentials", "findCredentialById"]
calls = ["lookupCredentials", "lookupCredentialsInItem", "findCredentialById", "getCredentialsFromId"]
requires = ["path_guard", "ownership_check", "identity_constraint"]
sensitivity = "high"
@@ -46,6 +56,13 @@ calls = ["executeQuery", "execute", "prepareCall"]
requires = ["path_guard", "ownership_check", "identity_constraint"]
sensitivity = "high"
[[operation]]
id = "jenkins-outbound-connection"
kind = "privileged_action"
calls = ["testConnection", "verifyCredentials", "listPropertiesOfSecrets", "openConnection", "connect", "publish"]
requires = ["path_guard", "ownership_check"]
sensitivity = "high"
[[discharger]]
id = "jenkins-permission-guard"
kind = "path_guard"
@@ -4,7 +4,8 @@ An obligated operation is reported when it is reached (a path record ends at it,
an entry point) and none of the dischargers it accepts dominates it. The label says why the obligation exists: a declared
policy clause (``declared_policy_violation``), the practice of sibling handlers (``consistency_violation``), or, in the
degraded mode without path records, the operation fact alone (``function_local``). With path records and neither a clause
nor a sibling norm, nothing justifies the obligation and nothing is reported (design gating 3). Findings are ``suspicion``;
nor a sibling norm, nothing is reported unless the caller opts into the recognised-entry premise (``unguarded_path``).
Findings are ``suspicion``;
the obligation, the missing discharger, the evidence and the known fix travel as data.
"""
@@ -12,7 +13,7 @@ from __future__ import annotations
import re
from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from dataclasses import dataclass, field, replace
from ...findings import StaticFinding
from ..facts import SemanticFacts
@@ -23,7 +24,7 @@ from .operations import Discharge, Operation, find_discharges, find_operations,
from .policy import DeclaredPolicy
from .siblings import Anomaly, SiblingSet, consistency_anomalies, sibling_sets
LABELS = ("declared_policy_violation", "consistency_violation", "function_local")
LABELS = ("declared_policy_violation", "consistency_violation", "function_local", "unguarded_path")
_ROUTE_LITERAL = re.compile(r"@\w[\w.]*\.(?:route|get|post|put|delete|patch|api_route|websocket)\(\s*['\"]([^'\"]+)['\"]")
@@ -64,6 +65,7 @@ def check_obligations(
store_shapes: Sequence[object],
min_siblings: int,
extra_witnesses: Sequence[Discharge] = (),
assume_entries_protected: bool = False,
) -> ObligationResult:
operations: list[Operation] = []
witnesses: list[Discharge] = list(extra_witnesses)
@@ -85,6 +87,7 @@ def check_obligations(
anomalies, under = consistency_anomalies(sets, min_siblings=min_siblings)
anomaly_index: dict[tuple[str, str], Anomaly] = {(a.handler, a.missing): a for a in anomalies}
entry_handlers = {f"{getattr(e, 'path', '')}::{getattr(e, 'function_name', '')}" for e in entries}
entry_index = {f"{getattr(e, 'path', '')}::{getattr(e, 'function_name', '')}": e for e in entries}
reached = _reached(operations, paths, entry_handlers)
if not paths and operations:
for language in sorted({irs[op.path][0].language for op in operations if op.path in irs}):
@@ -103,9 +106,18 @@ def check_obligations(
degradations.append({"stage": "obligations", "reason": "obligations_store_row_invalid", "records": list(invalid_rows)})
for operation, path_record in reached:
handler = f"{operation.path}::{operation.function}"
hops = getattr(path_record, "hops", ())
first = hops[0][0] if hops and isinstance(hops[0], (tuple, list)) else str(hops[0]) if hops else ""
entry = entry_index.get(str(first))
route = _route_path(operation, texts.get(operation.path, ""))
if assume_entries_protected and entry is not None and route is None:
route = str(getattr(entry, "name", "")) or None
declared_access = policy.route_access(route) if policy is not None and route else None
same = [w for w in witnesses if _relevant(w, operation)]
same = [
w
for w in witnesses
if _relevant(w, operation) or (assume_entries_protected and _on_caller_hop(w, path_record, operation, dominance))
]
dominated_kinds = {w.kind for w in same if valid_witness(w) and dominance.dominates(w, operation, path_record)}
# (a) the operation's own obligation: discharged when any accepted discharger dominates. A declared public route
# waives the path guard here too (Req 4.4), so the operation falls through to the next kind it accepts.
@@ -116,7 +128,17 @@ def check_obligations(
required[0],
)
finding = _finding(
operation, missing, same, anomaly_index.get((handler, missing)), policy, route, declared_access, store_shapes, paths, facts
operation,
missing,
same,
anomaly_index.get((handler, missing)),
policy,
route,
declared_access,
store_shapes,
paths,
facts,
entry if assume_entries_protected else None,
)
if finding is not None:
findings.append(finding)
@@ -127,7 +149,17 @@ def check_obligations(
if kind == "path_guard" and declared_access == "public":
continue # declared public: the guard obligation is waived, the identity one is not
finding = _finding(
operation, kind, same, anomaly_index[(handler, kind)], policy, route, declared_access, store_shapes, paths, facts
operation,
kind,
same,
anomaly_index[(handler, kind)],
policy,
route,
declared_access,
store_shapes,
paths,
facts,
entry if assume_entries_protected else None,
)
if finding is not None:
findings.append(finding)
@@ -152,6 +184,8 @@ def findings_to_static(result: ObligationResult) -> list[StaticFinding]:
if finding.known_fix:
tags.append(f"mechanism:{finding.known_fix}")
why = "; ".join(finding.evidence) if finding.evidence else "the operation fact alone (function-local mode)"
if finding.label == "unguarded_path":
why = "unguarded path from a recognised protected entry; " + why
resource = op.resource or "a resource"
provenance = f" (a {finding.missing} is present but bound from {finding.provenance})" if finding.provenance else ""
statics.append(
@@ -190,6 +224,7 @@ def _finding(
store_shapes: Sequence[object],
paths: Sequence[object],
facts: ObligationFacts,
assumed_entry: object | None = None,
) -> ObligationFinding | None:
wrong = next((w for w in same if w.kind == missing and not valid_witness(w)), None)
declared = policy.resources.get(operation.resource or "") if policy is not None else None
@@ -211,6 +246,16 @@ def _finding(
elif anomaly is not None:
label = "consistency_violation"
evidence = anomaly.siblings + anomaly.evidence[len(anomaly.siblings) :]
elif (
paths and assumed_entry is not None and getattr(assumed_entry, "access_level", "public") != "public" and declared_access != "public"
):
label = "unguarded_path"
evidence = (
f"entry {getattr(assumed_entry, 'kind', '')}: {getattr(assumed_entry, 'path', '')}::"
f"{getattr(assumed_entry, 'function_name', '')}",
*(str(item) for item in getattr(assumed_entry, "access_evidence", ())),
"no declared policy clause and no sibling norm; the prover assumes a recognised entry is protected",
)
elif paths:
return None # path-aware mode: without a clause or a sibling norm, nothing justifies the obligation (gating 3)
else:
@@ -245,6 +290,20 @@ def _reached(operations: Sequence[Operation], paths: Sequence[object], entry_han
return reached
def _on_caller_hop(witness: Discharge, path_record: object | None, operation: Operation, dominance: Dominance) -> bool:
"""Use the same dominance contract at the caller's outgoing call in opt-in mode."""
for node, call_line in getattr(path_record, "hops", ())[:-1]:
if node != f"{witness.path}::{witness.function}":
continue
if witness.scope in {"router", "decorator"}:
return True
if witness.kind not in {"path_guard", "ownership_check"} or call_line is None:
return False
caller_operation = replace(operation, path=witness.path, function=witness.function, line=call_line)
return dominance.dominates(witness, caller_operation, None)
return False
def _relevant(witness: Discharge, operation: Operation) -> bool:
return witness.scope == "hop" or (witness.path, witness.function) == (operation.path, operation.function)
@@ -0,0 +1,53 @@
"""Compose bounded caller chains whose first hop is a recognised entry."""
from __future__ import annotations
from collections.abc import Callable, Sequence
from dataclasses import dataclass
@dataclass(frozen=True)
class PathRecord:
hops: tuple[tuple[str, int | None], ...]
entry: str
complete: bool
def compose_paths(
*,
callers: Callable[[str, str], Sequence[tuple[str, int]]],
entries: set[str],
targets: Sequence[tuple[str, str]],
max_depth: int = 3,
fan_out: int = 25,
) -> tuple[list[PathRecord], list[str]]:
"""Walk at most max_depth caller edges. The adapter resolves sites to path::function nodes.
Each caller hop holds the line of its outgoing call; the final target has no outgoing call.
Stop at the first entry, and never revisit a node within a chain.
"""
if max_depth < 0 or fan_out < 1:
raise ValueError("invalid path bounds")
found: dict[tuple[tuple[str, int | None], ...], PathRecord] = {}
unreached: list[str] = []
for path, function in dict.fromkeys(targets):
target = f"{path}::{function}"
reached = False
pending: list[tuple[tuple[str, int | None], ...]] = [((target, None),)]
while pending:
hops = pending.pop()
node = hops[0][0]
if node in entries:
found.setdefault(hops, PathRecord(hops, node, True))
reached = True
continue
if len(hops) - 1 >= max_depth:
continue
node_path, _, node_function = node.rpartition("::")
seen = {hop[0] for hop in hops}
for caller, line in list(dict.fromkeys(callers(node_path, node_function)))[:fan_out]:
if caller not in seen:
pending.append(((caller, line), *hops))
if not reached:
unreached.append(target)
return list(found.values()), unreached
+183
View File
@@ -0,0 +1,183 @@
"""Prover integration on instrumented local Git fixtures."""
import subprocess
import pytest
from openultrasast.hunt.producers import Context
from openultrasast.hunt.prover import prove_change
def repository(tmp_path, text, extra=None, lines=None):
def git(*args):
return subprocess.run(
["git", "-C", str(tmp_path), "-c", "core.hooksPath=/dev/null", *args], check=True, capture_output=True, text=True
).stdout.strip()
git("init", "-q")
git("config", "user.name", "Fixture")
git("config", "user.email", "[email protected]")
(tmp_path / "Fixture.java").write_text("class Plain {}\n")
git("add", "Fixture.java")
git("commit", "-qm", "base")
base = git("rev-parse", "HEAD")
(tmp_path / "Fixture.java").write_text(text)
for path, value in (extra or {}).items():
(tmp_path / path).parent.mkdir(parents=True, exist_ok=True)
(tmp_path / path).write_text(value)
git("add", path)
git("add", "Fixture.java")
git("commit", "-qm", "head")
change = dict(
id="fixture",
repo="fixture/app",
kind="ordinary",
base=base,
head=git("rev-parse", "HEAD"),
files=[{"path": "Fixture.java", "head_lines": lines or [[1, len(text.splitlines())]]}],
)
return tmp_path, change
@pytest.mark.parametrize("guard,findings", [("", 1), ("Jenkins.get().checkPermission(Jenkins.ADMINISTER);", 0)])
def test_stapler_verdict(tmp_path, guard, findings):
root, change = repository(
tmp_path,
f"""class X extends Descriptor<Thing> {{
public Object doTestConnection() {{
{guard}
return CredentialsProvider.lookupCredentials(Thing.class);
}}
}}""",
)
ctx = Context(root, change)
result = prove_change(root, change, ctx)
assert len(result.findings) == findings
assert result.coverage.operations_checked == 1
assert result.coverage.operations_discharged == 1 - findings
assert result.coverage.reached == 1
assert len(ctx.read("Fixture.java").encode()) > 0
if findings:
(finding,) = result.findings
assert finding["family"] == "access_control" and finding["function"] == "doTestConnection"
assert finding["id"] == "obligation:protected_read:Fixture.java:4:path_guard"
assert finding["entry"]["kind"] == "route" and finding["hops"] == [["Fixture.java::doTestConnection", None]]
assert finding["operation"]["calls"] == ["lookupCredentials"] and finding["missing"] == "path_guard"
assert finding["evidence"]
def test_unrecognised_is_coverage_only(tmp_path):
root, change = repository(tmp_path, "class Plain { public Object run() { return lookupCredentials(); } }")
result = prove_change(root, change)
assert not result.findings and result.coverage.frameworks_recognised == {}
assert result.coverage.unchecked_by_reason["framework_unrecognised"] == 1
@pytest.mark.parametrize("guard_position,expected", [("before", 0), ("after", 1), ("none", 1)])
def test_callee_and_caller_guard_order(tmp_path, guard_position, expected):
guard = "Jenkins.get().checkPermission(Jenkins.ADMINISTER);"
root, change = repository(
tmp_path,
f"""class X extends Descriptor<Thing> {{
public Object doTestConnection() {{
{guard if guard_position == "before" else ""}
Object result = Helper.readSecret();
{guard if guard_position == "after" else ""}
return result;
}}
}}""",
{"Helper.java": "class Helper { public static Object readSecret() { return lookupCredentials(); } }"},
[[2, 7]],
)
result = prove_change(root, change)
assert len(result.findings) == expected and result.coverage.operations_checked == 1
if expected:
assert result.findings[0]["path"] == "Helper.java"
assert len(result.findings[0]["hops"]) == 2
def test_touched_helper_walks_to_entry(tmp_path):
root, change = repository(
tmp_path,
"""class X extends Descriptor<Thing> {
public Object doTestConnection() { return readSecret(); }
public Object readSecret() { return lookupCredentials(); }
}""",
lines=[[3, 3]],
)
result = prove_change(root, change)
assert len(result.findings) == 1 and len(result.findings[0]["hops"]) == 2
def test_all_paths_are_checked(tmp_path):
root, change = repository(
tmp_path,
"""class X extends Descriptor<Thing> {
public Object doFirst() {
Jenkins.get().checkPermission(Jenkins.ADMINISTER);
return readSecret();
}
public Object doSecond() { return readSecret(); }
public Object readSecret() { return lookupCredentials(); }
}""",
lines=[[7, 7]],
)
result = prove_change(root, change)
assert len(result.findings) == 1 and result.coverage.operations_discharged == 0
def test_generated_output_and_parse_failure_counted(tmp_path):
root, change = repository(
tmp_path,
"class X { public Object broken( { return lookupCredentials(); }",
{"dist/bundle.java": "class X { public void doTest() {} }"},
)
change["files"].append({"path": "dist/bundle.java", "head_lines": [[1, 1]]})
result = prove_change(root, change)
assert not result.findings and result.coverage.unchecked_by_reason["parse_failed"] >= 1
assert result.coverage.unchecked_by_reason["excluded_dir"] == 1
@pytest.mark.parametrize("exits,expected", [(True, 0), (False, 1)])
def test_caller_permission_predicate_requires_exit(tmp_path, exits, expected):
root, change = repository(
tmp_path,
f"""class X extends Descriptor<Thing> {{
public Object doTestConnection() {{
boolean allowed = Jenkins.get().hasPermission(Jenkins.ADMINISTER);
{"if (!allowed) {" if exits else ""}
{"return null;" if exits else ""}
{"}" if exits else ""}
return Helper.readSecret();
}}
}}""",
{"Helper.java": "class Helper { public static Object readSecret() { return lookupCredentials(); } }"},
)
result = prove_change(root, change)
assert len(result.findings) == expected and result.coverage.reached == 1
assert result.coverage.operations_discharged == 1 - expected
def test_declared_public_route_suppresses_premise(tmp_path):
root, change = repository(
tmp_path,
"""class X extends Descriptor<Thing> {
public Object doTestConnection() { return lookupCredentials(); }
}""",
{".openultrasast/obligations.toml": 'version = 1\n[[route]]\npath = "doTestConnection"\naccess = "public"\n'},
)
result = prove_change(root, change)
assert not result.findings and result.coverage.operations_checked == 1
assert result.coverage.operations_discharged == 0
def test_nonpublic_unreached_function_in_partly_covered_repo(tmp_path):
root, change = repository(
tmp_path,
"class Plain { public Object run() { return lookupCredentials(); } }",
{"Exposed.java": "class X extends Descriptor<Thing> { public void doTest() {} }"},
)
result = prove_change(root, change)
assert result.coverage.frameworks_recognised == {"stapler": 1}
assert result.coverage.unchecked_by_reason["framework_unrecognised"] == 1
assert not result.findings
+69
View File
@@ -0,0 +1,69 @@
"""Java framework entries are tied to their declaring class and public method."""
import pytest
from openultrasast.mapping import analyze_entry_points
from openultrasast.preprocess import build_file_target
def entries(tmp_path, text):
path = tmp_path / "Fixture.java"
path.write_text(text)
return [e for e in analyze_entry_points(tmp_path, [build_file_target(tmp_path, path)]) if e.function_name]
def test_stapler_class_scope_visibility_and_ranges(tmp_path):
rows = entries(
tmp_path,
"""class Exposed extends Descriptor<Thing> {
public String doTestConnection() {
return lookupCredentials();
}
private void doHidden() {}
public void ordinary() {}
}
class Plain {
public void doUnexposed() {}
public void doCheck(@QueryParameter String value) {}
}
""",
)
assert [(e.function_name, e.line, e.end_line) for e in rows] == [("doTestConnection", 2, 4), ("doCheck", 10, 10)]
assert all(e.access_level == "unknown" and e.trust_boundary == "http_request" and e.access_evidence for e in rows)
@pytest.mark.parametrize("base", ["Action", "RootAction", "Plugin", "AbstractDescribableImpl<Thing>", "Descriptor<Thing>"])
def test_exposed_types(tmp_path, base):
assert len(entries(tmp_path, f"class X extends {base} {{ public void doCheckX() {{}} }}")) == 1
def test_unprotected_root_and_stapler_parameters(tmp_path):
rows = entries(
tmp_path,
"""class Open implements UnprotectedRootAction {
public void doTest() {}
}
class Plain {
public void doCheck(StaplerRequest2 request, StaplerResponse response) {}
public void doLowercase() {}
}""",
)
assert [e.access_level for e in rows] == ["public", "unknown"]
assert rows[0].access_declared
@pytest.mark.parametrize(
"text,name",
[
("class X extends AbstractHandler { public void handle(String target) {} }", "handle"),
("class X extends HttpServlet { protected void doGet(Request r) {} }", "doGet"),
('@Path("/api") class X { @GET public String read() { return "ok"; } }', "read"),
],
)
def test_jetty_servlet_and_jaxrs(tmp_path, text, name):
(row,) = entries(tmp_path, text)
assert row.function_name == name and row.kind == "route" and row.access_evidence
def test_unrecognised_and_comments(tmp_path):
assert entries(tmp_path, "/* class X extends Descriptor { public void doFake() {} } */\nclass Plain { public void doCheck() {} }") == []
+31
View File
@@ -206,3 +206,34 @@ def test_a_malformed_store_row_is_a_degradation_not_a_crash() -> None:
(degradation,) = [d for d in result.degradations if d["reason"] == "obligations_store_row_invalid"]
assert degradation["records"] == ["corpus:bad", "corpus:worse"]
assert all(f.known_fix is None for f in result.findings)
def test_recognised_entry_premise_is_opt_in_and_never_applies_to_public():
from dataclasses import replace
from openultrasast.semantic.obligations import load_obligation_facts
from openultrasast.semantic.obligations.check import check_obligations, findings_to_static
from openultrasast.semantic.obligations.dominance import OrderDominance
text = "def exposed(title):\n return Book.query.filter_by(book_title=title).first()\n"
entry = replace(_entry("exposed", 1, 2, "authenticated", ["framework entry"]), access_level="unknown")
kwargs = dict(
irs={"app.py": (parse_file("app.py", text, "python"), text)},
entries=[entry],
facts=load_obligation_facts(),
flow_facts=load_facts(),
policy=None,
paths=[_Path((("app.py::exposed", 1),))],
dominance=OrderDominance({"app.py": text}),
store_shapes=(),
min_siblings=3,
)
assert check_obligations(**kwargs).findings == ()
result = check_obligations(**kwargs, assume_entries_protected=True)
(finding,) = result.findings
assert finding.label == "unguarded_path"
assert "no declared policy clause and no sibling norm" in "; ".join(finding.evidence)
assert "framework entry" in "; ".join(finding.evidence)
assert "unguarded_path" in findings_to_static(result)[0].rationale
kwargs["entries"] = [replace(entry, access_level="public")]
assert check_obligations(**kwargs, assume_entries_protected=True).findings == ()
+24
View File
@@ -91,3 +91,27 @@ def test_java_php_facts_resolve_operations_and_dischargers(language, text, opera
assert ir.parse_ok
assert operation in {op.fact_id for op in find_operations(ir, facts, text=text)}
assert discharger in {d.fact_id for d in find_discharges(ir, facts, load_facts().for_language(language), text=text)}
def test_delivery_a_java_operations_and_dischargers_resolve():
from openultrasast.semantic.obligations.facts import load_obligation_facts
facts = load_obligation_facts().for_language("java")
operations = {call: fact for fact in facts.operations for call in fact.calls}
dischargers = {call: fact for fact in facts.dischargers for call in fact.calls}
for call in (
"lookupCredentials",
"lookupCredentialsInItem",
"getCredentialsFromId",
"testConnection",
"verifyCredentials",
"listPropertiesOfSecrets",
"openConnection",
"connect",
"publish",
"getItemByFullName",
"updateByXml",
):
assert "path_guard" in operations[call].requires
assert dischargers["checkPermission"].kind == dischargers["checkAnyPermission"].kind == "path_guard"
assert dischargers["hasPermission"].kind == "ownership_check"
+65
View File
@@ -0,0 +1,65 @@
"""Bounded paths preserve file identity and call-site lines."""
import pytest
from openultrasast.semantic.obligations.paths import PathRecord, compose_paths
@pytest.mark.parametrize("depth,reached", [(0, False), (1, False), (2, True), (3, True)])
def test_depth_and_call_sites(depth, reached):
graph = {"c.java::target": [("b.java::helper", 8)], "b.java::helper": [("a.java::doTest", 4)]}
paths, unreached = compose_paths(
callers=lambda p, f: graph.get(f"{p}::{f}", []), entries={"a.java::doTest"}, targets=[("c.java", "target")], max_depth=depth
)
assert bool(paths) == reached
assert unreached == ([] if reached else ["c.java::target"])
if reached:
assert paths == [PathRecord((("a.java::doTest", 4), ("b.java::helper", 8), ("c.java::target", None)), "a.java::doTest", True)]
def test_entry_dedupe_cycles_fan_out_and_file_identity():
graph = {"c.java::save": [("b.java::helper", 8), ("a.java::doTest", 4), ("a.java::doTest", 4)], "b.java::helper": [("c.java::save", 5)]}
kwargs = dict(
callers=lambda p, f: graph.get(f"{p}::{f}", []),
entries={"a.java::doTest"},
targets=[("c.java", "save"), ("other.java", "save"), ("a.java", "doTest"), ("c.java", "save")],
)
paths, unreached = compose_paths(**kwargs)
assert len(paths) == 2 and unreached == ["other.java::save"]
assert any(p.hops == (("a.java::doTest", None),) for p in paths)
assert compose_paths(**kwargs, fan_out=1)[1] == ["c.java::save", "other.java::save"]
@pytest.mark.parametrize("key,value", [("max_depth", -1), ("fan_out", 0)])
def test_invalid_bounds(key, value):
with pytest.raises(ValueError):
compose_paths(callers=lambda p, f: [], entries=set(), targets=[], **{key: value})
def test_records_are_consumed_by_the_checker():
from types import SimpleNamespace
from openultrasast.semantic.facts import load_facts
from openultrasast.semantic.ir import parse_file
from openultrasast.semantic.obligations.check import check_obligations
from openultrasast.semantic.obligations.dominance import OrderDominance
from openultrasast.semantic.obligations.facts import load_obligation_facts
text = "def entry(title):\n return Book.query.filter_by(book_title=title).first()\n"
paths, _ = compose_paths(callers=lambda p, f: [], entries={"app.py::entry"}, targets=[("app.py", "entry")])
result = check_obligations(
irs={"app.py": (parse_file("app.py", text, "python"), text)},
entries=[
SimpleNamespace(path="app.py", function_name="entry", access_level="unknown", kind="route", access_evidence=["fixture entry"])
],
facts=load_obligation_facts(),
flow_facts=load_facts(),
policy=None,
paths=paths,
dominance=OrderDominance({"app.py": text}),
store_shapes=(),
min_siblings=3,
assume_entries_protected=True,
)
(finding,) = result.findings
assert finding.operation.function == "entry" and finding.label == "unguarded_path"
+137
View File
@@ -0,0 +1,137 @@
"""Offline replay, resume, scoring and instrument refusal for the prover arm."""
import hashlib
import json
import socket
import subprocess
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from test_hunt_prover import repository # noqa: E402
from benchmarks.unseen import score # noqa: E402
from benchmarks.valid import prover_arm # noqa: E402
@pytest.fixture
def pool(tmp_path, monkeypatch):
def forbidden(*args, **kwargs):
pytest.fail("network access attempted")
monkeypatch.setattr(socket.socket, "connect", forbidden)
root = tmp_path / "repo"
root.mkdir()
text = """class X extends Descriptor<Thing> {
public Object doTestConnection() {
return lookupCredentials();
}
}"""
root, positive = repository(root, text)
positive.update(
id="positive",
kind="vulnerability",
family="access_control",
path="Fixture.java",
function="doTestConnection",
function_lines=[2, 4],
targets=[{"path": "Fixture.java", "function": "doTestConnection", "function_lines": [2, 4], "family": "access_control"}],
)
(root / "Fixture.java").write_text(text.replace(" return", " Jenkins.get().checkPermission(Jenkins.ADMINISTER);\n return"))
subprocess.run(["git", "-C", str(root), "add", "Fixture.java"], check=True)
subprocess.run(["git", "-C", str(root), "-c", "core.hooksPath=/dev/null", "commit", "-qm", "safe"], check=True)
head = subprocess.run(["git", "-C", str(root), "rev-parse", "HEAD"], check=True, capture_output=True, text=True).stdout.strip()
ordinary = {
**positive,
"id": "ordinary",
"kind": "ordinary",
"base": positive["head"],
"head": head,
"targets": [],
"files": [{"path": "Fixture.java", "head_lines": [[3, 3]]}],
}
changes = [positive, ordinary]
manifest = tmp_path / "frozen.json"
manifest.write_text(
json.dumps({"changes": changes, "sha256": hashlib.sha256(json.dumps(changes, sort_keys=True).encode()).hexdigest()})
)
repos = tmp_path / "repos.json"
repos.write_text(json.dumps({"fixture/app": str(root)}))
return root, manifest, repos, tmp_path / "out"
def test_run_resume_summary_and_locations(pool, monkeypatch, capsys):
root, manifest, repos, out = pool
assert prover_arm.main(["run", str(manifest), "--repos", str(repos), "--out", str(out)]) == 0
rows = score.load_records(out)
assert len(rows) == 2 and all(r["done"] and r["seconds"] > 0 for r in rows)
assert score.catches(rows[0]) == ["alerts"] and not score.false_alarm(rows[1])
summary = json.loads((out / "summary.json").read_text())
assert summary["recall"]["count"] == summary["recall"]["n"] == 1
assert summary["location_recall"]["count"] == summary["location_recall"]["n"] == 1
assert summary["false_alarms_per_100_pushes"]["rate"] == 0
assert summary["precision"]["rate"] == 1
assert summary["instrument_failures"] == 0
assert "fixture/app" in summary["per_repository"] and "stapler" in summary["per_framework"]
assert summary["changes_over_120_seconds"] == [] and summary["uncovered_repositories"] == []
assert summary["seconds_per_change"]["n"] == 2
assert "bytes read" in capsys.readouterr().out
old = {p.name: p.read_bytes() for p in out.glob("*.json") if p.name not in {"inputs.json", "summary.json"}}
def forbidden(*args, **kwargs):
pytest.fail("completed change re-run")
monkeypatch.setattr(prover_arm, "prove_change", forbidden)
assert prover_arm.run(manifest, repos, out) == 0
assert old == {p.name: p.read_bytes() for p in out.glob("*.json") if p.name not in {"inputs.json", "summary.json"}}
assert (
len(
subprocess.run(
["git", "-C", str(root), "worktree", "list", "--porcelain"], check=True, capture_output=True, text=True
).stdout.split("worktree ")
)
== 2
)
def test_zero_byte_record_refuses(pool):
_, manifest, repos, out = pool
assert prover_arm.run(manifest, repos, out) == 0
path = out / (hashlib.sha256(b"positive").hexdigest() + ".json")
record = json.loads(path.read_text())
record["instrument"]["changed_bytes"]["Fixture.java"] = 0
path.write_text(json.dumps(record))
with pytest.raises(ValueError, match="zero-byte"):
prover_arm.summarize(out)
def test_checkout_error_is_type_only(pool):
_, manifest, repos, out = pool
repos.write_text("{}")
assert prover_arm.run(manifest, repos, out) == 2
rows = score.load_records(out)
assert all(row["status"] == "instrument_failure" and row["instrument"]["failure"] == "KeyError" for row in rows)
def test_resume_rejects_new_depth(pool):
_, manifest, repos, out = pool
assert prover_arm.run(manifest, repos, out) == 0
with pytest.raises(ValueError, match="inputs"):
prover_arm.run(manifest, repos, out, max_depth=2)
def test_unchanged_file_empty_read_refuses(pool, monkeypatch):
_, manifest, repos, out = pool
real = prover_arm.Context.read
def empty(ctx, path):
if path == "Fixture.java":
return ""
return real(ctx, path)
monkeypatch.setattr(prover_arm.Context, "read", empty)
assert prover_arm.run(manifest, repos, out) == 2
with pytest.raises(ValueError, match="zero-byte"):
prover_arm.summarize(out)