pre-push(17.2): name every skipped check in plain words

One terminal line per distinct coverage reason (push/explain.py): hex-encoded
paths decoded, per-question sha256 tokens dropped and counted, unknown reasons
still printed; the JSON artifact keeps the raw values. A changed language with
no engine frontend and no quick rules is recorded and printed as "language not
covered: <lang> (...)", in pre-push and in `ousast scan`. The scan report
renders the eight degradations cli.py recorded but never showed, and any
reason without a sentence. The hook forwards OUSAST_COMPARISON_BASE.
Requirement 9.2.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 22:10:33 +02:00
co-authored by Claude Opus 5.5
parent bf07cd4d1d
commit f555dee640
11 changed files with 445 additions and 15 deletions
+2 -1
View File
@@ -630,10 +630,11 @@ Preserve all completed task evidence below.
- Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0.
- _Done 2026-10-02:_ the hook prints one line and exits 0 (chaining a prior hook if present) when `ousast` (or `OUSAST_COMMAND`) is missing; in hook mode without `--mode blocking`, argparse/usage errors and unexpected exceptions print one line and exit 0. Tests: `test_missing_ousast_never_blocks_the_push`, `test_hook_mode_usage_errors_are_advisory_unless_blocking` (blocking keeps exit 2), and `test_cli_refuses_the_flag_outside_explicit_replay` updated (the experimental flag is still refused in hook mode, now advisory). Gates: 1987 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.1, 5.3, 5.4_
- [ ] 17.2 Name every skipped check in plain words
- [x] 17.2 Name every skipped check in plain words
- One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: <lang> (<n> files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`).
- Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`.
- Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch.
- _Done 2026-10-02:_ new `push/explain.py` turns every raw coverage reason into one plain `Skipped:` line per distinct kind (hex paths decoded, sha256 tokens dropped and counted, unknown reasons still printed); the existing one-line `Notice:` keeps its next action and gains an engine-missing variant. `language_not_covered:<lang>:<changed>:<total>` is recorded for a changed language with no engine frontend and no quick rules; `ousast scan` prints `notice: language not covered: go (N files)` and its report says so. `reports.py` renders the eight previously dropped degradations and prints any reason it has no sentence for. The hook passes `OUSAST_COMPARISON_BASE`. The new-branch line names the base it lacks; what was checked instead lands with the quick tier (17.3). Tests: `tests/test_push_explain.py` (6), two in `tests/test_reports.py`; module-audit manifest regenerated for the new module. Gates: 1995 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.2, 5.2, 8.2_
- [ ] 17.3 Run the quick rules as the fast first tier
- Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block.
@@ -1,7 +1,7 @@
{
"modules": 151,
"modules": 152,
"counts": {
"load_bearing": 146,
"load_bearing": 147,
"standalone_capability": 5
},
"orphaned": [],
@@ -229,6 +229,7 @@
"model.report",
"pairs",
"plane.tasks.alerts",
"push.runner",
"rank",
"regress.candidate",
"reports",
@@ -579,6 +580,7 @@
"cli",
"cpg.backend",
"cpg.session",
"findings",
"learn.cli",
"learn.features",
"learn.labels",
@@ -716,7 +718,9 @@
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"model.scan"
"findings",
"model.scan",
"push.runner"
]
},
{
@@ -1234,6 +1238,15 @@
"push.runner"
]
},
{
"module": "push.explain",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.report",
"push.runner"
]
},
{
"module": "push.policy",
"classification": "load_bearing",
+10 -9
View File
@@ -14,15 +14,16 @@ if ! command -v "$command" >/dev/null 2>&1; then
fi
exit 0
fi
if [ -e "$prior" ] || [ -L "$prior" ]; then
exec "$command" pre-push . --artifact "$artifact_dir/push-$$.json" \
remote_name=$1
remote_url=$2
set -- pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
--prior-hook "$prior" --remote "$1" "$2"
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}"
if [ -n "${OUSAST_COMPARISON_BASE:-}" ]; then
set -- "$@" --comparison-base "$OUSAST_COMPARISON_BASE"
fi
exec "$command" pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
--remote "$1" "$2"
if [ -e "$prior" ] || [ -L "$prior" ]; then
set -- "$@" --prior-hook "$prior"
fi
exec "$command" "$@" --remote "$remote_name" "$remote_url"
+20 -1
View File
@@ -37,7 +37,7 @@ from .calibration import (
from .complexity import map as complexity_map
from .complexity.ledger import persist_verdicts
from .config import ModelLayerConfig, ObligationsConfig, ResolvedConfig, RetiredConfigError, load_config, load_dotenv
from .findings import StaticFinding, quick_scan_findings, write_findings
from .findings import StaticFinding, quick_scan_findings, uncovered_languages, write_findings
from .fusion import FusionDecision, fuse_findings
from .gate import FALSE_POSITIVE_CEILING, RECALL_FLOOR
from .harness import HarnessRuntime, HarnessTraceWriter, write_harness_config
@@ -103,6 +103,7 @@ class ScanOutcome:
finding_count: int
calibrations_applied: int
exit_code: int
notices: tuple[str, ...] = ()
if TYPE_CHECKING:
@@ -529,6 +530,17 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
"preprocess",
lambda: preprocess_repository(run.target, run.root / "preprocess" / "file_targets.json", static_hints),
)
# A language nothing here covers is named, never a silent zero (pre-push 17.2, Requirement 9.2).
for language, count in uncovered_languages(targets, ruleset, engine=mode != "quick"):
runtime.state["degradations"].append(
{
"stage": "quick_findings",
"reason": "language_not_covered",
"language": language,
"count": count,
"engine": "" if mode == "quick" else " and no engine frontend",
}
)
entry_points = runtime.run_stage("entry_point_mapping", lambda: analyze_entry_points(run.target, targets))
write_entry_points(entry_points, run.root / "mapping" / "entry_points.json")
targets = attach_reachability_hints(targets, entry_points)
@@ -827,6 +839,11 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
finding_count=len(findings),
calibrations_applied=len(applied_calibrations),
exit_code=scan_exit_code(findings, verifications, fail_on, worth_fixing_verdicts=verdict_records),
notices=tuple(
f"language not covered: {item['language']} ({item['count']} files)"
for item in runtime.state["degradations"]
if item.get("reason") == "language_not_covered"
),
)
@@ -901,6 +918,8 @@ def _print_scan_outcome(outcome: ScanOutcome) -> None:
print(f"ranked_targets={outcome.ranked_target_count}")
print(f"findings={outcome.finding_count}")
print(f"calibrations_applied={outcome.calibrations_applied}")
for notice in outcome.notices:
print(f"notice: {notice}")
def _load_static_hints(sarif_paths: tuple[str, ...]) -> list[object]:
+25
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import json
import re
from collections.abc import Sequence
from dataclasses import asdict, dataclass
from pathlib import Path
@@ -74,6 +75,30 @@ def quick_scan_findings(
return sorted(findings, key=lambda item: (_severity_sort(item.severity), -item.ranking_priority, item.path))
def uncovered_languages(
targets: Sequence[FileTarget], ruleset: Sequence[PatternRule] | None = None, *, engine: bool = False
) -> list[tuple[str, int]]:
"""Languages present in `targets` that no enabled quick rule (and, with `engine`, no engine frontend) covers.
A repository in such a language produces zero findings whatever it contains; callers name it so that
zero is never read as a clean result.
"""
from .model.partitions import FRONTENDS
rules = PATTERN_RULES if ruleset is None else ruleset
enabled = [rule for rule in rules if rule.status != "disabled"]
if any(not rule.languages for rule in enabled): # a rule without a language list applies to every file
return []
covered = {language for rule in enabled for language in rule.languages}
if engine:
covered |= set(FRONTENDS)
counts: dict[str, int] = {}
for target in targets:
if target.language not in covered and target.language != "unknown":
counts[target.language] = counts.get(target.language, 0) + 1
return sorted(counts.items())
def build_quick_hunter_prompt(target: FileTarget, source_excerpt: str) -> str:
return (
"Review this file for security-relevant issues. Return structured findings only when evidence exists.\n"
+176
View File
@@ -0,0 +1,176 @@
"""Plain-language lines for pre-push coverage reasons (Requirement 9.2).
The artifact keeps every raw reason exactly as recorded. The terminal gets one line per distinct
kind of skip, in words a first-time user can act on: hex-encoded repository paths are decoded,
per-question sha256 tokens are dropped and counted, and a reason this table does not know is
still printed rather than hidden.
"""
from __future__ import annotations
import os
import re
from collections.abc import Sequence
_HEX_PATH = re.compile(r"^((?:[0-9a-f]{2})+):(.+)$")
_TOKEN = re.compile(r"^(.+?):(?:[a-z][a-z0-9-]*:)?[0-9a-f]{64}$")
# Reasons that summarise others; printing them would only repeat a line already shown.
_SUMMARY = frozenset({"analysis_incomplete"})
_ENGINE_MISSING = (
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, "
"or run the hook through ops/ousast-docker)"
)
_NOVELTY = "whether this push introduced some engine findings could not be established ({detail})"
GLOSSARY: dict[str, str] = {
"cpg_build_failed": _ENGINE_MISSING,
"cpg_unavailable": _ENGINE_MISSING,
"cpg_empty": "engine read no code: the code graph came back empty",
"frontend_unsupported": "engine has no frontend for at least one language in this push",
"deadline_exhausted": (
"engine did not finish within the {deadline} s deadline; rerun with a longer --deadline, "
"or set OUSAST_PUSH_ENGINE=background to finish it after the push"
),
"change_context_deadline_exhausted": "the deadline ran out while relating the change to the code",
"report_deadline_exhausted": "the deadline ran out while saving the details",
"base_comparison_unavailable": "no base revision was analysed, so no engine finding can be called new or worsened",
"base_counterpart_unresolved": "the changed code has no counterpart in the base revision to compare against",
"base_unavailable": "the base revision is unavailable",
"declaration_paths_unavailable": "project declaration files could not be listed",
"ambiguous_line_correspondence": "changed lines could not be matched one-to-one to base lines in {paths}",
"line_correspondence_unavailable": "changed lines could not be mapped to the base in {paths}",
"ambiguous_path_rename_correspondence": "a renamed file could not be matched to one base file: {paths}",
"unmatched_added_deleted_path_correspondence": (
"files were both added and deleted, so a moved file cannot be told apart from a new one"
),
"capability_unavailable": ("engine findings are not qualified as alerts yet: no capability has passed independent evaluation"),
"capability_disabled": "an engine capability is declared but disabled",
"capability_unevaluated": "an engine capability has not passed its evaluation",
"capability_ambiguous": "more than one capability declaration matches a finding",
"eligibility": "the capability registry does not match this build ({detail}), so no capability can be used",
"comparison_unknown": _NOVELTY,
"dynamic_external_or_depth_context_unresolved": ("{n} engine check(s) depend on dynamic or external code the engine cannot follow"),
"query_failed": "an engine query failed, so some regions were not decided",
"query_budget_reserved": "some engine questions were not asked, to keep time for the ones that were",
"query_too_expensive": "some engine questions were too expensive to answer",
"regions_truncated": "only the highest-ranked regions were examined (--max-regions)",
"files_unparsed": "some files could not be parsed by the engine",
"cross_partition_semantics_unresolved": "flows between languages in this repository are not followed",
"resolution": "the pushed ref could not be resolved ({detail})",
"push_input_failed": "the push input could not be read ({detail})",
"snapshot": "some files could not be materialised for analysis ({detail})",
"dependency_unresolved": "{n} engine finding(s) depend on code the comparison could not resolve",
"change_unsupported": "{n} engine finding(s) have no supported relationship to the change",
}
# Comparison outcomes recorded per finding; each means novelty was not established.
_NOVELTY_REASONS = frozenset(
{
"base_incomplete",
"head_context_incomplete",
"witness_identity_unresolved",
"semantics_mismatch",
"change_context_unresolved",
"operation_correspondence_unresolved",
"base_reachability_bounded",
"base_operation_inventory_unavailable",
"base_enumeration_scope_reachability_derived",
"operation_present_in_base_without_traced_flow",
"operation_moved_within_change",
"source_correspondence_unresolved",
}
)
_RESOLUTION = {
"missing_base": (
"new branch: the remote has no base to compare against, so the engine comparison did not run "
"(OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)"
),
"unsupported_base": "the remote's current commit is not available locally, so there is no base to compare against",
"no_merge_base": "the pushed commit shares no history with the remote branch",
"ambiguous_base": "more than one base could apply to the pushed ref",
"missing_target": "the pushed commit could not be found locally",
"unsupported_target": "the pushed object is not a commit",
"resolution_error": "the pushed ref could not be resolved",
}
def _words(value: str) -> str:
return value.replace("_", " ").replace(":", ": ")
def _decode(path_hex: str) -> str | None:
try:
text = os.fsdecode(bytes.fromhex(path_hex))
except ValueError:
return None
return text if text and text.isprintable() else None
def _split(reason: str) -> tuple[str, str, str | None]:
"""(key, detail, decoded path) for one raw reason."""
path = None
match = _HEX_PATH.match(reason)
if match and (decoded := _decode(match.group(1))) is not None:
path, reason = decoded, match.group(2)
token = _TOKEN.match(reason)
if token:
reason = token.group(1)
if reason in GLOSSARY or reason in _NOVELTY_REASONS or reason in _SUMMARY:
return reason, "", path
head, _, rest = reason.partition(":")
if head not in GLOSSARY and head.endswith("_failed") and rest:
return "stage_failed", f"{head[: -len('_failed')]}:{rest}", path
return head, rest, path
def language_reason(language: str, changed: int, total: int) -> str:
"""The raw reason recorded when a language has no engine frontend and no quick rules."""
return f"language_not_covered:{language}:{changed}:{total}"
def explain(reasons: Sequence[str], *, deadline: float | None = None) -> list[str]:
"""One plain line per distinct kind of skipped or incomplete check, in first-seen order."""
groups: dict[tuple[str, str], list[str | None]] = {}
for reason in reasons:
key, detail, path = _split(str(reason))
if key in _SUMMARY:
continue
if key in _NOVELTY_REASONS:
key, detail = "comparison_unknown", _words(key)
groups.setdefault((key, detail), []).append(path)
novelty_details = [detail for (key, detail) in groups if key == "comparison_unknown" and detail]
lines: list[str] = []
for (key, detail), paths in groups.items():
if key == "comparison_unknown":
if detail or not novelty_details:
lines.append(_NOVELTY.format(detail=detail or "see the details file"))
continue
named = sorted({p for p in paths if p})
shown = ", ".join(named[:3]) + (f" and {len(named) - 3} more" if len(named) > 3 else "")
if key == "language_not_covered":
language, _, counts = detail.partition(":")
changed, _, total = counts.partition(":")
lines.append(
f"language not covered: {language} ({changed} changed file(s), {total} in the repository): "
"no engine frontend and no quick rules, so silence here is not a clean result"
)
elif key == "resolution":
lines.append(_RESOLUTION.get(detail, GLOSSARY["resolution"].format(detail=_words(detail))))
elif key == "stage_failed":
stage, _, error = detail.partition(":")
lines.append(f"the {_words(stage)} step failed ({error})")
elif key in GLOSSARY:
text = GLOSSARY[key].format(
n=len(paths),
paths=shown or "some files",
detail=_words(detail),
deadline=f"{deadline:g}" if deadline is not None else "configured",
)
lines.append(text)
else:
suffix = f" in {shown}" if shown else ""
lines.append(f"{_words(key if not detail else key + ':' + detail)}{suffix} (see the details file)")
return lines
+11
View File
@@ -18,6 +18,7 @@ from typing import Literal
from openultrasast.model.contracts import ChangeContext, ExecutionBudget
from openultrasast.model.scan import ModelScanResult
from openultrasast.push.contracts import PushComparison, PushResolution, PushResult, SnapshotManifest
from openultrasast.push.explain import explain
from openultrasast.push.policy import AdmissionResult, _context_boundaries
from openultrasast.redaction import redact_secrets
@@ -187,6 +188,10 @@ def _save(report: PushReport, target: Path, budget: ExecutionBudget) -> str | No
process.close()
# Distinct skip kinds shown in the terminal; the artifact keeps every reason.
_SKIP_LINES = 12
def _short(value: str, limit: int = 180) -> str:
# Bound before escaping: repository-controlled names/witnesses cannot add lines
# or terminal escapes. Full spelling and every witness remain in the artifact.
@@ -260,8 +265,14 @@ def render_report(report: PushReport, *, artifact: Path | None, error: str | Non
notices = []
if report.result.coverage_status in ("incomplete", "unavailable"):
reasons = report.admission.coverage_reasons
# One plain line per distinct skipped check (Requirement 9.2); the artifact keeps the raw values.
deadline = report.timings.get("deadline_seconds")
for skipped in explain(reasons, deadline=float(deadline) if deadline is not None else None)[:_SKIP_LINES]:
lines.append("Skipped: " + _short(skipped, 400))
if any("deadline" in reason for reason in reasons):
notices.append("Analysis did not finish. Rerun with a longer analysis deadline.")
elif any(reason in ("cpg_build_failed", "cpg_unavailable") for reason in reasons):
notices.append("Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.")
elif any("capability" in reason for reason in reasons):
notices.append("Some checks are not enabled for normal alerts. Review capability coverage in the details.")
else:
+27
View File
@@ -9,6 +9,7 @@ import os
import pickle
import selectors
import time
from collections import Counter
from collections.abc import Callable, Sequence
from contextlib import nullcontext
from dataclasses import asdict, replace
@@ -28,6 +29,7 @@ from openultrasast.preprocess import build_file_target, enumerate_source_files
from openultrasast.push.cache import ArtifactCache, SemanticKeys
from openultrasast.push.contracts import ComparisonAnalysis, PushComparison, PushResult, SnapshotManifest
from openultrasast.push.eligibility import fingerprint, load_registry
from openultrasast.push.explain import language_reason
from openultrasast.push.policy import (
ActionableDefect,
AdmissionCandidate,
@@ -206,6 +208,30 @@ def _record(
return report.to_payload()
def _uncovered_languages(manifest: SnapshotManifest, context: ChangeContext) -> list[str]:
"""`language_not_covered` reasons for changed languages with no engine frontend and no quick rules.
Only a language this push changes is named: a Go helper nobody touched is not this push's gap.
"""
from openultrasast.findings import PATTERN_RULES
from openultrasast.model.partitions import FRONTENDS
from openultrasast.preprocess import LANGUAGE_BY_EXTENSION
covered = set(FRONTENDS) | {language for rule in PATTERN_RULES if rule.status != "disabled" for language in rule.languages}
if any(not rule.languages for rule in PATTERN_RULES if rule.status != "disabled"):
return []
def language(path: str) -> str | None:
return LANGUAGE_BY_EXTENSION.get(os.path.splitext(path)[1].lower())
totals = Counter(language(os.fsdecode(bytes.fromhex(item.path_hex))) for item in manifest.files)
changed = Counter(language(context.decode_path(path)) for path in context.changed_paths if path not in context.deleted_paths)
return [
language_reason(name, count, totals.get(name, 0))
for name, count in sorted((k, v) for k, v in changed.items() if k is not None and k not in covered)
]
def _discovery_identity() -> str:
root = Path(__file__).resolve().parents[1]
# Discovery invokes the shipped mapper, preprocessing, layout and family specs.
@@ -347,6 +373,7 @@ def _analyze(
context = adapter.compare(comparison, declaration_paths=tuple(set(head_declarations + base_declarations)), budget=budget)
context = replace(context, unresolved_boundaries=tuple(dict.fromkeys((*context.unresolved_boundaries, *reasons))))
reasons.extend(context.unresolved_boundaries)
reasons.extend(_uncovered_languages(tip.manifest, context))
stage_started = _stage(timings, stage, stage_started)
stage = "head"
head_scan = scan_repository(
+26
View File
@@ -49,7 +49,10 @@ def write_markdown_report(
lines = ["# OpenUltraSAST Report", "", f"Findings: {len(findings)}", "", "## Inventory", ""]
if not findings:
uncovered = [item for item in degradations or () if item.get("reason") == "language_not_covered"]
lines.append("No quick-mode findings were emitted.")
for item in uncovered:
lines.append(f"Not a clean result for {item.get('language')} ({item.get('count')} files): no rules cover that language.")
lines.append("")
for finding in reasoned:
verification = verification_by_id.get(finding.finding_id)
@@ -446,6 +449,25 @@ _COVERAGE_DEGRADATIONS: dict[str, str] = {
"about the rest of the repository."
),
"region_failed": "The region `{path}` could not be arbitrated.",
# Recorded by cli.py and, until 2026-10-02, never rendered (pre-push 17.2, Requirement 9.2).
"cpg_unavailable": (
"The Joern engine is not installed here, so the model layer did not run and **no flow was followed**. "
"Install Joern (`joern-parse` on PATH) or run through the Docker image."
),
"learning_endpoint_unavailable": (
"No model endpoint is configured, so the graph's entailed findings are reported and the residual "
"`suspicion` questions went unasked."
),
"model_withheld_execution_budget": "The model was not asked, because the run's time budget leaves no room for it.",
"facts_unavailable": "The fact tables for the `{stage}` stage could not be loaded, so that stage decided nothing. {detail}",
"policy_invalid": "The `{stage}` policy could not be loaded, so that stage decided nothing. {detail}",
"max_findings_exceeded": "Only the first {requested} of {actual} findings are reported (`max_findings`).",
"hunter_model_unavailable": "The `{stage}` stage needs a model and none is available, so it was skipped.",
"sandbox_unavailable": "Docker is not available, so no finding was reproduced in the sandbox (REGRESS skipped).",
"language_not_covered": (
"language not covered: {language} ({count} files). No quick rules{engine} exist for it, so **silence about "
"these files is not a clean result**."
),
}
@@ -460,8 +482,12 @@ def _append_not_analysed(lines: list[str], degradations: Sequence[Mapping[str, o
for item in degradations:
template = _COVERAGE_DEGRADATIONS.get(str(item.get("reason") or ""))
if template is None:
# Never hide a recorded degradation because nobody wrote a sentence for it yet.
rendered.append(f"`{item.get('reason')}` (stage `{item.get('stage', 'unknown')}`)")
continue
fields = dict(item)
fields.setdefault("detail", "")
fields.setdefault("engine", "")
files = fields.get("files")
if isinstance(files, (list, tuple)):
fields["files"] = ", ".join(f"`{name}`" for name in files)
+88
View File
@@ -0,0 +1,88 @@
"""Plain-language skip lines (pre-push 17.2, Requirement 9.2): nothing hidden, nothing opaque."""
import re
from dataclasses import replace
from test_push_report import budget, report_with
from openultrasast.push.explain import explain, language_reason
from openultrasast.push.report import deliver_report
# Coverage reasons recorded by the 2026-10-02 audit's live runs, verbatim in shape.
AUDIT = (
"7265636f7264732e7079:ambiguous_line_correspondence",
"base_counterpart_unresolved",
"cpg_build_failed",
"frontend_unsupported",
"analysis_incomplete",
"base_comparison_unavailable",
"eligibility:eligibility_provenance_mismatch",
*(f"dynamic_external_or_depth_context_unresolved:contributor-scan:{i:064x}" for i in range(20)),
"resolution:missing_base",
"admission_failed:TimeoutError",
"base_incomplete",
"comparison_unknown",
)
def test_audit_reasons_become_one_plain_line_each():
lines = explain(AUDIT, deadline=30)
text = "\n".join(lines)
assert "records.py" in text and "7265636f7264732e7079" not in text
assert not re.search(r"[0-9a-f]{64}", text)
assert "20 engine check(s) depend on dynamic or external code" in text
assert "Joern" in text and "new branch: the remote has no base" in text
assert "the admission step failed (TimeoutError)" in text
assert "analysis_incomplete" not in text and "analysis incomplete" not in text
# One line per distinct kind: twenty hashed reasons are one line, comparison_unknown folds into its cause.
assert len(lines) == 10
assert sum("could not be established" in line for line in lines) == 1
def test_unknown_reason_is_still_printed():
assert explain(("some_future_reason",)) == ["some future reason (see the details file)"]
def test_deadline_line_names_the_deadline_and_the_background_option():
(line,) = explain(("deadline_exhausted",), deadline=30)
assert "30 s" in line and "background" in line
def test_language_not_covered_is_named_with_counts():
(line,) = explain((language_reason("go", 1, 12),))
assert line.startswith("language not covered: go (1 changed file(s), 12 in the repository)")
def test_terminal_shows_skip_lines_while_artifact_keeps_raw_reasons(tmp_path):
import json
report = report_with(0)
raw = ("7265636f7264732e7079:ambiguous_line_correspondence", f"dynamic_external_or_depth_context_unresolved:contributor-scan:{1:064x}")
report = replace(
report,
admission=replace(report.admission, coverage_reasons=raw),
result=replace(report.result, coverage_status="incomplete"),
)
delivered = deliver_report(report, tmp_path / "run.json", execution_budget=budget())
assert "Skipped: changed lines could not be matched one-to-one to base lines in records.py" in delivered.text
assert "7265636f" not in delivered.text and f"{1:064x}" not in delivered.text
assert delivered.text.count("Notice:") == 1
assert tuple(json.loads(delivered.artifact.read_text())["admission"]["coverage_reasons"]) == raw
def test_replay_names_a_changed_language_nothing_covers(tmp_path):
import json
from test_push_runner import history
from openultrasast.cpg.backend import NullBackend
from openultrasast.push.runner import replay
root, base, _, git = history(tmp_path)
(root / "run.go").write_text("package main\n")
git("add", "run.go")
git("commit", "-m", "go")
head = git("rev-parse", "HEAD")
delivery = replay(root, base=base, head=head, artifact=tmp_path / "r.json", backend=NullBackend())
assert "Skipped: language not covered: go (1 changed file(s), 1 in the repository)" in delivery.text
assert "language_not_covered:go:1:1" in json.loads((tmp_path / "r.json").read_text())["admission"]["coverage_reasons"]
+43
View File
@@ -654,3 +654,46 @@ def test_a_clean_scan_states_no_coverage_gaps(tmp_path) -> None: # type: ignore
degradations=[],
)
assert "### What could not be analysed" not in path.read_text()
def test_every_recorded_scan_degradation_is_rendered(tmp_path) -> None: # type: ignore[no-untyped-def]
"""pre-push 17.2 / Req 9.2: the eight degradations cli.py records and the report used to drop."""
from openultrasast.reports import write_markdown_report
reasons = [
{"stage": "model", "reason": "learning_endpoint_unavailable"},
{"stage": "model", "reason": "cpg_unavailable"},
{"stage": "overlay", "reason": "facts_unavailable", "detail": "missing table"},
{"stage": "obligations", "reason": "facts_unavailable"},
{"stage": "obligations", "reason": "policy_invalid", "detail": "bad toml"},
{"stage": "model", "reason": "model_withheld_execution_budget"},
{"stage": "budget", "reason": "max_findings_exceeded", "requested": 5, "actual": 9},
{"stage": "map", "reason": "hunter_model_unavailable"},
{"stage": "regress", "reason": "sandbox_unavailable"},
{"stage": "x", "reason": "not_yet_described"},
]
path = tmp_path / "report.md"
write_markdown_report([], path, [], degradations=reasons)
text = path.read_text()
for needle in (
"No model endpoint is configured",
"Joern engine is not installed",
"fact tables for the `overlay` stage",
"fact tables for the `obligations` stage",
"`obligations` policy could not be loaded",
"time budget leaves no room",
"first 5 of 9 findings",
"needs a model and none is available",
"Docker is not available",
"`not_yet_described`",
):
assert needle in text, needle
def test_quick_scan_names_a_language_nothing_covers(tmp_path, capsys) -> None: # type: ignore[no-untyped-def]
"""pre-push 17.2 / Req 9.2: a Go-only tree is not a silent zero."""
(tmp_path / "run.go").write_text('package main\n\nimport "os/exec"\n\nfunc run(c string) { exec.Command("sh", "-c", c).Run() }\n')
assert main(["scan", str(tmp_path), "--mode", "quick"]) == 0
assert "notice: language not covered: go (1 files)" in capsys.readouterr().out
report = next((tmp_path / ".openultrasast" / "runs").iterdir()) / "report.md"
assert "Not a clean result for go (1 files)" in report.read_text()